Article
Automation & Control Systems
Jeongho Kim, Insoon Yang
Summary: Maximum entropy reinforcement learning methods have been successfully applied to a range of challenging sequential decision-making and control tasks. However, there is a need to extend these methods to continuous-time systems. This article studies the theory of maximum entropy optimal control in continuous time and derives a novel class of equations. The results demonstrate the performance of the maximum entropy method in continuous-time optimal control and reinforcement learning problems.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL
(2023)
Article
Mathematics
Kaizhi Wang, Lin Wang, Jun Yan
Summary: The paper provides necessary and sufficient conditions for the existence of viscosity solutions of nonlinear first order PDEs, proving compactness of the set of solutions. Furthermore, it explores the long-term behavior of viscosity solutions for Cauchy problems using weak KAM theory and dynamic methods.
JOURNAL OF DIFFERENTIAL EQUATIONS
(2021)
Article
Mathematics, Applied
Kaizhi Wang, Jun Yan, Kai Zhao
Summary: This article discusses the existence and multiplicity of nontrivial time periodic viscosity solutions to a contact Hamilton-Jacobi equation, and investigates the long time behavior of these solutions. It is found that for a certain class of initial data, the corresponding viscosity solutions converge to asymptotic time periodic viscosity solutions. The article also analyzes a bifurcation phenomenon for a parameter-dependent Hamilton-Jacobi equation.
JOURNAL DE MATHEMATIQUES PURES ET APPLIQUEES
(2023)
Article
Automation & Control Systems
Jun Moon
Summary: In this article, a generalized risk-sensitive optimal control problem is studied with the objective functional defined by a controlled BSDE. The dynamic programming principle and viscosity solution for the value function are obtained, with applications to the risk-sensitive European option pricing problem.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL
(2021)
Article
Mathematics, Applied
Yang Xu, Jun Yan, Kai Zhao
Summary: This paper studies the relationship between the stability of viscosity solutions and the set structure of weak KAM solutions to the contact Hamilton-Jacobi equation.
PROCEEDINGS OF THE AMERICAN MATHEMATICAL SOCIETY
(2023)
Article
Mathematics, Applied
Panrui Ni
Summary: This paper discusses the convergence of viscosity solutions for the generalized discounted Hamilton-Jacobi equation and provides two examples where the solutions do not converge as lambda tends to zero.
PROCEEDINGS OF THE AMERICAN MATHEMATICAL SOCIETY
(2023)
Article
Automation & Control Systems
Jianjun Zhou
Summary: This article introduces the concept of viscosity solutions for first-order path-dependent Hamilton-Jacobi-Bellman (PHJB) equations associated with optimal control problems for path-dependent evolution equations in Hilbert space. We identify the value functional of optimal control problems as the unique viscosity solution to the associated PHJB equations without a specific assumption. We also demonstrate that our notion of viscosity solutions is consistent with classical solutions and exhibits a stability property.
Article
Mathematics, Applied
Xiaofan Bao, Shanjian Tang
Summary: We study the ergodic control problem for McKean-Vlasov stochastic differential equations and establish the existence and uniqueness of the viscosity solution to the associated fully nonlinear HJB equation in a lifted sense. Moreover, we demonstrate the convergence of the solutions of finite-horizon time-averaging optimal control problems to that of the ergodic control problem as the time horizon tends to infinity. Our results rely on dissipativity conditions and dissipativity-like conditions on the distribution variables of both drift and diffusion coefficients. (c) 2023 Elsevier Inc. All rights reserved.
JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS
(2023)
Article
Mathematics
M. I. Gomoyunov, A. R. Plaksin
Summary: The paper discusses a path-dependent Hamilton-Jacobi equation with coinvariant derivatives over the space of continuous functions. It studies generalized solutions of the equation in both the minimax and viscosity senses, and proves the equivalence between these two notions. The paper also obtains comparison and uniqueness results for viscosity solutions of a Cauchy problem with a right-end boundary condition.
JOURNAL OF FUNCTIONAL ANALYSIS
(2023)
Article
Mathematics, Applied
Phan Trong Tien, Tran Van Bang
Summary: This paper focuses on optimal control problems on junctions using the viscosity solution approach. Compared to previous studies, the paper works on a less restrictive set of assumptions and shows that the value function is a unique viscosity solution of an associated Hamilton-Jacobi equation. The viscosity solution method is used to establish a necessary and sufficient condition for optimal control in a class of optimal control problems.
APPLICABLE ANALYSIS
(2021)
Article
Automation & Control Systems
Anton Plaksin
Summary: This paper addresses a zero-sum differential game for a dynamical system described by a nonlinear delay differential equation under a initial condition defined by a piecewise continuous function. It derives the corresponding Cauchy problem for Hamilton-Jacobi-Bellman-Isaacs equation with coinvariant derivatives, and considers the definition of a viscosity solution for this problem. It proves that the differential game has a unique viscosity solution value, and obtains an infinitesimal description of the viscosity solution based on notions of sub- and superdifferentials corresponding to coinvariant derivatives.
SIAM JOURNAL ON CONTROL AND OPTIMIZATION
(2021)
Article
Automation & Control Systems
Arash Komaee
Summary: The paper introduces an inverse optimal feedback design approach, exploring analytical solutions for the HJB equation by selecting cost functionals with a specific structural constraint. This method effectively addresses certain feedback design problems.
OPTIMAL CONTROL APPLICATIONS & METHODS
(2021)
Article
Mathematics, Applied
Kaizhi Wang, Jun Yan, Kai Zhao
Summary: This paper deals with the long-time behavior of viscosity solutions of evolutionary contact Hamilton-Jacobi equations. It shows the connection between viscosity solutions of the ergodic contact Hamilton-Jacobi equation and solutions of the evolutionary equation.
PROCEEDINGS OF THE AMERICAN MATHEMATICAL SOCIETY
(2022)
Article
Mathematics, Applied
Jun Moon
Summary: In this paper, we investigate the state-constrained stochastic optimal control problem in infinite-dimensional separable Hilbert spaces. By utilizing stochastic target theory and backward reachability approach, we establish the representation of the original value function and prove the uniqueness of the auxiliary value function as a viscosity solution to the associated HJB equation.
NONLINEAR ANALYSIS-THEORY METHODS & APPLICATIONS
(2022)
Article
Computer Science, Interdisciplinary Applications
Michael Klibanov, Loc H. Nguyen, Hung Tran
Summary: This article proposes a globally convergent numerical method, called the convexification, to compute the viscosity solution to first-order Hamilton-Jacobi equations through the vanishing viscosity process. The method employs a suitable Carleman weight function to convexify the cost functional defined directly from the form of the Hamilton-Jacobi equation and utilizes the gradient descent method to find the unique minimizer of this convex functional.
JOURNAL OF COMPUTATIONAL PHYSICS
(2022)
