Article
Computer Science, Artificial Intelligence
Xiong Yang, Yuanheng Zhu, Na Dong, Qinglai Wei
Summary: The study focuses on the decentralized event-driven control problem of nonlinear dynamical systems, transforming it into a group of nonlinear optimal control problems and solving it using event-driven Hamilton-Jacobi-Bellman equations to ensure overall system stability. The critic neural network architecture is utilized to address the problem, with Lyapunov method ensuring signal stability in closed-loop subsystems.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2022)
Article
Automation & Control Systems
Mingming Ha, Ding Wang, Derong Liu
Summary: In this paper, a new approach is proposed to address the tracking control problem. By introducing a new cost function and a novel stability analysis method, the issue of incomplete elimination of tracking error in traditional approaches is solved. The specific implementation scheme for the special case of linear systems is also provided.
IEEE-CAA JOURNAL OF AUTOMATICA SINICA
(2022)
Article
Automation & Control Systems
Chuanhao Hu, Yuanyuan Zou, Shaoyuan Li
Summary: This paper presents an ADP-based decentralized event-triggered control strategy for large-scale nonlinear systems, utilizing event-triggered scheme and local neural networks observer to reduce communication cost and computational burden, with a decentralized triggering condition designed to ensure overall stabilization of the large-scale systems.
ASIAN JOURNAL OF CONTROL
(2022)
Article
Automation & Control Systems
Antonio Sala, Leopoldo Armesto
Summary: This study introduces a new criterion for adaptive meshing in polyhedral partitions to interpolate value functions, employing an initial condition probability density function, uncertainty propagation, and temporal-difference error to determine the addition of new points. A collection of lemmas justifies the algorithmic proposal, with comparative analysis highlighting the advantages of this proposal over other options in literature. The developed methods are applied in simulation examples and an experimental robotic setup.
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE
(2022)
Article
Computer Science, Artificial Intelligence
Linghuan Kong, Wei He, Chenguang Yang, Changyin Sun
Summary: This paper aims to improve the robustness of robot tracking control under unknown nonlinear perturbations by introducing an auxiliary system and utilizing approximate optimal control provided by neural networks, relaxing the requirement of initial stabilizing control. The solution of the Hamilton-Jacobi-Isaacs equation is approximated under the framework of adaptive dynamic programming, and training of neural networks is conducted based on the designed updating law. The Lyapunov stability theory is used to prove all error signals to be uniformly ultimately bounded, and simulation studies are carried out to demonstrate the effectiveness of the proposed control method.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2021)
Article
Automation & Control Systems
Xiong Yang, Mengmeng Xu, Qinglai Wei
Summary: We study the dynamic event-driven Hop constrained control problem through approximate dynamic programming (ADP). Differing from the existing literature considering systems with either symmetric constraints or asymmetric constraints, we consider the two different constraints simultaneously. Initially, by constructing a generalized nonquadratic value function, we transform the H-8 constrained control problem into an unconstrained two-player zero-sum game. Then, we present an event-driven Hamilton-Jacobi-Isaacs equation (ED-HJIE) corresponding to the zero-sum game for lowering down the computational load. To solve the ED-HJIE, we propose a dynamic triggering mechanism together with a sole critic neural network (CNN) being built under the ADP framework. The CNN's weights are tuned via the gradient descent approach. After that, we prove uniform ultimate boundedness of the closed-loop system and the CNN's weight estimation error via Lyapunov's method. Finally, we separately use an F16 aircraft plant and an inverted pendulum system to validate the present theoretical claims.
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
(2023)
Article
Computer Science, Artificial Intelligence
Linghuan Kong, Shuang Zhang, Xinbo Yu
Summary: An approximate optimal scheme is proposed for an uncertain n-link robot subject to saturation nonlinearity. The proposed method takes into account model uncertainty in robotic dynamics and designs an optimal control under the frame of adaptive dynamic programming. The method is proved to be effective in stabilizing the unknown system and reducing control cost.
Article
Automation & Control Systems
Mingming Liang, Derong Liu
Summary: This article focuses on designing the optimal impulsive controller (IMC) of continuous-time nonlinear systems and proposes a new adaptive dynamic programming algorithm with high generality and feasibility. The introduced policy-improving mechanism makes the algorithm more flexible for memory-limited computing devices.
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
(2023)
Article
Mathematics, Applied
Zhongyang Ming, Huaguang Zhang, Yuling Liang, Hanguang Su
Summary: In this paper, a single network adaptive dynamic programming (ADP) control method is proposed for the non-zero sum (NZS) differential game problem of the autonomous nonlinear system. The Osgood condition is introduced to ensure the existence and uniqueness of the solution of the dynamic nonlinear systems and to weaken the limited conditions of nonlinear dynamic functions. The proposed method achieves real-time approximations of the optimal value and the non-zero sum Nash-equilibrium, while ensuring the uniform ultimate epsilon-boundedness of the closed-loop system. The effectiveness of the method is verified through a simulation example.
APPLIED MATHEMATICS AND COMPUTATION
(2022)
Article
Computer Science, Artificial Intelligence
Ziyu Lin, Jingliang Duan, Shengbo Eben Li, Haitong Ma, Jie Li, Jianyu Chen, Bo Cheng, Jun Ma
Summary: The research addresses the challenge of solving the finite-horizon HJB equation, proposes a new algorithm, and validates its effectiveness through simulations.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Automation & Control Systems
Derong Liu, Shan Xue, Bo Zhao, Biao Luo, Qinglai Wei
Summary: This article reviews the recent development of adaptive dynamic programming (ADP) with applications in control, highlighting efficient algorithms and future research directions. ADP is applied in optimization, game theory, and large-scale systems, showing great potential in the era of artificial intelligence.
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
(2021)
Article
Automation & Control Systems
Yongwei Zhang, Bo Zhao, Derong Liu, Shunchao Zhang
Summary: In this article, the event-triggered robust control problem of unknown multiplayer nonlinear systems with constrained inputs and uncertainties is investigated using adaptive dynamic programming. A neural network-based identifier is constructed to relax the requirement of system dynamics. By designing a nonquadratic value function, the stabilization problem is converted into a constrained optimal control problem. The approximate solution of the event-triggered Hamilton-Jacobi equation is obtained using a critic network with a novel weight updating law, and the Lyapunov stability theorem ensures that the multiplayer system is uniformly ultimately bounded.
IEEE TRANSACTIONS ON CYBERNETICS
(2023)
Article
Automation & Control Systems
Mingming Ha, Ding Wang, Derong Liu
Summary: Inspired by the successive relaxation method, a novel discounted iterative adaptive dynamic programming framework is developed, which possesses an adjustable convergence rate for the iterative value function sequence. The convergence properties of the value function sequence and the stability of the closed-loop systems under the new discounted value iteration (VI) are investigated. An accelerated learning algorithm with convergence guarantee is presented based on the properties of the given VI scheme. Amidst the implementation of the new VI scheme and its accelerated learning design, value function approximation and policy improvement are involved. The performance of the developed approaches is verified using a nonlinear fourth-order ball-and-beam balancing plant, showing significant acceleration of the convergence rate of the value function and reduction in computational cost compared to traditional VI.
IEEE TRANSACTIONS ON CYBERNETICS
(2023)
Article
Computer Science, Artificial Intelligence
Chaoxu Mu, Ke Wang, Zhen Ni
Summary: This study investigates control problems based on static event triggering in the implementation of adaptive dynamic programming algorithms, proposing improvements in dynamic event triggering and providing mathematical proofs for system stability and weight convergence. Theoretical analysis reveals the characteristics of dynamic rule and its relations with static rules; a numerical example confirms the established claims.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2022)
Article
Automation & Control Systems
Yongwei Zhang, Bo Zhao, Derong Liu, Shunchao Zhang
Summary: This article investigates the event-triggered optimal tracking control problem for multiplayer unknown nonlinear systems using adaptive critic designs. It proposes a novel weight updating rule and derives the optimal tracking control policy for each player by solving coupled event-triggered Hamilton-Jacobi equations. Additionally, an event-triggering condition is designed to ensure the uniform ultimate boundedness of the closed-loop multiplayer systems.
INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL
(2022)
Article
Computer Science, Artificial Intelligence
Mingming Ha, Ding Wang, Derong Liu
Summary: In this article, a novel value iteration scheme is proposed, which introduces a relaxation factor and combines with other methods to accelerate and guarantee the convergence. The theoretical results and numerical examples demonstrate its fast convergence speed and stability.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Computer Science, Artificial Intelligence
Mingduo Lin, Bo Zhao, Derong Liu
Summary: A novel policy gradient (PG) adaptive dynamic programming method is proposed for nonlinear discrete-time zero-sum games with unknown dynamics. A policy iteration algorithm is used to approximate the Q-function and the control and disturbance policies using neural network approximators. The control and disturbance policies are then updated using the PG method based on the iterative Q-function. The experience replay technique is applied to improve training stability and data usage efficiency. Simulation results show the effectiveness of the proposed method.
Article
Computer Science, Artificial Intelligence
Mingming Liang, Derong Liu
Summary: This article presents a novel neural-network-based optimal event-triggered impulsive control method. The proposed method utilizes a general-event-based impulsive transition matrix (GITM) to represent the evolving characteristics of all system states across impulsive actions. Through the developed event-triggered impulsive adaptive dynamic programming (ETIADP) algorithm and its high-efficiency version (HEIADP), the optimization problems for stochastic systems with event-triggered impulsive controls are addressed. The results show that the proposed methods can reduce computational and communication burdens and fulfill the desired goals.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Automation & Control Systems
Mingming Ha, Ding Wang, Derong Liu
Summary: Inspired by the successive relaxation method, a novel discounted iterative adaptive dynamic programming framework is developed, which possesses an adjustable convergence rate for the iterative value function sequence. The convergence properties of the value function sequence and the stability of the closed-loop systems under the new discounted value iteration (VI) are investigated. An accelerated learning algorithm with convergence guarantee is presented based on the properties of the given VI scheme. Amidst the implementation of the new VI scheme and its accelerated learning design, value function approximation and policy improvement are involved. The performance of the developed approaches is verified using a nonlinear fourth-order ball-and-beam balancing plant, showing significant acceleration of the convergence rate of the value function and reduction in computational cost compared to traditional VI.
IEEE TRANSACTIONS ON CYBERNETICS
(2023)
Article
Automation & Control Systems
Runqi Chai, Derong Liu, Tianhao Liu, Antonios Tsourdos, Yuanqing Xia, Senchun Chai
Summary: This paper presents an integrated real-time trajectory planning and tracking control framework for autonomous ground vehicles (AGV) parking maneuver problems, utilizing deep neural networks and recurrent network structures. Two transfer learning strategies are applied to adapt the motion planner for different AGV types. Experimental studies demonstrate enhanced performance in fulfilling parking missions.
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING
(2023)
Article
Computer Science, Artificial Intelligence
Danyu Lin, Shan Xue, Derong Liu, Mingming Liang, Yonghua Wang
Summary: In this paper, a problem of multiplayer hierarchical decision-making for non-affine systems is solved using adaptive dynamic programming. The control dynamics are obtained and combined with the original system dynamics, transforming the non-affine multiplayer system into a general affine form. The hierarchical decision problem is modeled as a Stackelberg game, and a neural network is used to reconstruct the augmented system and approximate the value function. The feasibility and effectiveness of the algorithm are confirmed through simulation.
Article
Automation & Control Systems
Mingming Liang, Yonghua Wang, Derong Liu
Summary: In this study, a novel general impulsive transition matrix is defined to reveal the transition dynamics and probability distribution evolution patterns between impulsive events. Based on this matrix, policy iteration-based impulsive adaptive dynamic programming algorithms are developed to solve optimal impulsive control problems. The algorithms demonstrate convergence to the optimal impulsive performance index function and allow for optimization on computing devices with low memory spaces.
IEEE TRANSACTIONS ON CYBERNETICS
(2023)
Proceedings Paper
Automation & Control Systems
Jinquan Lin, Bo Zhao, Derong Liu
Summary: In this paper, an integral reinforcement learning (IRL)-based approximate optimal control (AOC) method is developed for unknown nonaffine systems using dynamic feedback. The optimal control policy for nonaffine systems cannot be explicitly expressed due to the unknown input gain matrix. Thus, a dynamic feedback signal is introduced to transform the nonaffine system into an augmented affine system. The AOC for unknown nonaffine systems is formulated by designing an appropriate value function for the augmented affine system, and the IRL method is adopted to derive the approximate solution of the Hamilton-Jacobi-Bellman equation.
2023 IEEE 12TH DATA DRIVEN CONTROL AND LEARNING SYSTEMS CONFERENCE, DDCLS
(2023)
Article
Automation & Control Systems
Bo Zhao, Guang Shi, Derong Liu
Summary: This article investigates local control problems for nonlinear interconnected systems by using adaptive dynamic programming (ADP) with particle swarm optimization (PSO). It constructs a proper local value function and employs a local critic neural network to solve the local Hamilton-Jacobi-Bellman equation. The event-triggering mechanism is introduced to determine the sampling time instants and ensure asymptotic stability through Lyapunov stability analysis.
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
(2023)
Article
Automation & Control Systems
Hui Wang, Zhigang Liu, Zhiwei Han, Yanbo Wu, Derong Liu
Summary: Active pantograph control is a promising technique for improving train's current collection quality. Existing solutions have limitations in handling various operating conditions and lack of adaptability. In this study, a context-based deep meta-reinforcement learning algorithm is proposed to alleviate these problems. Experimental results show that the proposed algorithm can quickly adapt to new conditions and reduce contact force fluctuations.
IEEE TRANSACTIONS ON CYBERNETICS
(2023)
Article
Computer Science, Artificial Intelligence
Zhanyu Yang, Bo Zhao, Derong Liu
Summary: In this article, a novel pinning control method that requires only partial node information is developed to synchronize drive-response memristor-based neural networks with time delay. An improved mathematical model of the networks is established to accurately describe their dynamic behaviors. Unlike previous literature that requires information from all nodes, the proposed method only relies on local information to achieve synchronization of delayed networks, reducing communication and calculation burdens. Sufficient conditions for synchronization are provided, and numerical simulation and comparative experiments validate the effectiveness and superiority of the proposed method.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Automation & Control Systems
Mingduo Lin, Bo Zhao, Derong Liu
Summary: In this article, an event-triggered robust adaptive dynamic programming (ETRADP) algorithm is proposed to solve multiplayer Stackelberg-Nash games (MSNGs) for uncertain nonlinear continuous-time systems. The hierarchical decision-making process considering different roles of players is described, transforming the robust control problem into an optimal regulation problem. An online policy iteration algorithm is used to solve the derived Hamilton-Jacobi equation with an event-triggered mechanism to reduce computational and communication burdens. Critic neural networks (NNs) are constructed to obtain the event-triggered approximate optimal control policies for all players, ensuring the stability of the closed-loop system.
IEEE TRANSACTIONS ON CYBERNETICS
(2023)
Article
Automation & Control Systems
Mingming Liang, Derong Liu
Summary: This article focuses on designing the optimal impulsive controller (IMC) of continuous-time nonlinear systems and proposes a new adaptive dynamic programming algorithm with high generality and feasibility. The introduced policy-improving mechanism makes the algorithm more flexible for memory-limited computing devices.
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
(2023)
Article
Automation & Control Systems
Ke Wang, Chaoxu Mu, Zhen Ni, Derong Liu
Summary: This paper presents a novel composite obstacle avoidance control method that generates safe motion trajectories for autonomous systems in an adaptive manner. The method combines model-based policy iteration and state-following-based approximation in a safe reinforcement learning framework. The proposed learning-based controller achieves stable reaching of target points while maintaining a safe distance from obstacles. The effectiveness of the method is demonstrated through simulations and comparisons with other avoidance control methods.
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING
(2023)
Article
Computer Science, Artificial Intelligence
Bo Zhao, Yongwei Zhang, Derong Liu
Summary: This article presents a cooperative motion/force control scheme for modular reconfigurable manipulators (MRMs) based on adaptive dynamic programming (ADP). The dynamic model of the entire MRM system is treated as a set of joint modules interconnected by coupling torque, and the Jacobian matrix is mapped into each joint. A neural network is used as a robust decentralized observer, and an improved local value function is constructed for each joint module. The control scheme is achieved by using force feedback compensation and is proven to be uniformly ultimately bounded through Lyapunov stability analysis.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
