Article
Mathematics, Applied
Zhongyang Ming, Huaguang Zhang, Yuling Liang, Hanguang Su
Summary: In this paper, a single network adaptive dynamic programming (ADP) control method is proposed for the non-zero sum (NZS) differential game problem of the autonomous nonlinear system. The Osgood condition is introduced to ensure the existence and uniqueness of the solution of the dynamic nonlinear systems and to weaken the limited conditions of nonlinear dynamic functions. The proposed method achieves real-time approximations of the optimal value and the non-zero sum Nash-equilibrium, while ensuring the uniform ultimate epsilon-boundedness of the closed-loop system. The effectiveness of the method is verified through a simulation example.
APPLIED MATHEMATICS AND COMPUTATION
(2022)
Article
Automation & Control Systems
Xiong Yang, Mengmeng Xu, Qinglai Wei
Summary: This article proposes a simultaneous policy iteration (SPI) algorithm to solve the H-infinity control problem of nonlinear systems with unavailable dynamics and asymmetric saturating actuators. The SPI algorithm converts the control problem into a zero-sum game and solves the corresponding Hamilton-Jacobi-Isaacs equation. A critic, an actor, and a perturbation neural network (NN) are constructed to estimate the cost function, control policy, and perturbation, respectively. The SPI algorithm allows arbitrary control policies and perturbations in the learning process and does not require the persistence of the excitation condition.
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
(2023)
Article
Automation & Control Systems
Xiumei Han, Haiqin Qin, Zhitao Wang, Ning Xu, Xudong Zhao, Jinfeng Zhao
Summary: This paper studies the optimal event-triggered control for constrained-input discrete-time switched nonlinear systems and proposes an algorithm based on triggered states. Neural networks are used to approximate the control input and costate vector functions of each subsystem. A trigger condition is designed to ensure the asymptotic stability of the switched system.
INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL
(2021)
Article
Automation & Control Systems
Fanghua Tang, Huanqing Wang, Xiao-Heng Chang, Liang Zhang, Khalid H. Alharbi
Summary: This paper investigates a dynamic event-triggered optimal control problem of discrete-time nonlinear Markov jump systems via policy iteration adaptive dynamic programming algorithms. The performance index function (PIF) defined in each subsystem is updated using an online PI algorithm, and the control policy is derived by solving the optimal PIF. Neural network techniques are used to estimate the iterative PIF and control policy. The designed dynamic event-triggered mechanism (DETM) is employed to avoid wasting additional resources. The developed control scheme guarantees system stability and convergence of all signals, as proven using the Lyapunov difference method. A simulation example is presented to demonstrate the feasibility of the control design scheme.
NONLINEAR ANALYSIS-HYBRID SYSTEMS
(2023)
Article
Automation & Control Systems
Jingliang Sun, Teng Long
Summary: This paper investigates an adaptive event-triggered distributed iterative differential game strategy for multi-agent systems, approximating the solution of coupled HJI equation with a critic neural network and designing a novel PE-free updating law. The developed strategy ensures the uniformly ultimately bounded of all closed-loop signals and avoids the Zeno behavior. The simulation results show a significant reduction in controller updates, saving computational and communication resources.
Article
Mathematics
Wenrui Yang, Yang Gu, Xia Xie, Chengze Jiang, Zhiyuan Song, Yudong Zhang
Summary: In this paper, a bounded adaptive function activated recurrent neural network (BAFARNN) is proposed to solve the dynamic QR factorization problem with faster convergence speed and enhanced robustness compared to existing solutions.
Article
Computer Science, Artificial Intelligence
Shan Xue, Biao Luo, Derong Liu
Summary: In this article, an event-triggered adaptive dynamic programming (ADP) method is proposed to solve the robust control problem of unmatched uncertain systems. By introducing an event-trigger mechanism, concurrent learning, and utilizing a critic neural network (NN) to approximate the value function, this method avoids the requirement of an initial admissible control and excitation condition. Simulation results using examples of F-16 aircraft and inverted pendulum show the effectiveness of the developed event-triggered ADP method in guaranteeing robustness and boundedness of the system.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2021)
Article
Engineering, Mechanical
Guoping Zhang, Quanxin Zhu
Summary: This paper investigates the event-triggered optimal control (ETOC) for nonlinear Ito-type stochastic systems using the adaptive dynamic programming (ADP) approach. The value function of the Hamilton-Jacobi-Bellman (HJB) equation is approximated using critical neural network (CNN), and a new event-triggering scheme is proposed. The Lyapunov direct method is used to prove that the ETOC based on ADP approach guarantees that the CNN weight errors and system states are semi-globally uniformly ultimately bounded in probability.
NONLINEAR DYNAMICS
(2021)
Article
Computer Science, Artificial Intelligence
Yang Yang, Xin Fan, Weinan Gao, Wenbin Yue, Aaron Liu, Shuocong Geng, Jinran Wu
Summary: An event-triggered output feedback control approach is proposed in this paper by utilizing a disturbance observer and adaptive dynamic programming (ADP). The use of a disturbance observer and state observer helps achieve output-feedback ADP control. The approach is validated through simulation examples, and an event-triggered mechanism is introduced to reduce the communication burden.
IEEE TRANSACTIONS ON FUZZY SYSTEMS
(2023)
Article
Engineering, Mechanical
Xiang Liu, Yang Wu, Nailong Wu, Huaicheng Yan, Yueying Wang
Summary: This article addresses the issues of unmeasurable states and actuator hysteresis in multi-input multi-output nonstrict-feedback nonlinear systems. It proposes a neural network observer to estimate the unmeasurable states and uses radial basis function neural networks to approximate the nonlinear terms. A variable separation technique is employed to solve the algebraic loop problem and a command filter design technique reduces the complexity. A finite-time performance function is implemented to ensure the tracking error enters a preassigned range within a specified time. The effectiveness of the developed controller is demonstrated through simulation examples.
NONLINEAR DYNAMICS
(2023)
Article
Automation & Control Systems
Shaojie Zhang, Kun Ji, Han Zhang
Summary: This study proposes a model-free incremental adaptive fault-tolerant control scheme for nonlinear systems with actuator faults. To overcome actuator faults and ensure optimal performance of the nominal nonlinear system without prior knowledge of system dynamics, a single-network incremental adaptive dynamic programming algorithm based on incremental neural network observer is developed to design an active fault-tolerant control policy. An approximate linear time-varying system is obtained using incremental nonlinear technique, with relevant matrix parameters identified by recursive least square estimation. A fault-tolerant controller based on the SIADP algorithm is developed. The proposed scheme incorporates a simplified single critic neural network to reduce learning time and computational burden, with the weight estimations of critic neural network updated.
INTERNATIONAL JOURNAL OF ADAPTIVE CONTROL AND SIGNAL PROCESSING
(2023)
Article
Computer Science, Artificial Intelligence
Chujian Zeng, Bo Zhao, Derong Liu
Summary: This paper proposes a neuro-dynamic programming-based fault tolerant control scheme for a class of nonlinear systems, considering the occurrence of both actuator and sensor faults simultaneously. The scheme combines a descriptor observer with an adaptive observer to estimate system states and multiple faults. By employing a critic neural network, the approximate optimal control policy is obtained for the fault-free system. An FTC law is developed to suppress the influence of actuator faults by combining the estimations of actuator faults with the approximate optimal control policy. The stability of the closed-loop nonlinear system is analyzed using the Lyapunov stability theorem.
Article
Computer Science, Artificial Intelligence
Kewen Li, Yongming Li
Summary: This article introduces a finite-time neural network adaptive dynamic surface control design for single-input single-output nonlinear systems. The control algorithm utilizes a novel nonlinear filter and finite-time Lyapunov stable theory to ensure tracking error convergence within a small neighborhood of origin in finite time. Simulation examples demonstrate the superiority and effectiveness of the proposed control algorithm.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2021)
Article
Computer Science, Artificial Intelligence
Xiang Liu, Yiqi Shi, Nailong Wu, Huaicheng Yan, Yueying Wang
Summary: In this article, an adaptive neural network (NN) control problem is studied for nonstrict-feedback multi-input multi-output (MIMO) nonlinear systems with unmeasurable states and unknown hysteresis. The unmeasurable states are estimated using a NN state observer, and the unknown nonlinear terms are approximated using radial basis function-neural networks (RBF-NNs). The complexity problem is addressed by using dynamic surface control (DSC), and a nonlinear gain feedback function is introduced to improve the dynamic performance of the closed-loop system. A prescribed performance control (PPC) technique is implemented to guarantee the tracking error convergence. The proposed control scheme ensures that all closed-loop signals are semi-global uniformly ultimately bounded (SGUUB).
NEURAL COMPUTING & APPLICATIONS
(2023)
Article
Automation & Control Systems
Yang Zhou, Shubo Wang
Summary: This paper investigates asymptotic tracking control of nonlinear robotic systems with prescribed performance. The control strategy is developed based on a modified prescribed performance function (PPF) and fuzzy logic system (FLS) to approximate the unknown dynamics. A robust integral of the sign of the error (RISE) term is incorporated into the control design to achieve asymptotic convergence. Numerical simulation and experimental results validate the effectiveness of the proposed control scheme.
JOURNAL OF THE FRANKLIN INSTITUTE-ENGINEERING AND APPLIED MATHEMATICS
(2023)
Article
Computer Science, Artificial Intelligence
Mingming Ha, Ding Wang, Derong Liu
Summary: In this article, a novel value iteration scheme is proposed, which introduces a relaxation factor and combines with other methods to accelerate and guarantee the convergence. The theoretical results and numerical examples demonstrate its fast convergence speed and stability.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Automation & Control Systems
Shunchao Zhang, Bo Zhao, Derong Liu, Cesare Alippi, Yongwei Zhang
Summary: In this article, an event-triggered robust control (ETRC) method is investigated for multi-player nonzero-sum games of continuous-time input constrained nonlinear systems with mismatched uncertainties. The method transforms the robust control problem into an optimal regulation problem by constructing an auxiliary system and designing an appropriate value function. A critic neural network (NN) is used to approximate the value function of each player and obtain control laws. The method reduces computational burden and communication bandwidth by updating the control laws when events occur. The effectiveness of the developed ETRC method is demonstrated through theoretical analysis and examples.
INTERNATIONAL JOURNAL OF ROBUST AND NONLINEAR CONTROL
(2023)
Article
Computer Science, Artificial Intelligence
Qiuye Wu, Bo Zhao, Derong Liu, Marios M. Polycarpou
Summary: This paper proposes an event-triggered adaptive dynamic programming method to solve the decentralized tracking control problem for input constrained unknown nonlinear interconnected systems. A neural-network-based local observer is established to reconstruct the system dynamics using local input-output data and desired trajectories. The DTC problem is transformed into an optimal control problem using a nonquadratic value function. The DTC policy is obtained by solving the local Hamilton-Jacobi-Bellman equation through the observer-critic architecture, with weights tuned by the experience replay technique. Simulation examples demonstrate the effectiveness of the proposed scheme.
Article
Computer Science, Artificial Intelligence
Mingduo Lin, Bo Zhao, Derong Liu
Summary: A novel policy gradient (PG) adaptive dynamic programming method is proposed for nonlinear discrete-time zero-sum games with unknown dynamics. A policy iteration algorithm is used to approximate the Q-function and the control and disturbance policies using neural network approximators. The control and disturbance policies are then updated using the PG method based on the iterative Q-function. The experience replay technique is applied to improve training stability and data usage efficiency. Simulation results show the effectiveness of the proposed method.
Article
Automation & Control Systems
Runqi Chai, Derong Liu, Tianhao Liu, Antonios Tsourdos, Yuanqing Xia, Senchun Chai
Summary: This paper presents an integrated real-time trajectory planning and tracking control framework for autonomous ground vehicles (AGV) parking maneuver problems, utilizing deep neural networks and recurrent network structures. Two transfer learning strategies are applied to adapt the motion planner for different AGV types. Experimental studies demonstrate enhanced performance in fulfilling parking missions.
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING
(2023)
Article
Computer Science, Artificial Intelligence
Danyu Lin, Shan Xue, Derong Liu, Mingming Liang, Yonghua Wang
Summary: In this paper, a problem of multiplayer hierarchical decision-making for non-affine systems is solved using adaptive dynamic programming. The control dynamics are obtained and combined with the original system dynamics, transforming the non-affine multiplayer system into a general affine form. The hierarchical decision problem is modeled as a Stackelberg game, and a neural network is used to reconstruct the augmented system and approximate the value function. The feasibility and effectiveness of the algorithm are confirmed through simulation.
Article
Computer Science, Artificial Intelligence
Chujian Zeng, Bo Zhao, Derong Liu
Summary: This paper proposes a neuro-dynamic programming-based fault tolerant control scheme for a class of nonlinear systems, considering the occurrence of both actuator and sensor faults simultaneously. The scheme combines a descriptor observer with an adaptive observer to estimate system states and multiple faults. By employing a critic neural network, the approximate optimal control policy is obtained for the fault-free system. An FTC law is developed to suppress the influence of actuator faults by combining the estimations of actuator faults with the approximate optimal control policy. The stability of the closed-loop nonlinear system is analyzed using the Lyapunov stability theorem.
Article
Automation & Control Systems
Yongwei Zhang, Bo Zhao, Derong Liu, Shunchao Zhang
Summary: In this article, the event-triggered robust control problem of unknown multiplayer nonlinear systems with constrained inputs and uncertainties is investigated using adaptive dynamic programming. A neural network-based identifier is constructed to relax the requirement of system dynamics. By designing a nonquadratic value function, the stabilization problem is converted into a constrained optimal control problem. The approximate solution of the event-triggered Hamilton-Jacobi equation is obtained using a critic network with a novel weight updating law, and the Lyapunov stability theorem ensures that the multiplayer system is uniformly ultimately bounded.
IEEE TRANSACTIONS ON CYBERNETICS
(2023)
Article
Automation & Control Systems
Mingming Liang, Yonghua Wang, Derong Liu
Summary: In this study, a novel general impulsive transition matrix is defined to reveal the transition dynamics and probability distribution evolution patterns between impulsive events. Based on this matrix, policy iteration-based impulsive adaptive dynamic programming algorithms are developed to solve optimal impulsive control problems. The algorithms demonstrate convergence to the optimal impulsive performance index function and allow for optimization on computing devices with low memory spaces.
IEEE TRANSACTIONS ON CYBERNETICS
(2023)
Proceedings Paper
Automation & Control Systems
Jinquan Lin, Bo Zhao, Derong Liu
Summary: In this paper, an integral reinforcement learning (IRL)-based approximate optimal control (AOC) method is developed for unknown nonaffine systems using dynamic feedback. The optimal control policy for nonaffine systems cannot be explicitly expressed due to the unknown input gain matrix. Thus, a dynamic feedback signal is introduced to transform the nonaffine system into an augmented affine system. The AOC for unknown nonaffine systems is formulated by designing an appropriate value function for the augmented affine system, and the IRL method is adopted to derive the approximate solution of the Hamilton-Jacobi-Bellman equation.
2023 IEEE 12TH DATA DRIVEN CONTROL AND LEARNING SYSTEMS CONFERENCE, DDCLS
(2023)
Article
Computer Science, Artificial Intelligence
Zhanyu Yang, Bo Zhao, Derong Liu
Summary: In this article, a novel pinning control method that requires only partial node information is developed to synchronize drive-response memristor-based neural networks with time delay. An improved mathematical model of the networks is established to accurately describe their dynamic behaviors. Unlike previous literature that requires information from all nodes, the proposed method only relies on local information to achieve synchronization of delayed networks, reducing communication and calculation burdens. Sufficient conditions for synchronization are provided, and numerical simulation and comparative experiments validate the effectiveness and superiority of the proposed method.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Automation & Control Systems
Mingduo Lin, Bo Zhao, Derong Liu
Summary: In this article, an event-triggered robust adaptive dynamic programming (ETRADP) algorithm is proposed to solve multiplayer Stackelberg-Nash games (MSNGs) for uncertain nonlinear continuous-time systems. The hierarchical decision-making process considering different roles of players is described, transforming the robust control problem into an optimal regulation problem. An online policy iteration algorithm is used to solve the derived Hamilton-Jacobi equation with an event-triggered mechanism to reduce computational and communication burdens. Critic neural networks (NNs) are constructed to obtain the event-triggered approximate optimal control policies for all players, ensuring the stability of the closed-loop system.
IEEE TRANSACTIONS ON CYBERNETICS
(2023)
Article
Automation & Control Systems
Mingming Liang, Derong Liu
Summary: This article focuses on designing the optimal impulsive controller (IMC) of continuous-time nonlinear systems and proposes a new adaptive dynamic programming algorithm with high generality and feasibility. The introduced policy-improving mechanism makes the algorithm more flexible for memory-limited computing devices.
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
(2023)
Article
Automation & Control Systems
Ke Wang, Chaoxu Mu, Zhen Ni, Derong Liu
Summary: This paper presents a novel composite obstacle avoidance control method that generates safe motion trajectories for autonomous systems in an adaptive manner. The method combines model-based policy iteration and state-following-based approximation in a safe reinforcement learning framework. The proposed learning-based controller achieves stable reaching of target points while maintaining a safe distance from obstacles. The effectiveness of the method is demonstrated through simulations and comparisons with other avoidance control methods.
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING
(2023)
Article
Computer Science, Artificial Intelligence
Bo Zhao, Yongwei Zhang, Derong Liu
Summary: This article presents a cooperative motion/force control scheme for modular reconfigurable manipulators (MRMs) based on adaptive dynamic programming (ADP). The dynamic model of the entire MRM system is treated as a set of joint modules interconnected by coupling torque, and the Jacobian matrix is mapped into each joint. A neural network is used as a robust decentralized observer, and an improved local value function is constructed for each joint module. The control scheme is achieved by using force feedback compensation and is proven to be uniformly ultimately bounded through Lyapunov stability analysis.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
