Article
Automation & Control Systems
Guangyu Zhu, Xiaolu Li, Ranran Sun, Yiyuan Yang, Peng Zhang
Summary: In this paper, a new iterative adaptive dynamic programming algorithm called discrete-time time-varying policy iteration (DTTV) algorithm is developed for infinite horizon optimal control problems of discrete time-varying nonlinear systems. The algorithm updates the iterative value function to approximate the index function of optimal performance. The admissibility and convergence properties of the iterative control law are analyzed.
IEEE-CAA JOURNAL OF AUTOMATICA SINICA
(2023)
Article
Engineering, Industrial
Qinge Xiao, Zhile Yang, Yingfeng Zhang, Pai Zheng
Summary: Batch machining systems are crucial for productivity and quality improvement but consume significant energy due to continuous interaction with machine tools, workpieces, and cutting tools. This study focuses on adaptive process control considering time-varying tool wear using reinforcement learning (RL). An energy-efficient decision model is developed using the Markov decision process, and an actor-critic RL framework is proposed for dynamic process control. Experimental results show that the RL method can reduce energy consumption by over 20% compared to optimization ignoring tool wear effects and has three times faster learning efficiency than metaheuristic methods.
JOURNAL OF MANUFACTURING SYSTEMS
(2023)
Article
Automation & Control Systems
Xingwei Zhao, Shibo Han, Bo Tao, Zhouping Yin, Han Ding
Summary: This article proposes a method for learning interactive skills in complex robot applications by using a model-based approach and a safety learning strategy to find the optimal impedance control. Experimental results demonstrate the effectiveness and performance of the proposed method in human-robot interaction and robot machining tasks.
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS
(2022)
Article
Computer Science, Artificial Intelligence
Guangzhu Peng, C. L. Philip Chen, Chenguang Yang
Summary: An adaptive admittance control scheme is developed in this paper to address robot interaction with time-varying environments. The critic learning method is used to obtain the desired parameters, and a neural-network (NN)-based adaptive controller is utilized to deal with dynamic uncertainties. Experimental results have confirmed the effectiveness of the proposed method.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2022)
Article
Engineering, Marine
Zhengkun Wang, Lijun Zhang
Summary: This paper investigates the distributed optimal formation tracking control problem for multiple underactuated autonomous underwater vehicles (AUVs) using backstepping technique and reinforcement learning. A virtual distributed formation tracking controller is proposed based on graph theory, ensuring connectivity preservation and collision avoidance using barrier Lyapunov function. An optimal controller based on reinforcement learning is designed to minimize a cost function, with critic-actor neural networks used for online implementation. The proposed method allows online realization of optimal control design for uncertain hydrodynamic underactuated AUVs, with a command filter adopted to address complexity explosion. Simulation results confirm the effectiveness of the method.
Article
Automation & Control Systems
Jingyi Lu, Zhixing Cao, Qinran Hu, Zuhua Xu, Wenli Du, Furong Gao
Summary: This article proposes a new OILC method for addressing the robustness issue of OILC against model mismatch. The method minimizes a dynamic upper bound on tracking error and formulates the problem in the framework of convex-concave game, which can be efficiently solved by a subgradient method. Experimental results show that the proposed method is effective in handling nonlinearity.
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
(2022)
Article
Engineering, Electrical & Electronic
Lei Shi, Xuesong Wang, Yuhu Cheng
Summary: In this paper, the authors propose a safe reinforcement learning-based robust approximate optimal controller (SRL-RAOC) for hypersonic flight vehicles. They develop a system transformation approach to convert full-state safety constraints into an unconstrained optimization problem and utilize the safe reinforcement learning algorithm within an online actor-critic framework to design the approximate optimal controller. The stability analysis of the closed-loop system is conducted using the Lyapunov technique, demonstrating that SRL-RAOC guarantees asymptotic stability of the equilibrium point and yields control input close to the optimal input.
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY
(2023)
Article
Computer Science, Artificial Intelligence
Lei Guo, Han Zhao
Summary: In this study, a novel algorithm based on synchronous policy iteration is proposed for solving the continuous-time infinite-horizon optimal control problem of input affine system dynamics. The integral reinforcement is used as an excitation signal to estimate the solution to the Hamilton-Jacobi-Bellman equation. The proposed method is completely model-free and utilizes adaptive tuning law for approximating the optimal value function and policy.
Article
Engineering, Electrical & Electronic
Zhinan Peng, Rui Luo, Jiangping Hu, Kaibo Shi, Bijoy Kumar Ghosh
Summary: This paper addresses the event-triggered optimal tracking control of discrete-time multi-agent systems using reinforcement learning. An event-triggered mechanism is proposed to update the controller only when certain events occur, reducing computational burden and transmission load. The effectiveness and performance of the proposed event-triggered reinforcement learning controller are demonstrated through simulation examples.
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I-REGULAR PAPERS
(2022)
Article
Automation & Control Systems
Ding Wang, Jiangyu Wang, Mingming Zhao, Peng Xin, Junfei Qiao
Summary: This paper presents a novel integrated multi-step heuristic dynamic programming (MsHDP) algorithm for solving optimal control problems. The algorithm, initialized by the zero cost function, is shown to converge to the optimal solution of the Hamilton-Jacobi-Bellman (HJB) equation. The stability of the system is analyzed using control policies generated by MsHDP, and a general stability criterion is designed. Furthermore, an integrated MsHDP algorithm is developed to accelerate learning efficiency by utilizing immature control policies and implementing actor-critic with neural networks.
IEEE-CAA JOURNAL OF AUTOMATICA SINICA
(2023)
Article
Computer Science, Information Systems
Mehdi Mohammadi, Mohammad Mehdi Arefi, Peyman Setoodeh, Okyay Kaynak
Summary: This article investigates the design of an optimal tracking controller for a class of nonlinear continuous-time systems with time-delay, mismatched external disturbances, and input constraints using the technique of integral reinforcement learning (IRL). A disturbance observer is designed to enable the usage of an estimation of the external disturbances in the recursive objective function. The proposed approach utilizes a Hamilton-Jacobi-Bellman (HJB) equation and iterative IRL algorithm for deriving the optimal control input, ensuring that the output of the time-delayed nonlinear system tracks the desired trajectory with bounded error in the presence of mismatched disturbances. A critic neural network is designed for implementation, and the efficiency of the method is illustrated through a simulation example.
INFORMATION SCIENCES
(2021)
Article
Automation & Control Systems
Junfei Qiao, Mingming Zhao, Ding Wang, Mingming Ha
Summary: This article introduces the VIQL algorithm with adjustable convergence speed and explores its application through the verification of trajectory tracking in completely unknown nonaffine systems. The adjustable VIQL scheme is capable of speeding up learning and reducing computation burden by adjusting convergence speed.
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
(2023)
Article
Computer Science, Artificial Intelligence
Jie Lan, Yan-Jun Liu, Dengxiu Yu, Guoxing Wen, Shaocheng Tong, Lei Liu
Summary: This article investigates a distributed time-varying optimal formation protocol using adaptive neural network state observer and simplified reinforcement learning, which can achieve the desired formation configuration in the presence of uncertainty and measurement limitations.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2022)
Article
Engineering, Marine
Huizi Chen, Huaicheng Yan, Yueying Wang, Shaorong Xie, Dan Zhang
Summary: This paper studies the close formation control problem for a group of underactuated surface vehicles with 4 degrees of freedom. The control method includes a finite-time sliding mode control scheme based on reinforcement learning algorithm to guarantee the formation performance, and actor-critic neural network algorithm to estimate actuator faults and system uncertainties. It also introduces an exponential decreasing boundary function to suppress overshoot and a switching gain mechanism to alleviate chattering in sliding mode control. Numerical simulations and experimental results demonstrate the effectiveness and superior formation performance of the proposed control method.
Article
Computer Science, Artificial Intelligence
Xiong Yang, Qinglai Wei
Summary: This article investigates the optimal event-triggered control problem of nonlinear continuous-time systems with asymmetric control constraints. A discounted cost is introduced to obtain the optimal solution, and an event-triggered Hamilton-Jacobi-Bellman equation and triggering condition are proposed to solve the problem.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2021)
