Article
Automation & Control Systems
Lingzhi Zhang, Lei Xie, Wei Dai, Shan Lu, Hongye Su
Summary: It is common for industrial processes to have a hierarchical control structure with basic and operational loops. However, there is a multirate challenge where control and sampling rates may differ even within a single loop. The article presents a novel optimal tracking control method for multirate systems, utilizing a lifting technique for handling asynchronism and a disturbance observer for estimating unknown disturbances.
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
(2023)
Article
Automation & Control Systems
Chao Liu, Jinliang Ding, Jiyuan Sun
Summary: This article introduces a model-free reinforcement learning algorithm for decision making of operational indices in plant-wide production processes. Using a multiactor networks ensemble algorithm and an actor-critic framework with stochastic policy, a relatively optimal policy is achieved, along with addressing the issue of lacking sampling data through experience replay.
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
(2021)
Article
Computer Science, Artificial Intelligence
Qili Chen, Junfang Fan, Wenbai Chen, Ancai Zhang, Guangyuan Pan
Summary: In this paper, a dimension-reducible data-driven optimization control framework for wastewater treatment process (WWTP) is proposed. The constraint relationship between control variables is approximated using a neural network, and the optimization search is performed in a low-dimensional space. The convergence of the process is ensured through mathematical analysis. Experimental simulation results show the effectiveness of this approach in achieving an optimal solution in control systems.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Computer Science, Artificial Intelligence
Zhiwei Shang, Renxing Li, Chunhua Zheng, Huiyun Li, Yunduan Cui
Summary: In this article, a novel reinforcement learning approach, continuous dynamic policy programming (CDPP), is proposed to improve learning stability and sample efficiency in RL methods with continuous actions. The proposed method utilizes relative entropy regularization and Monte Carlo estimation to enhance the learning process, and outperforms baseline approaches in several tasks.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Automation & Control Systems
Jianguo Zhao, Chunyu Yang, Wei Dai, Weinan Gao
Summary: This article investigates the optimal operational control problem for a class of industrial systems consisting of multiple unit devices. The proposed decentralized composite control scheme, using singular perturbation theory, achieves the desired operational index tracking and disturbance rejection. The online and offline controller design methods are proposed for the slow and fast subsystems, respectively, ensuring optimal control without requiring knowledge of the operational process dynamics.
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
(2022)
Article
Automation & Control Systems
Huiyuan Shi, Chen Yang, Xueying Jiang, Chengli Su, Ping Li
Summary: This study introduces a novel data-driven two-dimensional off-policy Q-learning method based on output feedback for achieving optimal tracking control in batch processes. The proposed method effectively solves the optimal control problem using only system measurement data and demonstrates unbiasedness and convergence.
JOURNAL OF PROCESS CONTROL
(2022)
Article
Automation & Control Systems
Chenxi Wang, Youtian Du, Yuanlin Chang, Zihao Guo, Yanhao Huang
Summary: This article presents a human-machine collaborative framework for controlling line flow in power systems. Experimental results demonstrate that this approach can significantly improve the regulation performance.
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
(2023)
Article
Computer Science, Interdisciplinary Applications
Kalpesh M. Patel
Summary: Industrial process control using model-based technologies is well established but non-adaptive. Reinforcement Learning (RL) offers a model-free adaptive alternative. This paper presents a systematic method of formulating the RL problem incorporating domain-specific knowledge to enhance safety, speed, and explainability of online RL implementation without requiring a simulation model.
COMPUTERS & CHEMICAL ENGINEERING
(2023)
Article
Automation & Control Systems
Gangshan Jing, He Bai, Jemin George, Aranya Chakrabortty
Summary: The study proposes a method to decompose the large-scale LQR design problem into smaller problems and solve it as a graph clustering problem to reduce computational complexity. The resulting controller has a hierarchical structure with two components optimizing independent clusters' performance and handling the coupling of different clusters.
IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS
(2021)
Article
Computer Science, Artificial Intelligence
Amir Modares, Nasser Sadati, Babak Esmaeili, Farnaz Adib Yaghmaie, Hamidreza Modares
Summary: This article presents a data-driven safe reinforcement learning algorithm for discrete-time nonlinear systems. A data-driven safety certifier is designed to ensure both safety and stability of the RL agent's actions. It bypasses the identification of the system model and directly learns a robust safety certifier.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Automation & Control Systems
Qinge Xiao, Ben Niu, Bing Xue, Luoke Hu
Summary: This paper proposes a graph convolutional reinforcement learning (GCRL) method for advanced energy-aware process planning in machining systems. The method uses a graph convolutional policy network and graph embedding to adapt tasks and compress topology, while reinforcement learning achieves robust learning for process planning. A two-phase multitask training strategy is adopted to improve adaptability.
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
(2023)
Article
Computer Science, Artificial Intelligence
Majid Mazouchi, Yongliang Yang, Hamidreza Modares
Summary: This article presents an iterative data-driven algorithm for solving dynamic multiobjective optimal control problems, leveraging Hamiltonian functional and inequalities. Relaxed HJB equations in a dynamic constrained MO framework are used to find Pareto optimal solutions. A SOS-based iterative algorithm is developed for aspiration-satisfying MO optimization.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2022)
Article
Computer Science, Artificial Intelligence
Tomas Kulvicius, Minija Tamosiunaite, Florentin Worgotter
Summary: This paper presents a method for solving path-finding problems by transforming cost values into synaptic weights in a neural network. The method allows for online weight adaptation using network learning mechanisms, and has been demonstrated to be effective in navigating in environments with obstacles and following specific sequences of path nodes.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Engineering, Chemical
Max Mowbray, Robin Smith, Ehecatl A. Del Rio-Chanona, Dongda Zhang
Summary: The framework combines apprenticeship learning and reinforcement learning to analyze historical data offline for synchronous identification of rewards and parameterization, followed by online iterative improvement of the parameterization to achieve optimal control of the process.
Article
Computer Science, Artificial Intelligence
Liwei Huang, Mingsheng Fu, Ananya Rao, Athirai A. Irissappane, Jie Zhang, Chengzhong Xu
Summary: This article proposes a distributional multiagent cooperation (DMAC) framework to explicitly model the distributed dynamics of agents and achieve better performance. Experimental results demonstrate that DMAC significantly outperforms other algorithms in multiple scenarios.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2022)
Article
Engineering, Electrical & Electronic
Jin Wu, Yi Jiang, Chong Li, Ran Sun, Chengxi Zhang, Yang Yu, Yilong Zhu, Ming Liu
Summary: This brief introduces a circuit synthesis approach for solving the rotation orthonormalization problem, avoiding complex mathematical computations. The theoretical convergence of the designed circuit is proved, and experiments confirm the effectiveness of the method. The proposed circuit scheme has been successfully implemented on an FPGA platform.
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-EXPRESS BRIEFS
(2021)
Article
Automation & Control Systems
Yi Jiang, Weinan Gao, Jing Na, Di Zhang, Timo T. Hamalainen, Vladimir Stojanovic, Frank L. Lewis
Summary: This paper investigates the learning-based adaptive optimal output regulation problem with convergence rate requirement for disturbed linear continuous-time systems. It proposes an adaptive optimal control approach based on reinforcement learning and adaptive dynamic programming to learn the optimal regulator with assured convergence rate. The proposed approach successfully solves the problem by tackling static and dynamic optimization problems, and a novel online value iteration algorithm is introduced to learn both the optimal feedback control gain and the corresponding feedforward control gain. Numerical analysis demonstrates that the proposed approach can achieve desired disturbance rejection and tracking performance.
CONTROL ENGINEERING PRACTICE
(2022)
Article
Engineering, Electrical & Electronic
Chengxi Zhang, Choon Ki Ahn, Jin Wu, Wei He, Yi Jiang, Ming Liu
Summary: This paper studies a method for simultaneous estimation of states and uncertainties, and proposes a novel time-varying learning intensity learning observer. The observer has a simple structure and low computational costs, and improves its performance by using the time-varying learning intensity approach.
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-EXPRESS BRIEFS
(2022)
Article
Automation & Control Systems
Weinan Gao, Chao Deng, Yi Jiang, Zhong-Ping Jiang
Summary: This paper proposes a novel resilient reinforcement learning approach for solving robust optimal output regulation problems of a class of partially linear systems under both dynamic uncertainties and denial-of-service attacks. The approach rigorously analyzes the resilience of closed-loop systems against attacks and the robustness against dynamic uncertainties, and introduces a faster convergence learning strategy.
Article
Automation & Control Systems
Yi Jiang, Yao Jia, Jialu Fan, Tianyou Chai
Summary: This article addresses the operational control problem of the flotation industrial process, a strong nonlinear and coupling multivariable cascade process. A compensation-signal-based dual-rate operational feedback adaptive decoupling control approach is proposed, which involves the design of device layer and operational layer controllers. Convergence proofs and stability analysis are provided, and the effectiveness of the approach is verified through emulation experiments.
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS
(2022)
Article
Automation & Control Systems
Jin Wu, Miaomiao Wang, Yi Jiang, Bowen Yi, Rui Fan, Ming Liu
Summary: This study focuses on calibrating the extrinsic parameter between a camera and an IMU using an industrial robotic manipulator. A simultaneous calibration method for hand-eye/robot-world/camera-IMU is proposed, which eliminates the need for inertial integration and ensures robustness to uncertain IMU biases. The method involves a highly nonconvex optimization on the special Euclidean group to achieve globally optimal solutions. Experimental results demonstrate the accuracy of the proposed method and comparative studies prove its global optimality. The method is applicable for calibrating a robot/camera/IMU combination and guarantees accuracy and computational efficiency.
IEEE-ASME TRANSACTIONS ON MECHATRONICS
(2022)
Article
Robotics
Jin Wu, Yu Zheng, Zhi Gao, Yi Jiang, Xiangcheng Hu, Yilong Zhu, Jianhao Jiao, Ming Liu
Summary: Pose estimation problems are fundamental and challenging in robotics due to their nonconvex nature. This article presents a quaternion-based mathematical model to unify a class of related problems, and solves the nonconvex problems using the Grobner-basis method to obtain globally optimal and robust solutions. The uncertainty description is also analyzed, and it is shown that the covariance can be efficiently estimated via online optimization, ensuring global optimality for both the solution and covariance.
IEEE TRANSACTIONS ON ROBOTICS
(2022)
Article
Automation & Control Systems
Yi Jiang, Kai Zhang, Jin Wu, Chengxi Zhang, Wenqian Xue, Tianyou Chai, Frank L. Lewis
Summary: This work investigates the H infinity-based minimal energy control with a preset convergence rate problem for disturbed linear time-invariant continuous-time systems, proposing a modified game algebraic Riccati equation and a novel approach for solving actuator magnitude saturation problem. The effectiveness of the proposed methods is verified through simulation examples.
IEEE TRANSACTIONS ON CYBERNETICS
(2022)
Article
Automation & Control Systems
Yi Jiang, Weinan Gao, Jin Wu, Tianyou Chai, Frank L. Lewis
Summary: This paper proposes a novel control approach to solve the cooperative H infinity output regulation problem for linear continuous-time multi-agent systems (MASs). A distributed feedforward-feedback controller is developed to achieve asymptotic tracking and reject disturbances. A value iteration algorithm is proposed to learn the optimal feedback control gain and feedforward control gain using online data. The effectiveness of the proposed approach is demonstrated through numerical analysis.
Article
Automation & Control Systems
Jin Wu, Miaomiao Wang, Hassen Fourati, Hui Li, Yilong Zhu, Chengxi Zhang, Yi Jiang, Xiangcheng Hu, Ming Liu
Summary: This study investigates the generalized rigid registration problem in high-dimensional Euclidean spaces and proposes a method to minimize the loss function using the Cayley formula. By deriving a closed-form linear least-square solution, the registration covariances are obtained, providing accurate probabilistic descriptions. The proposed method demonstrates its efficiency in terms of computation time and accuracy compared to previous algorithms. Additionally, it is applied to an interpolation problem on a special Euclidean group and shows practical superiority in covariance-aided Lidar mapping for robotic navigation.
IEEE TRANSACTIONS ON CYBERNETICS
(2023)
Article
Automation & Control Systems
Ci Chen, Lihua Xie, Yi Jiang, Kan Xie, Shengli Xie
Summary: In this article, the optimal output tracking problem for linear discrete-time systems with unknown dynamics is investigated using reinforcement learning (RL) and robust output regulation theory. Different from most existing works, which depend on the state of the system, this problem only utilizes the outputs of the reference system and the controlled system. The proposed off-policy RL algorithm allows for solving the output tracking problem using only measured output data and the reference output, without requiring complete and accurate system dynamics knowledge.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL
(2023)
Article
Automation & Control Systems
Yi Jiang, Dawei Shi, Jialu Fan, Tianyou Chai, Tongwen Chen
Summary: In this work, an event-triggered adaptive control approach is developed to solve the state tracking problem of linear partially time-variant continuous-time systems with nonlinear state-dependent matched parametric uncertainty under unknown system dynamics. The approach includes the design of an event-triggered model reference adaptive controller and the analysis of the stability properties. Simulation results demonstrate the effectiveness of the proposed approach.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL
(2023)
Article
Computer Science, Information Systems
Kedi Xie, Yi Jiang, Xiao Yu, Weiyao Lan
Summary: In this study, a data-driven learning algorithm is developed to estimate the optimal distributed cooperative control policy for linear discrete-time multi-agent systems. The algorithm combines adaptive dynamic programming with an internal model to propose a model-free off-policy learning method. By accessing the measurable data of the multi-agent systems, the algorithm estimates the optimal control gain and distributed adaptive internal model. The online approximation of the distributed internal model is a key difference from traditional cooperative adaptive controller design methods. Convergence and stability analyses demonstrate that the estimated controller generated by the data-driven learning algorithm converges to the optimal distributed controller. Simulation results validate the effectiveness of the proposed method.
SCIENCE CHINA-INFORMATION SCIENCES
(2023)
Article
Computer Science, Artificial Intelligence
Lingzhi Zhang, Lei Xie, Yi Jiang, Zhishan Li, Xueqin Liu, Hongye Su
Summary: This article proposes a constrained optimal control approach for discrete-time nonlinear systems based on safe reinforcement learning. By introducing a barrier function, the constrained optimization problem is transformed into an unconstrained one, and a constrained policy iteration algorithm is developed to ensure optimal control and constraint satisfaction.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Computer Science, Artificial Intelligence
Yi Jiang, Lu Liu, Gang Feng
Summary: This article investigates the adaptive optimal control problem for networked discrete-time nonlinear systems with stochastic packet dropouts in both controller-to-actuator and sensor-to-controller channels. It first develops a Bernoulli model-based Hamilton-Jacobi-Bellman (BHJB) equation to deal with the nonadaptive optimal control problem. Reinforcement learning-based policy iteration (PI) and value iteration (VI) algorithms are further developed for solving the BHJB equation, and their convergence analysis is provided. When partial system dynamics and packet dropout probabilities are unknown, two online RL-based PI and VI algorithms are developed using critic-actor approximators and packet dropout probability estimator. Simulation studies are provided to illustrate the effectiveness of the proposed approaches.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2022)
