Article
Automation & Control Systems
Qinglai Wei, Tianmin Zhou, Jingwei Lu, Yu Liu, Shuai Su, Jun Xiao
Summary: In this article, a new stochastic adaptive dynamic programming (ADP) method is developed to solve the optimal control problem of continuous-time (CT) time-invariant nonlinear systems with stochastic nonlinear disturbances. The method simultaneously approximates the value function and the control law under the conditional expectation. The asymptotic stability of the closed-loop stochastic system in probability is analyzed using the stochastic Lyapunov direct method, and the convergence of the developed ADP method is proven. Four simulations are conducted to demonstrate the effectiveness of the proposed method.
IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS
(2023)
Article
Operations Research & Management Science
Sudeep Kundu, Karl Kunisch
Summary: The study examines the application of policy iteration in solving the HJB equation with control constraints, utilizing an implicit upwind scheme to solve the linear equations. Numerical examples are conducted to compare the results with the unconstrained cases.
COMPUTATIONAL OPTIMIZATION AND APPLICATIONS
(2021)
Article
Automation & Control Systems
Keyan Miao, Richard Vinter
Summary: This article discusses an optimal control problem in neo-classical macroeconomics, aiming to maximize expenditure on social programs by balancing investment for growth and consumption. A nonstandard verification technique is introduced and applied to handle singularities caused by fractional singularities, providing a detailed solution and analysis of the problem.
OPTIMAL CONTROL APPLICATIONS & METHODS
(2021)
Article
Environmental Sciences
Lu Xiao, Ya Chen, Chaojie Wang, Jun Wang
Summary: This paper discusses the cooperation between asymmetric countries in transboundary pollution control, with a focus on the impact of assistant investments provided by developed countries. The study finds that the provision of assistant investments can reduce common pollution stock and increase economic benefits for both countries by raising equilibrium emission strategies.
ENVIRONMENTAL SCIENCE AND POLLUTION RESEARCH
(2022)
Article
Automation & Control Systems
Javier de Frutos, Julia Novo
Summary: This paper provides error bounds for fully discrete approximations of infinite horizon problems using the dynamic programming approach. The paper revises the error bound of the fully discrete method and proves that, under assumptions similar to the time discrete case, the error of the fully discrete case is O(h+k), giving first order accuracy in time and space for the method. This error bound matches numerical experiments in the literature where the behavior predicted by the O(k/h) bound has not been observed.
SIAM JOURNAL ON CONTROL AND OPTIMIZATION
(2023)
Article
Mathematics, Applied
Karl Kunisch, Donato Vasquez-Varas
Summary: This article analyzes a learning technique for finite horizon optimal control problems and its approximation based on polynomials, and illustrates the practicality and efficiency of the method.
COMPUTERS & MATHEMATICS WITH APPLICATIONS
(2023)
Article
Automation & Control Systems
Lina Xia, Qing Li, Ruizhuo Song, Hamidreza Modares
Summary: This paper considers the asymmetric input-constrained optimal synchronization problem of heterogeneous unknown nonlinear multi-agent systems. By performing a state-space transformation and designing a novel distributed observer, the satisfaction of asymmetric input constraints is guaranteed. With the help of a network of augmented systems and a data-based off-policy reinforcement learning algorithm, the constrained Hamilton-Jacobi-Bellman equation is solved. Simulation results demonstrate the correctness and validity of the theoretical results.
IEEE-CAA JOURNAL OF AUTOMATICA SINICA
(2022)
Article
Multidisciplinary Sciences
Alex Tong Lin, Samy Wu Fung, Wuchen Li, Levon Nurbekyan, Stanley J. Osher
Summary: APAC-Net is a neural network algorithm designed for solving stochastic mean-field games (MFGs) in high dimensions. By leveraging the variational primal-dual structure and parameterizing value and density functions with neural networks, the problem can be interpreted as a special case of training a generative adversarial network (GAN), showing potential for solving up to 100-dimensional MFG problems.
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
(2021)
Article
Computer Science, Interdisciplinary Applications
Sudhanshu Agrawal, Wonjun Lee, Samy Wu Fung, Levon Nurbekyan
Summary: We propose an efficient solution approach for high-dimensional nonlocal mean-field game (MFG) systems based on the Monte Carlo approximation of interaction kernels via random features. By passing to the feature-space and avoiding costly space-discretizations of interaction terms, this approach enables seamless mean-field extension of almost any single-agent trajectory optimization algorithm. We demonstrate the efficiency of our method by solving MFG problems in high-dimensional spaces previously out of reach for conventional non-deep-learning techniques.
JOURNAL OF COMPUTATIONAL PHYSICS
(2022)
Review
Transportation Science & Technology
Mansour Johari, Mehdi Keyvan-Ekbatani, Ludovic Leclercq, Dong Ngoduy, Hani S. Mahmassani
Summary: Network macroscopic fundamental diagrams and related traffic dynamics models have theoretical support and empirical validation, but their readiness for practical implementation is still uncertain. This paper reviews the history of macroscopic modeling, assesses remaining gaps, and discusses opportunities for further development in theory and applications.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2021)
Article
Transportation Science & Technology
Rafegh Aghamohammadi, Jorge A. Laval
Summary: This paper extends the Stochastic Method of Cuts to approximate the Macroscopic Fundamental Diagram of urban networks and uses Maximum Likelihood Estimation method to estimate the model parameters. The results show satisfactory parameter estimates and graphical fits for the studied networks.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2022)
Article
Mathematics, Applied
Linlin Tian, Zhaoyang Liu
Summary: This paper investigates the optimal dividend problem for the renewal risk model with phase-type distributed interclaim times and exponentially distributed claim sizes. By proposing an algorithm and analyzing the theoretical properties, the optimal strategy is found, and the optimality of phasewise barrier strategy as well as the convergence of the algorithm is proved.
APPLIED MATHEMATICS AND OPTIMIZATION
(2022)
Article
Automation & Control Systems
Jeongho Kim, Insoon Yang
Summary: Maximum entropy reinforcement learning methods have been successfully applied to a range of challenging sequential decision-making and control tasks. However, there is a need to extend these methods to continuous-time systems. This article studies the theory of maximum entropy optimal control in continuous time and derives a novel class of equations. The results demonstrate the performance of the maximum entropy method in continuous-time optimal control and reinforcement learning problems.
IEEE TRANSACTIONS ON AUTOMATIC CONTROL
(2023)
Article
Automation & Control Systems
Tobias Breiten, Karl Kunisch
Summary: Nonlinear observers based on the minimum energy estimation concept are discussed, with an output injection operator approximated by a neural network. An optimization problem is proposed to learn the network parameters and numerically investigate linear and nonlinear oscillators.
SYSTEMS & CONTROL LETTERS
(2021)
Article
Chemistry, Analytical
Juan Parras, Patricia A. Apellaniz, Santiago Zazo
Summary: This study utilizes deep learning and optimal control tools to address underwater motion planning problems, taking into account disturbances in the underwater medium, and proposes an effective solution using the Deep Galerkin Method.
Editorial Material
Transportation
Andy H. F. Chow, Yong-Hong Kuo, Panagiotis Angeloudis, Michael G. H. Bell
TRANSPORTMETRICA B-TRANSPORT DYNAMICS
(2022)
Editorial Material
Transportation
Renxin Zhong, Zhengbing He, Andy H. F. Chow, Victor Knoop
TRANSPORTMETRICA A-TRANSPORT SCIENCE
(2022)
Article
Transportation Science & Technology
Andy H. F. Chow, Z. C. Su, E. M. Liang, R. X. Zhong
Summary: This paper introduces an adaptive signal controller that effectively manages traffic delays and urban bus service reliability using fully adaptable acyclic timing plans. The controller is built upon a reinforcement learning framework combining model-based and data-driven components to reduce traffic delays and bus service variabilities.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2021)
Article
Transportation Science & Technology
Z. C. Su, Andy H. F. Chow, R. X. Zhong
Summary: This paper presents an adaptive traffic controller for stochastic road networks, using an integrated model-based and data-driven solution framework. The model-based component facilitates the training of the data-driven ADP-based state approximator to improve the overall performance of the control system. A decentralised solution approach is further developed to enhance and stabilise the performance of the overall control system, even under congested conditions.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2021)
Article
Computer Science, Artificial Intelligence
Y. P. Huang, C. Chen, Z. C. Su, T. S. Chen, A. Sumalee, T. L. Pan, R. X. Zhong
Summary: Accurate bus arrival time prediction is crucial for maintaining stability and attracting more passengers to improve transit services. This paper proposes data-driven approaches based on FDA and BSVR, with a probabilistic nested delay operator, to increase prediction accuracy and conduct journey time reliability analysis. Empirical studies in Guangzhou show that the proposed methods are competitive in various traffic conditions, with FDA providing more accurate results and anticipating uncertainties effectively.
APPLIED SOFT COMPUTING
(2021)
Article
Transportation
Cheng Zhang, H. W. Ho, William H. K. Lam, Wei Ma, S. C. Wong, Andy H. F. Chow
Summary: This paper proposes a new method for estimating lane-based travel time distributions by vehicle type using low-resolution vehicle video images captured by conventional traffic surveillance cameras. The method utilizes deep learning and graph matching techniques, and performs well in vehicle type-specific traffic management schemes.
JOURNAL OF INTELLIGENT TRANSPORTATION SYSTEMS
(2023)
Article
Economics
Yimo Yan, Andy H. F. Chow, Chin Pang Ho, Yong-Hong Kuo, Qihao Wu, Chengshuo Ying
Summary: This paper provides a comprehensive review of the development and applications of reinforcement learning techniques in logistics and supply chain management. The most popular approach, Q-learning, is adopted by many studies, and recent research in urban logistics has been growing rapidly. Potential directions for future research are also presented.
TRANSPORTATION RESEARCH PART E-LOGISTICS AND TRANSPORTATION REVIEW
(2022)
Article
Engineering, Civil
Cheng Zhang, Bi Yu Chen, William H. K. Lam, H. W. Ho, Xiaomeng Shi, Xiaoguang Yang, Wei Ma, S. C. Wong, Andy H. F. Chow
Summary: The study proposes a new vehicle re-identification method to estimate lane-level travel time distributions by considering lane-level traffic conditions, vehicles' lane changing behaviors, and visual features. A comprehensive case study in Hong Kong demonstrates that the proposed method outperforms existing methods and provides accurate lane-level travel time distribution information on congested urban roads.
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
(2022)
Article
Engineering, Civil
Lubing Li, Wei Huang, Andy H. F. Chow, Hong K. Lo
Summary: This study develops a cell-based two-stage stochastic program to address the dynamic, spatial, and stochastic characteristics of traffic flow for arterial adaptive signal control. By incorporating the concept of Phase Clearance Reliability (PCR) and using a gradient-based solution algorithm, the study enhances solution efficiency and validates findings through VISSIM. Results show the importance of capturing dynamic, spatial, and stochastic features for traffic control in order to avoid delay performance degradation.
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
(2022)
Article
Engineering, Civil
Cheng-Shuo Ying, Andy H. F. Chow, Yi-Hui Wang, Kwai-Sang Chin
Summary: This study presents an integrated metro service scheduling and train unit deployment approach based on deep reinforcement learning framework with a proximal policy optimization method. By parameterizing the value function and control policy through artificial neural networks and incorporating operational constraints through a devised mask scheme, the optimization problem was successfully solved in real-world scenarios with superior performance. Results show the advantages of flexible train compositions in saving operational costs and reducing service irregularities.
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
(2022)
Article
Economics
Cheng-Shuo Ying, Andy H. F. Chow, Hoa T. M. Nguyen, Kwai-Sang Chin
Summary: This paper presents an adaptive control system for coordinated metro operations using a multi-agent deep reinforcement learning approach. The system outperforms previous methods in terms of solution quality and performance achieved.
TRANSPORTATION RESEARCH PART B-METHODOLOGICAL
(2022)
Article
Economics
Z. C. Su, Andy H. F. Chow, C. L. Fang, E. M. Liang, R. X. Zhong
Summary: This study proposes a hierarchical control framework to maximize the throughput of a road network driven by travel demand with uncertainties. The upper level uses a reinforcement learning algorithm to regulate the traffic influx into the core road network without the need for an underlying system model and macroscopic fundamental diagram. The lower level is a local signal control system that regulates the spatial distribution of traffic flow within the core network. The study contributes to the management of urban road networks with advanced computing technologies.
TRANSPORTATION RESEARCH PART B-METHODOLOGICAL
(2023)
Article
Transportation Science & Technology
Hoa T. M. Nguyen, Andy H. F. Chow
Summary: This paper presents an adaptive optimization framework for dynamic rail transit network operations using a rollout surrogate-approximate dynamic programming method. The proposed framework reduces passengers' waiting times significantly with reasonable computational time. The results suggest the potential of the proposed optimizer for real-time applications in large-scale rail transit networks.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2023)
Proceedings Paper
Computer Science, Artificial Intelligence
Enming Liang, Zicheng Su, Chilin Fang, Renxin Zhong
Summary: Efficient traffic signal control is crucial for alleviating urban traffic congestion. Reinforcement learning has the potential to devise optimal signal plans that can adapt to dynamic congestion, but faces challenges. To address these challenges, a universal multi-intersection control framework is proposed, which incorporates a well-known cell transmission model, regularized delay as reward, and a universal neural network structure. Results demonstrate that the proposed framework outperforms state-of-the-art controllers in reducing average travel time.
THIRTY-SIXTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTY-FOURTH CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE / THE TWELVETH SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE
(2022)
Article
Transportation Science & Technology
Yue Zhao, Liujiang Kang, Huijun Sun, Jianjun Wu, Nsabimana Buhigiro
Summary: This study proposes a 2-population 3-strategy evolutionary game model to address the issue of subway network operation extension. The analysis reveals that the rule of maximum total fitness ensures the priority of evolutionary equilibrium strategies, and proper adjustment minutes can enhance the effectiveness of operation extension.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2024)
Article
Transportation Science & Technology
Hongtao Hu, Jiao Mob, Lu Zhen
Summary: This study investigates the challenges of daily storage yard management in marine container terminals considering delayed transshipment of containers. A mixed-integer linear programming model is proposed to minimize various costs associated with transportation and yard management. The improved Benders decomposition algorithm is applied to solve the problem effectively and efficiently.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2024)
Article
Transportation Science & Technology
Zhandong Xu, Yiyang Peng, Guoyuan Li, Anthony Chen, Xiaobo Liu
Summary: This paper studied the impact of range anxiety among electric vehicle drivers on traffic assignment. Two types of range-constrained traffic assignment problems were defined based on discrete or continuous distributed range anxiety. Models and algorithms were proposed to solve the two types of problems. Experimental results showed the superiority of the proposed algorithm and revealed that drivers with heightened range anxiety may cause severe congestion.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2024)
Article
Transportation Science & Technology
Chuanjia Li, Maosi Geng, Yong Chen, Zeen Cai, Zheng Zhu, Xiqun (Michael) Chen
Summary: Understanding spatial-temporal stochasticity in shared mobility is crucial, and this study introduces the Bi-STTNP prediction model that provides probabilistic predictions and uncertainty estimations for ride-sourcing demand, outperforming conventional deep learning methods. The model captures the multivariate spatial-temporal Gaussian distribution of demand and offers comprehensive uncertainty representations.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2024)
Article
Transportation Science & Technology
Benjamin Coifman, Lizhe Li
Summary: This paper develops a partial trajectory method for aligning views from successive fixed cameras in order to ensure high fidelity with the actual vehicle movements. The method operates on the output of vehicle tracking to provide direct feedback and improve alignment quality. Experimental results show that this method can enhance accuracy and increase the number of vehicles in the dataset.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2024)
Article
Transportation Science & Technology
Mohsen Dastpak, Fausto Errico, Ola Jabali, Federico Malucelli
Summary: This article discusses the problem of an Electric Vehicle (EV) finding the shortest route from an origin to a destination and proposes a problem model that considers the occupancy indicator information of charging stations. A Markov Decision Process formulation is presented to optimize the EV routing and charging policy. A reoptimization algorithm is developed to establish the sequence of charging station visits and charging amounts based on system updates. Results from a comprehensive computational study show that the proposed method significantly reduces waiting times and total trip duration.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2024)