Article
Engineering, Civil
Weichang Wang, Yongming Liu, Rayadurgam Srikant, Lei Ying
Summary: This paper introduces a multi-agent reinforcement learning algorithm for UAV path planning, where each UAV makes decisions based on local observations and does not communicate with other UAVs. By training a routing policy using an Actor-Critic neural network, the algorithm addresses the curse-of-dimensionality problem in multi-agent reinforcement learning and achieves good routing policies in complex scenarios in both 2D and 3D space.
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
(2022)
Article
Computer Science, Artificial Intelligence
Xinqi Du, Hechang Chen, Che Wang, Yongheng Xing, Jielong Yang, Philip S. Yu, Yi Chang, Lifang He
Summary: This paper proposes a novel Bayesian Multi-Agent Reinforcement Learning method, named BMARL, which leverages the distributional value function calculated by Bayesian inference to improve the robustness of the model. Extensive experimental results demonstrate the superiority of BMARL in terms of both robustness and efficiency.
PATTERN RECOGNITION
(2024)
Article
Automation & Control Systems
Jianrui Wang, Yitian Hong, Jiali Wang, Jiapeng Xu, Yang Tang, Qing-Long Han, Jurgen Kurths
Summary: This article surveys the issues of cooperative optimization and games in multi-agent systems. It summarizes the research on distributed optimization and federated optimization from the perspective of cooperative optimization. It also introduces cooperative games and non-cooperative games to model the cooperative and non-cooperative behaviors of agents. Finally, it discusses future directions for research in cooperative optimization and games.
IEEE-CAA JOURNAL OF AUTOMATICA SINICA
(2022)
Article
Engineering, Electrical & Electronic
Ye Hu, Mingzhe Chen, Walid Saad, H. Vincent Poor, Shuguang Cui
Summary: This paper studies trajectory design for a group of energy-constrained drones in dynamic wireless network environments. A value decomposition based reinforcement learning solution with a meta-training mechanism is proposed to address this non-convex optimization problem, achieving improved performance in unpredictable environments. Simulation results show significant improvements in service coverage and convergence speed compared to baseline multi-agent algorithms.
IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS
(2021)
Article
Computer Science, Artificial Intelligence
Weiwei Liu, Wei Jing, Shanqi Liu, Yudi Ruan, Kexin Zhang, Jiang Yang, Yong Liu
Summary: Collaboration is key to achieving good teamwork, but credit assignment is crucial for multi-agent cooperation. This paper proposes using expert demonstrations to guide team reward decomposition, resulting in better performance and robustness to various reward functions. Extensive experiments validate the effectiveness of the proposed method in various multi-agent environments.
NEURAL COMPUTING & APPLICATIONS
(2023)
Article
Computer Science, Artificial Intelligence
Jueming Hu, Zhe Xu, Weichang Wang, Guannan Qu, Yutian Pang, Yongming Liu
Summary: This study addresses the challenge of learning complex temporally extended tasks in multi-agent reinforcement learning by proposing a decentralized learning algorithm based on graph and reward machine. Experimental results demonstrate the effectiveness of the algorithm in several different case studies.
Article
Thermodynamics
Lele Li, Weihao Zhang, Ya Li, Chiju Jiang, Yufan Wang
Summary: The study introduces a dynamic multi-objective optimization algorithm based on multi-agent reinforcement learning to optimize the aerodynamic performance of blade profiles. The algorithm can provide real-time Pareto front under different constraints and achieve optimization speed 51 times faster than traditional algorithms.
ENERGY CONVERSION AND MANAGEMENT
(2023)
Article
Engineering, Civil
Weibin Zhang, Chen Yan, Xiaofeng Li, Liangliang Fang, Yao-Jan Wu, Jun Li
Summary: To enhance intersections' throughput efficiency, this paper proposes an adaptive coordination control method based on multi-agent reinforcement learning. The method achieved stable performance in both simulated and real-world scenarios, effectively alleviating traffic congestion.
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
(2023)
Article
Computer Science, Artificial Intelligence
Zhimin Qiao, Liangjun Ke, Xiaoqiang Wang
Summary: This paper proposes a new MARL algorithm called CoTD3-EWMA, which introduces mean-field theory and dynamic delay updating to effectively solve the challenges in urban traffic signal control and improve traffic efficiency.
APPLIED INTELLIGENCE
(2023)
Article
Computer Science, Artificial Intelligence
Xin Cao, He Luo, Jianwei Tai, Ruhao Jiang, Guoqiang Wang
Summary: This study proposes a Hierarchical Deep Q Network with Multi-criteria Negative Feedback method (MNF-HDQN) for solving the challenges of target search tasks using unmanned platforms. The MNF-HDQN enhances cooperation competence and improves the successful search rate, especially in complex search scenarios and unpredictable target movements.
APPLIED SOFT COMPUTING
(2023)
Article
Computer Science, Interdisciplinary Applications
Tong Zhou, Dunbing Tang, Haihua Zhu, Zequn Zhang
Summary: This paper discusses the use of IoT and cloud technologies to build a multi-agent system for scheduling low-volume-high-mix orders in smart factories. By implementing intelligent schedulers and reinforcement learning, the proposed methodology improves the learning and scheduling efficiency of multiple AI schedulers and effectively manages unexpected events in the manufacturing environment.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2021)
Article
Automation & Control Systems
Junjia Liu, Huimin Zhang, Zhuang Fu, Yao Wang
Summary: This study proposes a multi-agent coordination framework based on deep reinforcement learning for traffic signal control, achieving success in improving traffic efficiency. The Spatial Differentiation method is introduced to amend the reward of each action. Theoretical analysis proves the model's ability to converge to Nash equilibrium.
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE
(2021)
Article
Automation & Control Systems
Milos S. Stankovic, Marko Beko, Srdjan S. Stankovic
Summary: This paper proposes two new distributed consensus-based algorithms for temporal-difference learning in multi-agent Markov decision processes. The algorithms are off-policy type and aim to linearly approximate the value function. By restricting agents' observations and communications to their local data and small neighborhoods, the algorithms consist of local updates of parameter estimates and a dynamic consensus scheme implemented over a time-varying communication network. The algorithms are completely decentralized, allowing for efficient parallelization and applications in scenarios where agents have different behavior policies and initial state distributions while evaluating a common target policy.
Article
Automation & Control Systems
Lixiong Leng, Jingchen Li, Jinhui Zhu, Kao-Shing Hwang, Haobin Shi
Summary: RIFQ is a multi-agent reward-iteration fuzzy Q-learning method suitable for multi-agent cooperative tasks, which makes training more stable and has a faster convergence speed by reshaping reward relationships.
INTERNATIONAL JOURNAL OF FUZZY SYSTEMS
(2021)
Article
Computer Science, Artificial Intelligence
Wenqian Liang, Ji Wang, Weidong Bao, Xiaomin Zhu, Qingyong Wang, Beibei Han
Summary: In this study, a gradient-based Self-Adaptive Meta-Learning algorithm, SAML, was developed to address the challenge of continuously adapting to multiple tasks in multi-agent reinforcement learning. Experimental results showed that the method enables significantly more efficient adaptation in a new multi-task multi-agent StarCraft environment, Meta-SMAC, compared to reactive baselines.
COMPLEX & INTELLIGENT SYSTEMS
(2022)
Article
Transportation Science & Technology
Yue Zhao, Liujiang Kang, Huijun Sun, Jianjun Wu, Nsabimana Buhigiro
Summary: This study proposes a 2-population 3-strategy evolutionary game model to address the issue of subway network operation extension. The analysis reveals that the rule of maximum total fitness ensures the priority of evolutionary equilibrium strategies, and proper adjustment minutes can enhance the effectiveness of operation extension.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2024)
Article
Transportation Science & Technology
Hongtao Hu, Jiao Mob, Lu Zhen
Summary: This study investigates the challenges of daily storage yard management in marine container terminals considering delayed transshipment of containers. A mixed-integer linear programming model is proposed to minimize various costs associated with transportation and yard management. The improved Benders decomposition algorithm is applied to solve the problem effectively and efficiently.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2024)
Article
Transportation Science & Technology
Zhandong Xu, Yiyang Peng, Guoyuan Li, Anthony Chen, Xiaobo Liu
Summary: This paper studied the impact of range anxiety among electric vehicle drivers on traffic assignment. Two types of range-constrained traffic assignment problems were defined based on discrete or continuous distributed range anxiety. Models and algorithms were proposed to solve the two types of problems. Experimental results showed the superiority of the proposed algorithm and revealed that drivers with heightened range anxiety may cause severe congestion.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2024)
Article
Transportation Science & Technology
Chuanjia Li, Maosi Geng, Yong Chen, Zeen Cai, Zheng Zhu, Xiqun (Michael) Chen
Summary: Understanding spatial-temporal stochasticity in shared mobility is crucial, and this study introduces the Bi-STTNP prediction model that provides probabilistic predictions and uncertainty estimations for ride-sourcing demand, outperforming conventional deep learning methods. The model captures the multivariate spatial-temporal Gaussian distribution of demand and offers comprehensive uncertainty representations.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2024)
Article
Transportation Science & Technology
Benjamin Coifman, Lizhe Li
Summary: This paper develops a partial trajectory method for aligning views from successive fixed cameras in order to ensure high fidelity with the actual vehicle movements. The method operates on the output of vehicle tracking to provide direct feedback and improve alignment quality. Experimental results show that this method can enhance accuracy and increase the number of vehicles in the dataset.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2024)
Article
Transportation Science & Technology
Mohsen Dastpak, Fausto Errico, Ola Jabali, Federico Malucelli
Summary: This article discusses the problem of an Electric Vehicle (EV) finding the shortest route from an origin to a destination and proposes a problem model that considers the occupancy indicator information of charging stations. A Markov Decision Process formulation is presented to optimize the EV routing and charging policy. A reoptimization algorithm is developed to establish the sequence of charging station visits and charging amounts based on system updates. Results from a comprehensive computational study show that the proposed method significantly reduces waiting times and total trip duration.
TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES
(2024)