Article
Engineering, Civil
Zikai Feng, Mengxing Huang, Di Wu, Edmond Q. Q. Wu, Chau Yuen
Summary: Unmanned aerial vehicle (UAV)-assisted communication is a significant technology in 6G communication. Model-free multi-agent reinforcement learning (MARL) algorithm is used to optimize decision-making and achieve optimal mobile strategies for UAV, ground users, and aerial jammer, leading to maximum cumulative rewards for each entity.
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
(2023)
Article
Engineering, Electrical & Electronic
Zhenquan Qin, Zhonghao Liu, Guangjie Han, Chuan Lin, Linlin Guo, Ling Xie
Summary: The study proposed a distributed UAV-BS control approach based on multi-agent deep reinforcement learning, which can improve the fairness of communication service by sacrificing a small amount of throughput. By designing the trajectory of UAV-BSs, it addressed the fairness issue at user-level and achieved weighted-throughput maximization.
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY
(2021)
Article
Chemistry, Analytical
Qilong Wu, Zitao Geng, Yi Ren, Qiang Feng, Jilong Zhong
Summary: This paper presents a deep reinforcement learning-based distributed reconfiguration strategy for optimizing the redeployment of multi-UAVs, aiming to improve swarm performance. By developing a multi-agent deep reinforcement learning framework, a two-layer reconfiguration between the swarm and single groups is achieved. The effectiveness of the proposed method as a high-quality reconfiguration strategy for large-scale scenarios is demonstrated through Python simulations and a case study.
Article
Computer Science, Artificial Intelligence
Shaorong Xie, Han Zhang, Hang Yu, Yang Li, Zhenyu Zhang, Xiangfeng Luo
Summary: This study proposes a novel information sharing model in multi-agent reinforcement learning, which combines observation information and topology information to generate superior cooperative strategies that adapt to partially observable environments.
KNOWLEDGE-BASED SYSTEMS
(2022)
Review
Chemistry, Multidisciplinary
Lorenzo Canese, Gian Carlo Cardarilli, Luca Di Nunzio, Rocco Fazzolari, Daniele Giardino, Marco Re, Sergio Spano
Summary: This review presents an analysis of the most commonly used multi-agent reinforcement learning algorithms, starting from single-agent algorithms and extending to multi-agent scenarios. The algorithms are grouped based on their features and a detailed taxonomy of main approaches proposed in literature is provided, focusing on mathematical models. Each algorithm is described in terms of possible application fields, pros and cons, and compared based on important characteristics such as nonstationarity, scalability, and observability. Benchmark environments used to evaluate the methods' performances are also discussed.
APPLIED SCIENCES-BASEL
(2021)
Article
Computer Science, Information Systems
Zhenhui Ye, Ke Wang, Yining Chen, Xiaohong Jiang, Guanghua Song
Summary: In this paper, a deep reinforcement learning (DRL) based control solution is proposed to navigate a swarm of unmanned aerial vehicles (UAVs) in an unexplored target area under partial observation, serving as Mobile Base Stations (MBSs) for optimal communication coverage. A novel network architecture called Deep Recurrent Graph Network (DRGN) is introduced to handle information loss and obtain spatial information through inter-UAV communication. Based on DRGN and maximum-entropy learning, a stochastic DRL policy named Soft Deep Recurrent Graph Network (SDRGN) is proposed. Extensive experiments demonstrate the superior performance and scalability of SDRGN compared to state-of-the-art approaches.
IEEE TRANSACTIONS ON MOBILE COMPUTING
(2023)
Article
Engineering, Aerospace
Shaowei Li, Yongchao Wang, Yaoming Zhou, Yuhong Jia, Hanyue Shi, Fan Yang, Chaoyue Zhang
Summary: Multiple unmanned aerial vehicle (multi-UAV) cooperative air combat is an important form of future air combat, and it requires high autonomy and cooperation among UAVs. This paper proposes a multi-agent double-soft actor-critic (MADSAC) algorithm for solving the cooperative decision-making problem of multi-UAV. The MADSAC approach treats the problem as a fully cooperative game and utilizes decentralized partially observable Markov decision process and centrally trained distributed execution framework.
Article
Automation & Control Systems
Feilong Jiang, Minqiang Xu, Yuqing Li, Hutao Cui, Rixin Wang
Summary: In this paper, we propose a multi-agent Transformer network structure, introducing virtual objects, to address the challenge of making accurate air combat decisions based on complex situation information in UAV swarm air combat. The proposed method utilizes self-attention to calculate the local situation information of each UAV, reducing the difficulty of processing UAV swarm situation information. By adding a virtual object and weighted fusion of local situations, a more effective representation of the global situation is obtained, leading to more accurate air combat maneuver decisions.
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE
(2023)
Article
Computer Science, Information Systems
Jian Xiao, Guohui Yuan, Jinhui He, Kai Fang, Zhuoran Wang
Summary: To address the poor performance of reinforcement learning (RL) in multi-agent flocking cooperative control under communication-restricted environments, a distance graph attention (GAT) mechanism is introduced into a multi-agent cooperative RL (MACRL) method. This mechanism changes the attention weights of agents in the flocking task related to neighbors and reduces the influence of remote neighbors with poor communication quality on the agent's behavioral decision-making. Furthermore, a distance GAT-based MACRL (DGAT-MACRL) algorithm is proposed for multi-agent flocking control in communication-restricted environments.
INFORMATION SCIENCES
(2023)
Article
Computer Science, Artificial Intelligence
Marco Crespi, Andrea Ferigo, Leonardo Lucio Custode, Giovanni Iacca
Summary: Multi-Agent Reinforcement Learning (MARL) has made significant progress in the past decade, but the lack of interpretability in Deep Neural Networks (DNNs) poses a challenge, especially in MARL applications. This work proposes a population-based algorithm that combines evolutionary principles with RL to train interpretable models in multi-agent systems. The proposed approach is evaluated in a highly dynamic task and demonstrates effective policies that are easy to inspect and interpret based on domain knowledge.
APPLIED SOFT COMPUTING
(2023)
Article
Computer Science, Artificial Intelligence
Corban Rivera, Edward Staley, Ashley Llorens
Summary: This article introduces a new artificial intelligence framework, AI Arena, designed to address the issue of learning in complex operating environments. The framework emphasizes the importance of curriculum design and measures the impact of multi-agent learning paradigms on the emergence of cooperation.
FRONTIERS IN NEUROROBOTICS
(2022)
Article
Automation & Control Systems
Zhixiao Sun, Haiyin Piao, Zhen Yang, Yiyang Zhao, Guang Zhan, Deyun Zhou, Guanglei Meng, Hechang Chen, Xing Chen, Bohao Qu, Yuanjie Lu
Summary: The researchers proposed a novel Multi-Agent Hierarchical Policy Gradient algorithm (MAHPG), capable of learning various strategies and surpassing expert cognition through adversarial self-play learning. The algorithm adopts a hierarchical decision network to handle complex and hybrid actions, similar to human decision-making ability, effectively reducing action ambiguity. Experimental results demonstrate that MAHPG excels in defense and offense ability compared to state-of-the-art air combat methods.
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE
(2021)
Article
Management
Hyun-Rok Lee, Taesik Lee
Summary: This study addresses the issue of multiple decision-makers in disaster response using a decentralized-partially observable Markov decision process model. The proposed MARL algorithm augmented by pretraining neural network shows effectiveness and advantages in solving dec-POMDP problems.
EUROPEAN JOURNAL OF OPERATIONAL RESEARCH
(2021)
Article
Computer Science, Artificial Intelligence
Zhiqiang Pu, Huimu Wang, Zhen Liu, Jianqiang Yi, Shiguang Wu
Summary: In this article, a novel method called AERL is proposed to address complex interaction and communication issues in multi agent cooperation. The method utilizes attention mechanism and parameter sharing to achieve effective and robust performance, as demonstrated in simulation experiments in three representative scenarios.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Computer Science, Artificial Intelligence
Shantian Yang, Bo Yang, Zhongfeng Kang, Lihui Deng
Summary: In multi-intersection traffic signal control, current MDRL algorithms have drawbacks in terms of transferability and flexibility in handling time-varying number of vehicles.
