Review
Automation & Control Systems
Fadi AlMahamid, Katarina Grolinger
Summary: There is a growing demand for using drones in various applications, and autonomous UAV navigation is commonly achieved using reinforcement learning algorithms. Understanding the navigation environment and algorithm limitations is crucial in selecting the right algorithms to solve navigation problems effectively. This study identifies the main UAV navigation tasks, discusses navigation frameworks and simulation software, and classifies RL algorithms based on their environment, characteristics, abilities, and applications.
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE
(2022)
Article
Computer Science, Information Systems
Douglas Lopes Da Silva, Renato Machado, Olympio L. Coutinho, Felix Antreich
Summary: In recent years, the use of unmanned aerial vehicles (UAVs) has become increasingly popular for various purposes. However, the need for regulation and control of unauthorized UAV flights has become a pressing issue. In this study, we propose an algorithm to speed up the training of a reinforcement learning drone agent for a counter unmanned aerial system (C-UAS), which aims to guide an invading drone to a safe-killing zone (SZ) using a hunter quadrotor drone. We conducted simulations to evaluate the performance of the proposed algorithm, and the results indicate that a high probability of successful target steering to the SZ can be achieved.
Article
Computer Science, Artificial Intelligence
Chenchen Fu, Xueyong Xu, Yuntao Zhang, Yan Lyu, Yu Xia, Zining Zhou, Weiwei Wu
Summary: This paper introduces a goal-conditioned reinforcement learning framework for vision-based UAV navigation. By developing a memory-enhanced model, the proposed approach improves the success rate of navigation and reduces the training steps.
NEURAL COMPUTING & APPLICATIONS
(2022)
Article
Engineering, Aerospace
Jiacheng Li, Hanlin Sheng, Jie Zhang, Haibo Zhang
Summary: This paper addresses the coverage path planning problem of an agricultural spraying UAV by proposing a margin reduction algorithm and a topology mapping algorithm. The optimal operating heading angle is found through iterative optimization. Simulation and flight test results demonstrate that the method can significantly reduce flight distance, minimize additional coverage, and avoid energy consumption and pesticide waste.
Article
Remote Sensing
Amudhini P. Kalidas, Christy Jackson Joshua, Abdul Quadir Md, Shakila Basheer, Senthilkumar Mohan, Sapiah Sakri
Summary: This study develops a methodology for training a drone to autonomously avoid obstacles using reinforcement learning algorithms. The study compares three different reinforcement learning strategies and finds that SAC algorithm performs the best in obstacle avoidance. These findings could have practical implications for the future development of safer and more efficient drones.
Article
Computer Science, Artificial Intelligence
Shuai Liu, Yuebin Bai
Summary: This paper presents an intelligent navigation method for UAV based on deep reinforcement learning, using geographic information systems as the training environment, and combining the knowledge-based Monte Carlo tree search method and local search method. The trained UAV can find an excellent flight path by intelligent navigation and make effective flight decisions in complex geometrical environments.
INTERNATIONAL JOURNAL OF COMPUTATIONAL INTELLIGENCE SYSTEMS
(2021)
Article
Robotics
Abhisek Konar, Bobak H. Baghi, Gregory Dudek
Summary: This letter presents a learning-based solution for socially compliant navigation of mobile robots, inferring navigational policies from human examples and validating its effectiveness through comparisons with classical algorithms and reinforcement learning agents. The proposed method and feature representation are found to produce higher quality trajectories and play a critical role in successful navigation.
IEEE ROBOTICS AND AUTOMATION LETTERS
(2021)
Article
Engineering, Electrical & Electronic
Weida Wang, Jing Zhao, Chao Yang, Tianqi Qie, Ying Li, Kun Huang, Changle Xiang
Summary: In this paper, a path planning learning strategy is proposed for a wheel-legged vehicle considering distance and energy consumption. A new reward function and update rule of the Q-Learning algorithm are presented to ensure path shortening and energy consumption reduction. Furthermore, the future energy consumption is introduced into the modification of path energy consumption. The proposed strategy is verified on different size maps with 0-1m obstacle height, showing effective path shortening and energy consumption reduction compared to other strategies tested.
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY
(2023)
Article
Computer Science, Information Systems
Guoyi Sun, Qian Xu, Guangyuan Zhang, Tengteng Qu, Chengqi Cheng, Haojiang Deng
Summary: With the rapid development of the big data era, the adoption of Unmanned Aerial Vehicles (UAVs) in various complex environments is increasing, posing new challenges for UAV path planning. This paper proposes a novel environment-modeling approach based on airspace grids to efficiently organize, manage, and express data in complex scenes and intelligently carry out fast and efficient path planning for UAVs.
ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION
(2023)
Article
Engineering, Electrical & Electronic
Gunasekaran Raja, Sailakshmi Suresh, Sudha Anbalagan, Aishwarya Ganapathisubramaniyan, Neeraj Kumar
Summary: The paper introduces a Particle Filter-based Indoor Navigation (PFIN) framework for drone navigation, utilizing Quadcopter Mapping Algorithm and Optimized Localization Algorithm to enhance precision and velocity estimation. The efficiency of the algorithms is demonstrated through SITL and HITL simulations, showcasing reduced position error and improved UAV exploration precision.
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY
(2021)
Article
Chemistry, Analytical
Ibrahim A. Nemer, Tarek R. Sheltami, Slim Belhaiza, Ashraf S. Mahmoud
Summary: This paper introduces a novel distributed control solution called SBG-AC, which focuses on improving coverage score with minimum energy consumption and high fairness value by placing a group of UAVs in a candidate area. The state-based potential game is used to model the complex interactions, and the actor-critic algorithm is merged to ensure convergence and distributed control of each UAV with learning capabilities in dynamic environments. Simulation results show that SBG-AC outperforms other algorithms in terms of fairness, coverage score, and energy consumption.
Article
Engineering, Multidisciplinary
JiaNan Yang, ShengAo Lu, MingHao Han, YunPeng Li, YuTing Ma, ZeFeng Lin, HaoWei Li
Summary: This paper addresses the problem of mapless navigation for unmanned aerial vehicles in scenarios with limited sensor accuracy and computing capability. It proposes a novel learning-based algorithm called soft actor-critic from demonstrations (SACfD), which integrates reinforcement learning with imitation learning. The algorithm utilizes maximum entropy reinforcement learning framework to enhance exploration capability and leverages demonstration data to accelerate convergence rate and improve policy performance. Experimental results demonstrate that the proposed algorithm outperforms existing algorithms.
SCIENCE CHINA-TECHNOLOGICAL SCIENCES
(2023)
Article
Robotics
Qianfan Zhao, Lu Zhang, Bin He, Zhiyong Liu
Summary: The study proposes a novel approach called the Semantic Policy Network (SPNet) to address the challenge of zero-shot object goal visual navigation. The proposed method utilizes semantic embeddings to generate unique navigation policies and outperforms other methods for both seen and unseen target classes, as shown in the experimental results.
IEEE ROBOTICS AND AUTOMATION LETTERS
(2023)
Article
Chemistry, Analytical
Koppaka Ganesh Sai Apuroop, Anh Vu Le, Mohan Rajesh Elara, Bing J. Sheu
Summary: Cleaning robots must achieve complete area coverage, and tiling robots offer an innovative solution to this problem. This study proposes a complete area coverage planning module for a modified honeycomb-shaped tiling robot based on deep reinforcement learning, which simultaneously generates tiling shapes and trajectories with minimum overall cost.
Article
Telecommunications
Sipra Swain, Pabitra Mohan Khilar, Biswa Ranjan Senapati
Summary: Unmanned Aerial Vehicles (UAVs) with visual sensors are widely used for various applications such as area mapping and crop management. This paper proposes a cluster-based routing approach with a dynamic planning algorithm to tackle changing environmental situations. The approach includes modules for path planning, network topology construction, cluster management, and data routing, and achieves better performance compared to existing methods.
VEHICULAR COMMUNICATIONS
(2023)
Article
Engineering, Civil
Jianxin Zhao, Xinyu Chang, Yanhao Feng, Chi Harold Liu, Ningbo Liu
Summary: Intelligent Transportation Systems (ITS) utilize communication technologies and intelligent analytics to enhance transportation systems. Federated Learning, a privacy-preserving machine learning paradigm, holds promise in this field. However, the heterogeneity of data and devices in ITS, as well as the dynamic environment, present challenges for federated learning. One approach to address these challenges is to select participants properly during training. This paper introduces Newt, an enhanced federated learning approach that considers accuracy performance and system progress during client selection, and includes a feedback control mechanism. Experimental results demonstrate a significant performance improvement of up to 20% compared to other methods.
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
(2023)
Article
Computer Science, Artificial Intelligence
Chi Harold Liu, Yu Wang, Chengzhe Piao, Zipeng Dai, Ye Yuan, Guoren Wang, Dapeng Wu
Summary: This paper proposes a time-aware location prediction model called t-LocPred, which utilizes coarse-grained convolutional processing of user trajectories and a memory-augmented attentive LSTM model to predict next visited points of interest. Experimental results demonstrate that t-LocPred outperforms 8 baselines, and the benefits of ConvAoI to these baselines are also shown.
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING
(2022)
Article
Computer Science, Information Systems
Jianxin Zhao, Rui Han, Yongkai Yang, Benjamin Catterall, Chi Harold Liu, Lydia Y. Chen, Richard Mortier, Jon Crowcroft, Liang Wang
Summary: This paper presents a new barrier control technique called Probabilistic Synchronous Parallel (PSP) to address the challenges of synchronization methods and barrier control methods in federated learning. The research shows that PSP improves convergence speed and scalability, achieving a good balance between system efficiency and model accuracy.
IEEE TRANSACTIONS ON SERVICES COMPUTING
(2022)
Article
Computer Science, Artificial Intelligence
Binhui Xie, Shuang Li, Mingjia Li, Chi Harold Liu, Gao Huang, Guoren Wang
Summary: Domain adaptive semantic segmentation aims to achieve satisfactory predictions on an unlabeled target domain by utilizing a supervised model trained on a labeled source domain. We propose SePiCo, a novel one-stage adaptation framework that emphasizes the semantic concepts of individual pixels to improve self-training methods.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Article
Computer Science, Information Systems
Zipeng Dai, Chi Harold Liu, Rui Han, Guoren Wang, Kin K. K. Leung, Jian Tang
Summary: This paper proposes a centralized control, distributed execution framework called DRL-eFresh, which utilizes unmanned aerial vehicles (UAVs) for mobile crowdsensing (MCS) applications. The framework aims to maximize data collection, maintain geographical fairness, minimize energy consumption, and guarantee data freshness. The proposed framework utilizes decentralized deep reinforcement learning (DRL) and features a synchronous computational architecture with GRU sequential modeling. Simulation results show that DRL-eFresh improves energy efficiency compared to the best baseline by 14% and 22% on average.
IEEE TRANSACTIONS ON MOBILE COMPUTING
(2023)
Article
Engineering, Multidisciplinary
Jianxin Zhao, Yanhao Feng, Xinyu Chang, Peng Xu, Shilin Li, Chi Harold Liu, Wenke Yu, Jian Tang, Jon Crowcroft
Summary: Recently, there has been a growing interest in the sixth generation network, which aims to support a wider range of applications with higher capacity and greater coverage than existing 5G connections. One promising application is Decentralised Federated Learning, which preserves privacy in machine learning and relies on peer-to-peer mobile connections among devices. However, the heterogeneity of data and devices, as well as the dynamic nature of mobile networks, pose challenges to the performance of federated learning. In this paper, the authors propose a data redistribution phase to balance data distribution on participating devices, improving system performance in the training phase. They model the problem as a bargaining game and propose centralised and decentralised algorithms to solve it, with the latter being more energy efficient. The proposed system is evaluated through simulations and DNN training tasks on large scale FEMNIST-based datasets.
IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING
(2023)
Article
Automation & Control Systems
Shuang Li, Kaixiong Gong, Binhui Xie, Chi Harold Liu, Weipeng Cao, Song Tian
Summary: This article proposes a Critical classes and samples discovering network (CSDN) to achieve more precise cross-domain alignment by identifying the most relevant source classes and critical target samples. CSDN introduces an adaptive source class weighting scheme and target ambiguous score to enable fine-grained alignment. Extensive experiments demonstrate that CSDN achieves excellent results on four highly competitive benchmark datasets.
IEEE TRANSACTIONS ON CYBERNETICS
(2023)
Article
Computer Science, Hardware & Architecture
Hao Wang, Chi Harold Liu, Haoming Yang, Guoren Wang, Kin K. Leung
Summary: This paper proposes a deep reinforcement learning framework for UAV trajectory planning in order to maximize data collection from multiple POIs while minimizing data freshness and threshold violation ratio, considering limited energy supply.
IEEE-ACM TRANSACTIONS ON NETWORKING
(2023)
Article
Computer Science, Artificial Intelligence
Binhui Xie, Shuang Li, Fangrui Lv, Chi Harold Liu, Guoren Wang, Dapeng Wu
Summary: This paper proposes a unified framework called Collaborative Alignment Framework (CAF), which reduces global domain discrepancy and preserves local semantic consistency for cross-domain knowledge transfer. It utilizes adversarial training or Wasserstein distance to learn domain-level invariant representations and enhances the features of target data to be consistent with the source data class-wise. Experimental results demonstrate the superiority of the proposed methods over existing methods on popular benchmarks.
IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING
(2023)
Article
Geochemistry & Geophysics
Ying Zhao, Shuang Li, Chi Harold Liu, Yuqi Han, Hao Shi, Wei Li
Summary: Scene recognition in remote sensing has gained increasing attention due to advances in remote sensing devices. However, the domain shift problem caused by the diverse sensor-specific characteristics of images obtained from various sensors weakens the transferability of models trained on one data domain to a different target domain. To address this challenge, we propose an adaptive remote sensing scene recognition network that transfers both discriminative knowledge and cross-scene relationship from source to target. Our approach aligns the distributions of different domains, discovers semantic relationships between scenes, and achieves superior performance on remote sensing benchmarks.
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
(2023)
Article
Engineering, Electrical & Electronic
Jianxin Zhao, Pierre Vandenhove, Peng Xu, Hao Tao, Liang Wang, Chi Harold Liu, Jon Crowcroft
Summary: In this paper, a parallel training method is proposed using operators as scheduling units, and a pebble-game-based memory-efficient optimization in training is discussed. Experiments show the flexibility and good performance of the proposed method.
IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING
(2023)
Proceedings Paper
Computer Science, Hardware & Architecture
Guyue Li, Haiyu Yang, Junqing Zhang, Hongbo Liu, Aiqun Hu
Summary: The research proposes a physical-layer secret key generation approach with channel obfuscation, which enhances the key generation rate and resilience to attacks by improving the dynamic properties of channel parameters. It can generate high entropy bits at a faster rate and pass randomness tests in NIST test suite.
IEEE CONFERENCE ON COMPUTER COMMUNICATIONS (IEEE INFOCOM 2022)
(2022)
Proceedings Paper
Computer Science, Artificial Intelligence
Tong Wu, Guangyu Gao, Junshi Huang, Xiaolin Wei, Xiaoming Wei, Chi Harold Liu
Summary: This paper proposes an adaptive Spatial Binary Cross-Entropy (Spatial-BCE) Loss for weakly-supervised semantic segmentation, aiming to enhance the discrimination between pixels. By calculating the loss independently for each pixel and assigning optimization directions for foreground and background pixels separately, as well as designing an alternate training strategy to generate thresholds for foreground and background, high-quality initial pseudolabels are generated, reducing the reliance on post-processing.
COMPUTER VISION, ECCV 2022, PT XXIX
(2022)
Article
Computer Science, Artificial Intelligence
Ying Zhao, Shuang Li, Rui Zhang, Chi Harold Liu, Weipeng Cao, Xizhao Wang, Song Tian
Summary: A novel semantic correlation transfer (SCT) method is proposed for heterogeneous domain adaptation (HDA), aiming to improve both feature transferability and category discriminability across domains.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2022)
Article
Computer Science, Artificial Intelligence
Guangyu Gao, Zhiyuan Fang, Cen Han, Yunchao Wei, Chi Harold Liu, Shuicheng Yan
Summary: DRNet proposes a Double Recalibration Network with two recalibration modules to enhance model robustness against intra-class variance. The method can more accurately mine target regions for query images.
IEEE TRANSACTIONS ON IMAGE PROCESSING
(2022)