Article
Engineering, Electrical & Electronic
Wenxing Zhu, Lihui Wang, Liangliang Chen, Ninghui Xu, Yuzuwei Su
Summary: This research proposes a visual-inertial calibration method using deep deterministic policy gradient learning. By analyzing nonlinear observability and establishing a relationship model, it achieves the self-calibration process of visual-inertial systems, and solves the problems of hyperparameter training and network instability through a reinforcement learning network model.
IEEE SENSORS JOURNAL
(2022)
Article
Computer Science, Artificial Intelligence
Shuai Han, Wenbo Zhou, Shuai Lu, Jiayu Yu
Summary: This paper introduces a new reinforcement learning algorithm RUD to address the inefficiency and instability of DDPG, demonstrating that RUD can better utilize new data and is more suitable for a specific strategy in terms of Q value variance. The experiments validate the effectiveness and superiority of RUD.
KNOWLEDGE-BASED SYSTEMS
(2021)
Article
Chemistry, Multidisciplinary
Tianhan Gao, Shen Gao, Jun Xu, Qihui Zhao
Summary: Recommendation systems, as an important branch of artificial intelligence, have become increasingly significant in people's daily lives. In this study, we propose a recommendation framework called DDRCN, which incorporates deep cross networks to model the cross relationships between data and obtain a representation of user interaction data. Our experiments show that our proposed framework outperforms the baseline approach in the recall and ranking phases of recommendations.
APPLIED SCIENCES-BASEL
(2023)
Article
Mathematics
Ibai Inziarte-Hidalgo, Erik Gorospe, Ekaitz Zulueta, Jose Manuel Lopez-Guede, Unai Fernandez-Gamiz, Saioa Etxebarria
Summary: This research builds upon previous work on robotic-arm-based force control in neurosurgical practice and proposes the use of reinforcement learning to create an agent capable of self-training for optimal solutions. The study draws conclusions for potential future enhancements and identifies areas for improvement in the analysis of results.
Article
Energy & Fuels
Cephas Samende, Jun Cao, Zhong Fan
Summary: This paper investigates the energy cost minimization problem for prosumers participating in peer-to-peer energy trading. A multi-agent deep deterministic policy gradient algorithm is proposed to learn optimal energy trading decisions, while distribution network tariffs are introduced to satisfy the distribution network constraints.
Article
Chemistry, Multidisciplinary
Sheng Yu, Wei Zhu, Yong Wang
Summary: Wargames have become essential for simulating different war scenarios, but traditional decision-making methods are no longer effective. To address this, a wargame decision-making method based on Multi-Agent Deep Deterministic Policy Gradient (MADDPG) is proposed. The method leverages techniques like Partially Observable Markov Decision Process (POMDP) and Gumbel-Softmax estimator to optimize the MADDPG algorithm for the wargame environment.
APPLIED SCIENCES-BASEL
(2023)
Article
Computer Science, Interdisciplinary Applications
Tanuja Joshi, Shikhar Makker, Hariprasad Kodamana, Harikumar Kandath
Summary: Control of batch processes is a challenging task due to their complex dynamics and non-steady state operating conditions. Developing control strategies that directly interact with the process and learning from experiences can help address some of these challenges. The study introduces a novel actor-critic RL algorithm and demonstrates its efficacy in various batch process examples.
COMPUTERS & CHEMICAL ENGINEERING
(2021)
Article
Engineering, Chemical
Chunyu Deng, Kehe Wu
Summary: This study proposed an optimization model of residential demand response strategy based on the DDPG algorithm using deep reinforcement learning, which can be trained under different incentive policies to maximize the benefits of different participants. The results demonstrated good optimization performance of the model in practical applications, showing promising prospects for achieving the overall goal of the electricity market.
Article
Computer Science, Information Systems
Ming Zhan, Jingjing Fan, Jianying Guo
Summary: This study proposes a generative adversarial inverse reinforcement learning algorithm based on deep deterministic policy gradient (DDPG). By replacing the random noise input of the initial GAN model with a deterministic strategy and reconstructing the generator based on the Actor-Critic mechanism, the quality of GAN-generated samples is improved. The mixture of GAN-generated virtual samples and original expert samples solves the problem of sparse expert samples and improves the decision-making process efficiency of IRL under GAN.
Article
Computer Science, Artificial Intelligence
Xi Hu, Yang Huang
Summary: Task offloading decision is a core technology in vehicular edge computing. This article proposes a deep reinforcement learning-based algorithm for task offloading decision in VEC. The algorithm takes into consideration the mobility of vehicles and signal blocking, and uses an improved deep deterministic policy gradient algorithm for model training to obtain optimal decisions. Experimental results show that the proposed algorithm performs better than other baseline algorithms.
PEERJ COMPUTER SCIENCE
(2022)
Article
Computer Science, Artificial Intelligence
Dong Xie, Xiangnan Zhong
Summary: In this article, a novel semicentralized deep deterministic policy gradient (SCDDPG) algorithm is proposed for cooperative multiagent games. The algorithm utilizes a two-level actor-critic structure to facilitate interactions and cooperation among agents in StarCraft combat. The local and global actor-critic structures work together to generate optimal control actions and improve cooperation in the games.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2022)
Article
Construction & Building Technology
Huixue Wang, Yunzhe Wang, You Lu, Qiming Fu, Jianping Chen
Summary: This paper introduces a visual analytics system called DDPGVis, which explores the experience data generated by Deep Deterministic Policy Gradient (DDPG) models for energy consumption prediction. The system uses temporal aggregation and feature importance analysis to help users understand the high-dimensional environment space. DDPGVis also provides a recommendation view for parameter tuning. Experimental results show that DDPGVis can help users understand the model training process and provide optimization recommendations.
JOURNAL OF BUILDING ENGINEERING
(2023)
Article
Engineering, Electrical & Electronic
Jiying Wu, Zhong Yang, Luwei Liao, Naifeng He, Zhiyong Wang, Can Wang
Summary: This paper proposes a control algorithm for UAV trajectory tracking, which achieves efficient training and stable convergence in unknown environments by establishing an MDP model and introducing a compensation network. Simulation results show that the algorithm significantly improves training efficiency and accuracy, achieving lower tracking error in tracking experiments.
Article
Computer Science, Artificial Intelligence
Ashkan Haji Hosseinloo, Munther A. Dahleh
Summary: This paper extends the DPG theorem from MDPs to SMDPs under the average-reward criterion, and presents two example actor-critic algorithms that demonstrate the efficacy of the method both mathematically and via simulations.
INTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS
(2022)
Article
Computer Science, Information Systems
Xiaohu Gao, Mei Choo Ang, Sara A. Althubiti
Summary: Mobile Edge Computing (MEC) is an up-and-coming method for advancing the Internet of Things (IoT) by offering cloud-like capabilities to mobile users. This research proposes a novel approach that integrates Deep Reinforcement Learning (DRL) with Deep Deterministic Policy Gradient (DDPG) and Markov Decision Problem for task offloading in MEC. The experimental results demonstrate that the proposed approach outperforms the baseline algorithms in terms of average compensation and convergence speed. Our approach delivers improved performance and can learn complex non-linear policies, making it highly effective for developing IoT environments.
JOURNAL OF GRID COMPUTING
(2023)
Article
Automation & Control Systems
Xu Zhang, Geyong Min, Qilin Fan, Hao Yin, Dapeng Wu, Zhan Ma
Summary: This article introduces a lightweight statistical latency measurement platform called DMS, which predicts end-to-end latency between hosts by introducing a metric space and measuring latency between DNS servers. DMS achieves low measurement cost and good scalability by clustering hosts with DNS infrastructure.
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
(2022)
Article
Biochemical Research Methods
Mohammad A. Rezaei, Yanjun Li, Dapeng Wu, Xiaolin Li, Chenglong Li
Summary: This paper introduces a data-driven framework called DeepAtom for accurately predicting protein-ligand binding affinity. By utilizing a 3D Convolutional Neural Network (3D-CNN) architecture, DeepAtom can automatically extract atomic interaction patterns related to binding. Experiment results demonstrate that the DeepAtom approach outperforms other methods in baseline scoring and can potentially be adopted in computational drug development protocols.
IEEE-ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS
(2022)
Article
Computer Science, Hardware & Architecture
Ning Chen, Tie Qiu, Zilong Lu, Dapeng Oliver Wu
Summary: The Internet of Things consists of numerous sensing nodes forming a large scale-free network. Optimizing network topology to increase resistance against malicious attacks is complex. Traditional genetic algorithms lack global search ability and may lead to premature convergence, slowing down population evolution. Therefore, an Adaptive Robustness Evolution Algorithm (AREA) with self-competition mechanism is proposed to address this issue.
IEEE-ACM TRANSACTIONS ON NETWORKING
(2022)
Article
Computer Science, Artificial Intelligence
Jiaxin Liu, Xucheng Song, Yingjie Zhou, Xi Peng, Yanru Zhang, Pei Liu, Dapeng Wu, Ce Zhu
Summary: With the wide deployment of edge devices, it is important and challenging to detect packet payload anomalies for the safe and efficient operations of edge applications. However, existing approaches have limitations in detecting anomalies with long-term dependency relationships and rely on in-depth expert knowledge. To overcome these limitations, a deep learning-based framework is proposed, which consists of a block sequence construction method and a detection model based on LSTM, CNN, and Multi-head Self Attention. Experimental results show that the proposed model achieves a higher detection rate and a lower false positive rate compared to traditional and state-of-the-art methods.
Article
Computer Science, Information Systems
Miaomiao Liu, Kang Yang, Yanjie Fu, Dapeng Wu, Wan Du
Summary: This paper proposes GeoDMA, which utilizes GPS data from multiple vehicles to detect anomalous driving maneuvers. The approach includes designing an unsupervised deep auto-encoder to learn unique features from normal GPS data, developing a geographical partitioning algorithm to incorporate peer dependency of drivers, and training specific driving anomaly models for each sub-region. The experimental results show that GeoDMA outperforms baseline methods with up to 8.5% higher detection accuracy.
ACM TRANSACTIONS ON SENSOR NETWORKS
(2023)
Article
Computer Science, Artificial Intelligence
LiChao Su, Mengqing Cao, Yue Yu, Jian Chen, XiuZhi Yang, Dapeng Wu
Summary: This paper proposes an in-loop filtering algorithm based on a dynamic convolutional capsule network, which adapt well to local features and improves the efficiency of video coding, and shows outstanding performance in terms of time efficiency.
IET IMAGE PROCESSING
(2023)
Article
Engineering, Electrical & Electronic
Jingxuan Chen, Xianbin Cao, Peng Yang, Meng Xiao, Siqiao Ren, Zhongliang Zhao, Dapeng Oliver Wu
Summary: This paper addresses the resource allocation problem in a multi-UAV-aided uplink communication scenario, aiming to minimize the total system latency and energy consumption while satisfying constraints on transmit power and system latency caused by transmission and computation. The proposed UMAP algorithm optimizes UAV movement, MU association, and MU power control iteratively. Simulation results demonstrate that the UMAP algorithm effectively reduces system latency and energy consumption, and improves coverage rate compared to benchmark algorithms.
IEEE TRANSACTIONS ON COMMUNICATIONS
(2023)
Article
Computer Science, Information Systems
Xiaoqiang Zhu, Tie Qiu, Wenyu Qu, Xiaobo Zhou, Mohammed Atiquzzaman, Dapeng Oliver Wu
Summary: This paper presents a novel indoor wireless fingerprint localization algorithm based on a broad learning system, which utilizes channel state information to overcome the problems of data loss, noise interference, and time-consuming offline training. Experimental results show that the algorithm outperforms several machine learning algorithms and existing methods in terms of training time reduction and accuracy.
IEEE TRANSACTIONS ON MOBILE COMPUTING
(2023)
Article
Computer Science, Artificial Intelligence
Kunpeng Liu, Dongjie Wang, Wan Du, Dapeng Oliver Wu, Yanjie Fu
Summary: In this paper, a single-agent Monte Carlo-based reinforced feature selection method is proposed, along with two efficiency improvement strategies: early stopping strategy and reward-level interactive strategy. The proposed method aims to find the optimal feature subset for a given machine learning task by traversing the feature set and selecting features one by one. Additionally, the early stopping strategy and reward-level interactive strategy are introduced to enhance the training efficiency.
KNOWLEDGE AND INFORMATION SYSTEMS
(2023)
Article
Computer Science, Hardware & Architecture
Haojun Huang, Zhaoxi Li, Jialin Tian, Geyong Min, Wang Miao, Dapeng Oliver Wu
Summary: This paper proposes an approach for accurately predicting the required virtual resources using deep reinforcement learning. By leveraging the inherent features hidden in network traffic and consolidating high-dimensional resources into a standardized value, the approach minimizes prediction errors through DRL-based matrix factorization. Experimental results demonstrate the accurate prediction of required virtual resources.
IEEE-ACM TRANSACTIONS ON NETWORKING
(2023)
Article
Computer Science, Artificial Intelligence
Zhenxing Zheng, Gaoyun An, Shan Cao, Dapeng Wu, Qiuqi Ruan
Summary: In this article, a novel collaborative and multilevel feature selection network (FSNet) is proposed for action recognition. FSNet can adaptively aggregate multilevel features into a new informative feature from both position and channel dimensions, enhancing the representational ability of existing networks.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Telecommunications
Hailin Cao, Wang Zhu, Zhengchuan Chen, Zhiwei Sun, Dapeng Oliver Wu
Summary: In this paper, we investigate priority-oriented UAV-aided time-sensitive data collection problems in an IoT network with movable SNs. We propose a novel autofocusing heuristic trajectory planning algorithm based on reinforcement learning (AHTP-RL) to minimize the energy consumed by a UAV and the average delay of different SNs through optimizing the trajectory of the UAV. Extensive simulations results demonstrate that the proposed AHTP-RL algorithm can achieve a superior balance between the communication delay and energy consumption.
IEEE TRANSACTIONS ON GREEN COMMUNICATIONS AND NETWORKING
(2023)
Article
Telecommunications
Zhong Tian, Jing Wang, Zhengchuan Chen, Min Wang, Yunjian Jia, Dapeng O. Wu
Summary: This letter proposes a hybrid deployment scheme of terrestrial and aerial reconfigurable intelligent surfaces (RISs) to overcome line-of-sight (LoS) communication obstruction in dense urban areas. The objective is to maximize the minimum average rate of users in different blind zones by jointly optimizing transmit beamforming at the base station, the reflecting coefficients of RISs, and the height deployment of the aerial RIS. An effective iterative method is proposed to solve the non-convex problem. Numerical simulations demonstrate the effectiveness of joint optimization on the passive beamforming of multi-hierarchical RISs and the height deployment of the aerial RIS, showing clear performance improvement over benchmark schemes.
IEEE COMMUNICATIONS LETTERS
(2023)
Article
Computer Science, Information Systems
Yanlin Zhou, Xiyao Ma, Dapeng Wu, Xiaolin Li
Summary: Federated Edge Learning is important for the development of cloud computing. However, current algorithms have high communication costs, while the proposed Distilled One-Shot Federated Learning method reduces the cost significantly while maintaining high performance.
IEEE TRANSACTIONS ON CLOUD COMPUTING
(2023)
Article
Computer Science, Artificial Intelligence
Xiang Fang, Yuchong Hu, Pan Zhou, Dapeng Oliver Wu
Summary: The paper proposes a novel Unbalanced Incomplete Multi-view Clustering method (UIMC) based on view evolution, which effectively addresses the issue of unbalanced incompleteness among different views through weighted multi-view subspace clustering and low-rank representation design.
IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE
(2022)