4.7 Article

Incentive-Driven Deep Reinforcement Learning for Content Caching and D2D Offloading

期刊

出版社

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/JSAC.2021.3087232

关键词

Device-to-device communication; Telecommunication network management; Mobile nodes; Cellular networks; Reinforcement learning; Optimization; Data models; D2D offloading; deep reinforcement learning; reverse auction; content caching; real mobility trace

资金

  1. National Natural Science Foundation of China (NSFC) [61872221]
  2. National Science Foundation (NSF) [CNS 1824440, CNS 1828363, CNS 1757533, CNS 1629746, CNS 1564128]

向作者/读者索取更多资源

An Incentive-driven and Deep Q Network (DQN) based Method, named IDQNM, utilizes a reverse auction as an incentive mechanism to motivate nodes to participate in D2D offloading and content caching in order to maximize the CSP's saving cost.
Offloading cellular traffic via Device-to-Device communication (or D2D offloading) has been proved to be an effective way to ease the traffic burden of cellular networks. However, mobile nodes may not be willing to take part in D2D offloading without proper financial incentives since the data offloading process will incur a lot of resource consumption. Therefore, it is imminent to exploit effective incentive mechanisms to motivate nodes to participate in D2D offloading. Furthermore, the design of the content caching strategy is also crucial to the performance of D2D offloading. In this paper, considering these issues, a novel Incentive-driven and Deep Q Network (DQN) based Method, named IDQNM is proposed, in which the reverse auction is employed as the incentive mechanism. Then, the incentive-driven D2D offloading and content caching process is modeled as Integer Non-Linear Programming (INLP), aiming to maximize the saving cost of the Content Service Provider (CSP). To solve the optimization problem, the content caching method based on a Deep Reinforcement Learning (DRL) algorithm, named DQN is proposed to get the approximate optimal solution, and a standard Vickrey-Clarke-Groves (VCG)-based payment rule is proposed to compensate for mobile nodes' cost. Extensive real trace-driven simulation results demonstrate that the proposed IDQNM greatly outperforms other baseline methods in terms of the CSP's saving cost and the offloading rate in different scenarios.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.7
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

Article Computer Science, Theory & Methods

Accelerating Deep Learning Inference via Model Parallelism and Partial Computation Offloading

Huan Zhou, Mingze Li, Ning Wang, Geyong Min, Jie Wu

Summary: With the rapid development of IoT and deep learning, there is an urgent need to enable deep learning inference on IoT devices in MEC. To address the computation limitation, computation offloading is proposed as a promising approach. This paper proposes a novel FL-based DNN model parallelism method to accelerate inference by converting a DNN layer into several smaller layers for increased offloading flexibility.

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS (2023)

Article Computer Science, Theory & Methods

Topology-Aware Scheduling Framework for Microservice Applications in Cloud

Xin Li, Junsong Zhou, Xin Wei, Dawei Li, Zhuzhong Qian, Jie Wu, Xiaolin Qin, Sanglu Lu

Summary: Loosely coupled and highly cohesive microservices running in containers have become the new paradigm for application development. Compared to monolithic applications, microservices architecture allows for independent deployment and scaling, promising to simplify software development and operation. However, the increase in microservices scale and east-west network traffic in data centers has made cluster management more complex. This paper proposes a Microservice-Oriented Topology-Aware Scheduling Framework (MOTAS) that optimizes the network overhead of microservice applications by effectively utilizing microservices and cluster topologies through a heuristic graph mapping algorithm. The framework also guarantees cluster resource utilization and incorporates a mechanism for detecting and handling QoS violations in dynamic microservice environments.

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS (2023)

Article Engineering, Electrical & Electronic

Predictive and Adaptive Deep Coding for Wireless Image Transmission in Semantic Communication

Wenyu Zhang, Haijun Zhang, Hui Ma, Hua Shao, Ning Wang, Victor C. M. Leung

Summary: This paper proposes a predictive and adaptive deep coding (PADC) framework that achieves flexible code rate optimization with a given target transmission quality requirement. By using a variable code length enabled DeepJSCC model, an Oracle Network model, and a CR optimizer, PADC can minimize bandwidth consumption while guaranteeing the PSNR constraint for each image data in wireless image transmission tasks.

IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS (2023)

Article Engineering, Electrical & Electronic

Network Topology Inference Based on Timing Meta-Data

Wenbo Du, Tao Tan, Haijun Zhang, Xianbin Cao, Gang Yan, Osvaldo Simeone

Summary: A set of low-cost sensors is used to infer the network topology of a self-organizing wireless network by extracting timing information from data packets and acknowledgment (ACK) packets. A new EM-based algorithm, called EM-CDA, is introduced to handle the impact of packet losses on causality metrics. Extensive experiments on the NS-3 simulation platform validate the effectiveness of the method.

IEEE TRANSACTIONS ON COMMUNICATIONS (2023)

Article Computer Science, Hardware & Architecture

Outlier-Concerned Data Completion Exploiting Intra-and Inter-Data Correlations in Sparse CrowdSensing

En Wang, Mijia Zhang, Wenbin Liu, Haoyi Xiong, Bo Yang, Yongjian Yang, Jie Wu

Summary: Mobile CrowdSensing (MCS) is a popular data collection paradigm that often faces the issue of sparse sensed data. To address this problem, sparse MCS recruits users to sense important areas and completes the data through data completion. However, in real-world scenarios, there are various types of data that can complement each other, including important outliers. Detecting and recovering these outliers poses challenges due to their infrequency and the complex spatiotemporal relations among the data.

IEEE-ACM TRANSACTIONS ON NETWORKING (2023)

Article Computer Science, Information Systems

Edge Resource Pricing and Scheduling for Blockchain: A Stackelberg Game Approach

Sijie Huang, He Huang, Guoju Gao, Yu-E Sun, Yang Du, Jie Wu

Summary: Blockchain is a distributed ledger system used in Bitcoin to protect transaction histories. In the mining process, high computing power is required, making it hard to implement on mobile devices. This article proposes a more realistic scenario where edge/cloud service providers have different propagation delays, and analyzes the pricing and scheduling problem in a three-stage multi-leader multi-follower Stackelberg game to achieve equilibrium. Extensive simulations demonstrate the effectiveness of the proposed solution.

IEEE TRANSACTIONS ON SERVICES COMPUTING (2023)

Article Computer Science, Information Systems

UAV-Aided Computation Offloading in Mobile-Edge Computing Networks: A Stackelberg Game Approach

Huan Zhou, Zhenning Wang, Geyong Min, Haijun Zhang

Summary: This article investigates a UAV-aided mobile-edge computing network for computation offloading to provide additional computation capability and wide coverage for mobile users. It proposes a game model and a gradient-based algorithm to achieve the maximization of system utility.

IEEE INTERNET OF THINGS JOURNAL (2023)

Article Computer Science, Information Systems

Cooperative Resource Allocation Based on Soft Actor-Critic With Data Augmentation in Cellular Network

Yunhui Qin, Zhongshan Zhang, Huangfu Wei, Haijun Zhang, Keping Long

Summary: This letter investigates the cooperative resource allocation of cellular networks with simultaneous wireless information and power transfer in the time-varying channel environment. The soft actor-critic (SAC) algorithm is exploited to tackle the optimization problem which aims to find a feasible resource allocation policy to maximize the data rate and system fairness while minimizing the channel switching penalty. Considering the costly agent-to-environment interactions and the restricted empirical dataset of the SAC algorithm, this letter explores the permutation equivalence of the optimization objective, and designs two data augmentation schemes for the experience replay buffer of SAC. The cumulative discount reward shows that data augmentation assisted algorithms outperform the baseline in the learning speed. The simulation results referring to the average data rate and system fairness show that the proposed schemes benefit to the training model and effectively improve the performance of algorithms.

IEEE WIRELESS COMMUNICATIONS LETTERS (2023)

Article Computer Science, Information Systems

Edge AI as a Service: Configurable Model Deployment and Delay-Energy Optimization With Result Quality Constraints

Wenyu Zhang, Sherali Zeadally, Wei Li, Haijun Zhang, Jingyi Hou, Victor C. M. Leung

Summary: The breakthrough of AI techniques has accelerated their applications in various industries, including security protection, transportation, agriculture, and medical care. With the support of edge computing environments, providing AIaaS with latency guarantee can speed up the deployment of data-intensive and computation-intensive AI applications and reduce customers' investment cost. However, existing studies have not addressed the specific deployment architecture, working mechanism design, and performance optimization problems for AIaaS with configurable data quality and model complexity. To tackle this, we propose a configurable model deployment architecture (CMDA) for edge AIaaS and a flexible working mechanism that allows joint configuration of data quality ratios (DQRs) and model complexity ratios (MCRs) for AI tasks.

IEEE TRANSACTIONS ON CLOUD COMPUTING (2023)

Article Engineering, Electrical & Electronic

Covert Federated Learning via Intelligent Reflecting Surfaces

Jie Zheng, Haijun Zhang, Jiawen Kang, Ling Gao, Jie Ren, Dusit Niyato

Summary: In this paper, we propose a method to support covert over-the-air computation (OAC) using intelligent reflecting surfaces (IRS). By optimizing the joint problem and designing a covert difference-of-convex-functions program (CDC), we can select the maximum covert devices participating in model aggregation while satisfying the mean squared error (MSE) requirement. Simulation results demonstrate that using the IRS in covert OAC can achieve significant performance gain compared to baseline algorithms.

IEEE TRANSACTIONS ON COMMUNICATIONS (2023)

Article Engineering, Electrical & Electronic

DRL-Driven Dynamic Resource Allocation for Task-Oriented Semantic Communication

Haijun Zhang, Hongyu Wang, Yabo Li, Keping Long, Arumugam Nallanathan

Summary: This paper proposes a dynamic resource allocation scheme for task-oriented semantic communication networks based on deep reinforcement learning, which allows data with richer semantic information to preferentially occupy limited communication resources to improve long-term transmission efficiency.

IEEE TRANSACTIONS ON COMMUNICATIONS (2023)

Article Engineering, Electrical & Electronic

Joint UAV Placement Optimization, Resource Allocation, and Computation Offloading for THz Band: A DRL Approach

Heng Wang, Haijun Zhang, Xiangnan Liu, Keping Long, Arumugam Nallanathan

Summary: Due to limited computation capacity of wireless user devices, multi-access edge computing (MEC) has become an effective way to meet the real-time demands. To increase system capacity, a UAV-assisted computation offloading architecture in the terahertz (THz) band is proposed. Deep reinforcement learning (DRL) based approaches, such as DDQN and DDPG, are used to solve the non-convex optimization problem of minimizing latency.

IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS (2023)

Article Engineering, Electrical & Electronic

Multi-Agent DRL for Resource Allocation and Cache Design in Terrestrial-Satellite Networks

Xiaonan Li, Haijun Zhang, Huan Zhou, Ning Wang, Keping Long, Saba Al-Rubaye, George K. Karagiannidis

Summary: This paper proposes a framework for resource allocation in the terrestrial-satellite network based on non-orthogonal multiple access (NOMA). A deployment method of local cache pools is also given to achieve lower time delay and maximize energy efficiency. The proposed method, which utilizes multi-agent deep deterministic policy gradient (MADDPG), shows better performance compared to traditional single-agent deep reinforcement learning algorithm in optimizing resource allocation and cache design in the integrated terrestrial-satellite network.

IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS (2023)

Article Computer Science, Theory & Methods

Accelerating distributed machine learning with model compression and graph partition

Yubin Duan, Jie Wu

Summary: This paper proposes a method to optimize the communication cost of the parameter server framework in distributed training by compressing the model and optimizing data and parameter allocation. Experimental results show that this compression and allocation scheme can efficiently reduce the communication overhead for both linear and deep neural network models.

JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING (2023)

Article Computer Science, Hardware & Architecture

ViChaser: Chase Your Viewpoint for Live Video Streaming With Block-Oriented Super-Resolution

Ning Chen, Sheng Zhang, Zhi Ma, Yu Chen, Yibo Jin, Jie Wu, Zhuzhong Qian, Yu Liang, Sanglu Lu

Summary: The usage of live streaming services has led to a substantial increase in live video traffic. However, the perceived quality of experience of users is frequently limited by variations in the upstream bandwidth of streamers. To address this issue, we propose ViChaser, a block-oriented super-resolution approach that performs neural super-resolution on potential blocks of interest in the media server and uses online learning to adapt to the dynamic content of the video. ViChaser achieves higher video quality and faster processing speed compared to existing methods.

IEEE-ACM TRANSACTIONS ON NETWORKING (2023)

暂无数据