Asynchronous Deep Reinforcement Learning for Data-Driven Task Offloading in MEC-Empowered Vehicular Networks

DOI number:10.1109/INFOCOM42981.2021.9488886

Journal:IEEE CONFERENCE ON COMPUTER COMMUNICATIONS (IEEE INFOCOM 2021)

Place of Publication:ELECTR NETWORK

Key Words:Mobile edge computing,vehicular networks,task offloading,asynchronous deep reinforcement learning

Abstract:Mobile edge computing (MEC) has been an effective paradigm to support real-time computation-intensive vehicular applications. However, due to highly dynamic vehicular topology, these existing centralized-based or distributed-based scheduling algorithms requiring high communication overhead, are not suitable for task offloading in vehicular networks. Therefore, we investigate a novel service scenario of MEC-based vehicular crowdsourcing, where each MEC server is an independent agent and responsible for making scheduling of processing traffic data sensed by crowdsourcing vehicles. On this basis, we formulate a data-driven task offloading problem by jointly optimizing offloading decision and bandwidth/computation resource allocation, and renting cost of heterogeneous servers, such as powerful vehicles, MEC servers and cloud, which is a mixed-integer programming problem and NP-hard. To reduce high time-complexity, we propose the solution in two stages. First, we design an asynchronous deep Q-learning to determine offloading decision, which achieves fast convergence by training the local DQN model at each agent in parallel and uploading for global model update asynchronously. Second, we decompose the remaining resource allocation problem into several independent subproblems and derive optimal analytic formula based on convex theory. Lastly, we build a simulation model and conduct comprehensive simulation, which demonstrates the superiority of the proposed algorithm.

Co-author:Penglin Dai,Kaiwen Hu,Xiao Wu,Huanlai Xing,Zhaofei Yu

Document Code:10.1109/INFOCOM42981.2021.9488886

ISSN No.:0743-166X

Translation or Not:no

Date of Publication:2021-11-18