A Hierarchical Deep Reinforcement Learning-Based Community Energy Trading Scheme for a Neighborhood of Smart Households

The uncertainties from the distributed energy resources (DERs) and the prosumer's heterogeneous characteristics bring great challenges to the operation of the community energy market. In this paper, the real-time energy sharing and management in the community market are decomposed into two-layered sub-problems: the household appliance scheduling problem and the internal energy trading pricing problem. A hierarchical deep reinforcement learning (HDRL) based scheme is proposed for the community energy trading with multiple households, containing a two-stage learning process. In the inner stage, a multi-agent deep reinforcement learning (MADRL) based approach is developed to learn the real-time appliances scheduling policy based on the local observations and given internal electricity price in a decentralized way. In the outer stage, a deep reinforcement learning (DRL) based pricing approach is proposed to determine the real-time internal electricity prices based on the participants' historical net power and external energy supplier's electricity prices. The scheduling policy and households' state are not required in the pricing process. Finally, the simulations with real-world datasets demonstrate that the internal energy trading can significantly reduce the prosumers' daily cost and the control performance of the proposed scheme is superior to the existing studies.

Automated summary

This paper proposes a deep reinforcement learning based scheme to address the challenges of real-time energy sharing and management in the community energy market. By decomposing the problem into household appliance scheduling and internal energy trading pricing, and utilizing a two-stage learning process, the proposed scheme can significantly reduce the prosumers' cost and outperforms existing studies in control performance.