Bi-level stochastic real-time pricing model in multi-energy generation system: A reinforcement learning approach

With the penetration of intermittent renewable energy sources, greater uncertainty has been brought to the power generation system, creating increased challenges to real-time pricing (RTP). Different from the existing studies, this paper aims to design an RTP strategy for the smart grid which integrates multi energy generation on the supply side. Without loss of generality, small-scale distributed energy generation and power storage devices for users are also considered. Taking the interests of both supply and demand sides into consideration, a bilevel stochastic model for real-time demand response in the framework of Markov decision process (MDP) is formulated. The model well captures the interactive characters of both sides. Regarding the difficulty of collecting exact information from users in a centralized way in practice, a novel distributed online multi-agent reinforcement learning algorithm is proposed to solve the MDP model without acquisition of the transition probabilities. Through the information interaction between the upper and lower levels, the real-time electricity prices are decided adaptively, meanwhile, the optimal strategy of power supply and consumption is obtained. Simulation results demonstrate that the proposed pricing method and algorithm have a good performance in cutting peak and filling the valley and guarantee the benefits of both supply and demand. (c) 2021 Elsevier Ltd. All rights reserved.

Automated summary

This paper aims to design a real-time pricing strategy for the smart grid integrating multi energy generation on the supply side and considering small-scale distributed energy generation and power storage devices for users. By formulating a bilevel stochastic model within the framework of Markov decision process, the interactive characters of both supply and demand sides are well captured. A novel distributed online multi-agent reinforcement learning algorithm is proposed to solve the model without acquisition of transition probabilities, leading to adaptive real-time electricity prices and optimal power supply and consumption strategy. Simulation results demonstrate the good performance of the proposed pricing method and algorithm in peak shaving, valley filling, and guaranteeing benefits for both supply and demand sides.