On the Feasibility Guarantees of Deep Reinforcement Learning Solutions for Distribution System Operation

Deep reinforcement learning (DRL) has scored unprecedented success in finding near-optimal solutions in high-dimensional stochastic problems, leading to its extensive use in operational research, including the operation of power systems. However, in practice, it has been adopted with extreme caution because the standard DRL does not guarantee the satisfaction of operational constraints. In this paper, the feasibility of solutions given by a DRL agent trained to operate a distribution system is guaranteed by modifying the exploration process and optimality criterion of standard DRL. To that end, first, a convex feasibility set in the form of a multi-dimensional polyhedron, called feasibility diamond, is formed inside the region defined by the power flow constraints, using which the feasibility of solutions given by DRL is checked in real-time. Solutions outside the feasibility diamond are projected on the diamond's surface, and the new modified action is used for DRL training. Further, the distance of infeasible solutions to their feasible projection is penalized in the DRL reward function. The impact of the proposed method on the feasibility and optimality of DRL solutions are tested on three test distribution systems, indicating that modifying the exploration process and a soft penalization of infeasibilities works best in achieving near-optimal and reliable DRL-trained operators.

Automated summary

This paper guarantees the feasibility of solutions given by a DRL agent trained to operate a distribution system by modifying the exploration process and optimality criterion. A convex feasibility set, called feasibility diamond, is formed inside the power flow constraints region, and solutions outside the diamond are projected on its surface for DRL training. The impact of this method on the feasibility and optimality of DRL solutions are tested on three test distribution systems, showing near-optimal and reliable operators can be achieved.