4.6 Article

Coordinated automatic generation control of interconnected power system with imitation guided exploration multi-agent deep reinforcement learning

Publisher

ELSEVIER SCI LTD
DOI: 10.1016/j.ijepes.2021.107471

Keywords

Performance-based frequency regulation market; Intelligent automatic power generation control; Imitation guided exploration multi-agent twin; delayed deep deterministic policy gradient al-gorithm; Multi-area coordinated control; Frequency regulation mileage

Ask authors/readers for more resources

The research proposes an intelligent automatic generation control (IAGC) framework that utilizes adaptive proportional-integral (PI) controllers and imitation guided-exploration multi-agent twin-delayed deep deterministic policy gradient (IGE-MATD3) algorithm to achieve multi-area coordinated AGC, improving control performance.
An intelligent automatic generation control (IAGC) framework is proposed to address the coordination problems between AGC controllers in multi-area power systems. In this framework, every area of the power system consists of an adaptive proportional-integral (PI) controller that employs a tuner to regulate coefficients in real time. The tuner of each adaptive proportional-integral (PI) controller adopts an imitation guided-exploration multi-agent twin-delayed deep deterministic policy gradient (IGE-MATD3) algorithm, thereby realizing a multiarea coordinated AGC. To improve the robustness and adaptability of the IAGC framework, the proposed algorithm incorporates imitation learning and outputs the optimal coordinate control strategy of several controllers. As demonstrated by a simulation of the China Southern Grid four-area power system model, an IAGC framework can improve dynamic control performance and reduce the regulation mileage payment of the operator in every area.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.6
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

Article Green & Sustainable Science & Technology

Data-driven optimal PEMFC temperature control via curriculum guidance strategy-based large-scale deep reinforcement learning

Jiawen Li, Shengchun Yang, Tao Yu, Xiaoshun Zhang

Summary: An advanced controller based on large-scale deep reinforcement learning is proposed for controlling the stack temperature of PEMFC, along with a new deep reinforcement learning algorithm named CGS-L4DPG. The inclusion of curriculum guidance strategy and imitation learning in the algorithm improves the performance and robustness of the controller, making it more effective in controlling the PEMFC stack temperature than existing control algorithms.

IET RENEWABLE POWER GENERATION (2022)

Editorial Material Energy & Fuels

Editorial: Advanced Optimization and Control for Smart Grids With High Penetration of Renewable Energy Systems

Bo Yang, Dongran Song, Xing He, Xiaoshun Zhang, Sungyun Choi, Chao Duan, Yaxing Ren

FRONTIERS IN ENERGY RESEARCH (2022)

Article Thermodynamics

An efficient multi-agent negotiation algorithm for multi-period photovoltaic array reconfiguration with a hydrogen energy storage system

Xiaoshun Zhang, Tao Yu, Xiaoguang Ma, Lexin Guo

Summary: Given the significant energy loss caused by partial shading in photovoltaic systems, various array reconfiguration techniques have been proposed to improve power generation efficiency. Previous studies focused on maximizing power output without considering multi-period power fluctuations, resulting in low total profit. This paper presents a new multi-period photovoltaic array reconfiguration model with a hydrogen energy storage system under partial shading conditions, aiming to maximize total profit by considering electricity selling profit, hydrogen selling profit, and regulation cost. A novel multi-agent negotiation algorithm with an auctioneer and multiple bidders is designed to address this problem, achieving efficient and distributed optimization. Case studies demonstrate that the proposed algorithm outperforms five centralized meta-heuristic algorithms, significantly increasing total profit under varying partial shading conditions.

ENERGY CONVERSION AND MANAGEMENT (2022)

Article Energy & Fuels

A multi-objective energy coordinative and management policy for solid oxide fuel cell using triune brain large-scale multi-agent deep deterministic policy gradient

Jiawen Li

Summary: This paper proposes a data-driven multi-objective energy coordinative management policy to enhance the net output power and efficiency of a solid oxide fuel cell (SOFC). The policy focuses on maintaining stable oxygen excess ratio (OER) and fuel utilization (FU) ratio while meeting load demand through optimization agent and controller design.

APPLIED ENERGY (2022)

Article Chemistry, Physical

Optimal dual-model controller of solid oxide fuel cell output voltage using imitation distributed deep reinforcement learning

Jiawen Li, Haoyang Cui, Wei Jiang, Hengwen Yu

Summary: To address the nonlinearity and constraints in solid oxide fuel cell (SOFC) control, a dual-model control framework (DMCF) is proposed, with a PID controller and a supplementary dynamic controller. The supplementary controller adapts to uncertainties and fuel utilization constraints, while an imitation distributed deep deterministic policy gradient (ID3PG) algorithm enhances the robustness and adaptive capacity. Simulation results demonstrate the effectiveness of the proposed framework in controlling SOFC output voltage and satisfying fuel utilization constraints.

INTERNATIONAL JOURNAL OF HYDROGEN ENERGY (2023)

Article Energy & Fuels

Decomposition-Based Multi-Classifier-Assisted Evolutionary Algorithm for Bi-Objective Optimal Wind Farm Energy Capture

Hongbin Zhu, Xiang Gao, Lei Zhao, Xiaoshun Zhang

Summary: A novel bi-objective optimal wind farm energy capture (OWFEC) algorithm is proposed in this study, which considers both maximum power output and the balance of fatigue load distribution to reduce maintenance cost. To rapidly obtain high-quality Pareto optimal solutions, a decomposition-based multi-classifier-assisted evolutionary algorithm is designed. Simulations are carried out with three different scales of wind farms, and five familiar Pareto-based meta-heuristic algorithms are introduced for performance comparison to evaluate the effectiveness and performance of the proposed technique.

ENERGIES (2023)

Article Automation & Control Systems

Distributed deep reinforcement learning-based gas supply system coordination management method for solid oxide fuel cell

Jiawen Li, Haoyang Cui, Wei Jiang

Summary: To maintain the net output power of solid oxide fuel cells (SOFC) and avoid violating oxygen excess ratio and fuel utilization constraints, a data-driven gas supply system coordination management method is proposed. The algorithm, called PE-MA4DPG, is based on population evolution and utilizes multi-agent double delay deep deterministic policy gradient. The algorithm's effectiveness is demonstrated in comparison to existing algorithms through three experiments.

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE (2023)

Article Engineering, Electrical & Electronic

Dropout Deep Neural Network Assisted Transfer Learning for Bi-Objective Pareto AGC Dispatch

Xiaoshun Zhang, Chuangzhi Li, Biao Xu, Zhenning Pan, Tao Yu

Summary: To balance power disturbances, an independent system operator (ISO) assigns power regulation commands to resources via AGC dispatch. This paper proposes a novel DDNN-TL for rapid approximation of high-quality Pareto optimal solutions in AGC dispatch. Training data is generated from Pareto optimal solutions obtained by various algorithms. DDNN parameters are updated through offline training with this data. The results demonstrate superior performance in optimization speed and stability compared to other algorithms.

IEEE TRANSACTIONS ON POWER SYSTEMS (2023)

Article Energy & Fuels

Distributed quantum multiagent deep meta reinforcement learning for area autonomy energy management of a multiarea microgrid

Jiawen Li, Zhou Tao, Keke He, Hengwen Yu, Hongwei Du, Shuangyu Liu, Haoyang Cui

Summary: This paper presents a distributed area autonomy load frequency control (DAA-LFC) method that balances the interests of different grid operators and achieves fast frequency recovery. The method treats each area controller in a multiarea microgrid as an agent and uses gameplay for global optimization. The agents make independent decisions and do not need to communicate during online operation. In addition, a distributed quantum multiagent deep meta-deterministic policy gradient (DQMA-DMDPG) algorithm is proposed to achieve collaborative multitask learning. The simulation results show that the proposed method reduces frequency deviation, power generation costs, and balances the interests of multiple operators.

APPLIED ENERGY (2023)

Article Green & Sustainable Science & Technology

Active fault-tolerant coordination energy management for a proton exchange membrane fuel cell using curriculum-based multiagent deep meta-reinforcement learning

Jiawen Li, Tao Zhou

Summary: This paper addresses the challenge of active fault-tolerant coordination control for proton exchange membrane fuel cells (PEMFCs). The proposed method aims to stabilize the output performance of four operating variables and prevent constraint violations in PEMFCs during failure scenarios. The method utilizes a curriculum-based multiagent deep meta-deterministic policy gradient algorithm to achieve multitask collaboration and enhance PEMFC robustness. The algorithm consists of a meta-learner and a base learner, which cooperate to detect faults and select appropriate control policies.

RENEWABLE & SUSTAINABLE ENERGY REVIEWS (2023)

Article Computer Science, Information Systems

Evolutionary Multi-Agent Deep Meta Reinforcement Learning Method for Swarm Intelligence Energy Management of Isolated Multi-Area Microgrid With Internet of Things

Jiawen Li, Tao Zhou

Summary: This article proposes a swarm intelligence load frequency control (SI-LFC) method for coordinating the interests of multiple operators in an isolated multiarea microgrid. The method treats the units in each area as independent agents and adopts swarm intelligence centralized offline learning policy to achieve a balance of interests. Online, each unit only needs to collect the frequency locally to achieve global optimal control, reducing network communication burden. The article also introduces an evolutionary multiagent deep meta-actor-critic (EMA-DMAC) algorithm, which enhances collaborative learning of swarm agents, improving the robustness and quality of SI-LFC strategies. The proposed method's effectiveness is demonstrated in a simulation of a four-area LFC model for Sansha island in the China Southern Grid (CSG).

IEEE INTERNET OF THINGS JOURNAL (2023)

Article Energy & Fuels

Multi-agent deep meta-reinforcement learning-based active fault tolerant gas supply management system for proton exchange membrane fuel cells

Jiawen Li, Yuanyuan Cheng, Hengwen Yu, Hongwei Du, Haoyang Cui

Summary: This paper proposes a data-driven active fault-tolerant control method for stable control of proton exchange membrane fuel cells (PEMFCs) in the event of a fault. It combines meta-reinforcement learning with multiagent reinforcement learning to provide independent multitask cooperative learning capabilities, ensuring excellent robustness.

ETRANSPORTATION (2023)

Article Automation & Control Systems

Brain-Inspired Deep Meta-Reinforcement Learning for Active Coordinated Fault-Tolerant Load Frequency Control of Multi-Area Grids

Jiawen Li, Tao Zhou, Haoyang Cui

Summary: This paper proposes an active coordinated fault tolerance load frequency control (AFCT-LFC) method to prevent sudden frequency changes caused by unit actuator failures or unplanned decommissioning in a multi-area interconnected grid. It also introduces a brain-Inspired deep meta-deterministic policy gradient algorithm (BIMA-DMDPG) for multi-agent centralized training and distributed training. The method is tested in a four-area LFC model and shows superior performance compared to existing algorithms.

IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING (2023)

Article Construction & Building Technology

Multiagent deep meta reinforcement learning for sea computing-based energy management of interconnected grids considering renewable energy sources in sustainable cities

Jiawen Li, Tao Zhou

Summary: This study proposes a sea computing-based grid-area coordinated load frequency control method which effectively reduces frequency fluctuations and improves quality and robustness. By introducing meta-reinforcement learning and curriculum learning, the method guides agent training to obtain LFC strategies that suit market requirements.

SUSTAINABLE CITIES AND SOCIETY (2023)

Article Engineering, Electrical & Electronic

A novel framework for the day-ahead market clearing process featuring the participation of distribution system operators and a hybrid pricing mechanism

Wandry R. Faria, Gregorio Munoz-Delgado, Javier Contreras, Benvindo R. Pereira Jr

Summary: This paper proposes a new bilevel mathematical model for competitive electricity markets, taking into account the participation of distribution systems operators. A new pricing method is introduced as an alternative to the inaccessible dual variables of the transmission system.

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS (2024)

Article Engineering, Electrical & Electronic

A smith-predictor-assisted adaptive load disturbance rejection controller for speed variation suppression of PMSM drive

Chao Zhang, Liwei Zhang, Dong Wang, Kaiyuan Lu

Summary: The load disturbance rejection ability of electrical machine systems is crucial in many applications. Existing studies mainly focus on improving disturbance observers, but the speed response control during the transient also plays a significant role. This paper proposes a sliding mode disturbance observer-based load disturbance rejection control with an adaptive filter and a Smith predictor-based speed filter delay compensator to enhance the transient speed response.

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS (2024)

Article Engineering, Electrical & Electronic

A communication-less islanding detection scheme for hybrid distributed generation systems using recurrent neural network

Arif Hussain, Arif Mehdi, Chul-Hwan Kim

Summary: The proposed scheme in this research paper is a communication-less islanding detection system based on recurrent neural network (RNN) for hybrid distributed generator (DG) systems. The scheme demonstrates good performance in feature extraction, feature selection, and islanding detection, and it also performs effectively in noisy environments.

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS (2024)

Article Engineering, Electrical & Electronic

A status pre-matching method for the real-time simulation of power electronic converters

Zonghui Sun, Xizheng Guo, Shinan Wang, Xiaojie You

Summary: This paper presents a status pre-matching method (SPM) that eliminates the iterative calculations for resistance switch model, and simulates all operation modes of PECs through a more convenient approach. Furthermore, a FPGA implementation scheme is proposed to fully utilize the multiplier units of FPGA.

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS (2024)

Article Engineering, Electrical & Electronic

Probabilistic spatiotemporal scenario generation method for dynamic optimal power flow in distribution networks

Rui Zhou, Shuheng Chen, Yang Han, Qunying Liu, Zhe Chen, Weihao Hu

Summary: In power system scheduling with variable renewable energy sources, considering both spatial and temporal correlations is a challenging task due to the complex intertwining of spatiotemporal characteristics and computational complexity caused by high dimensionality. This paper proposes a novel probabilistic spatiotemporal scenario generation (PSTSG) method that generates probabilistic scenarios accounting for spatial and temporal correlations simultaneously. The method incorporates Latin hypercube sampling, copula-importance sampling theory, and probability-based scenario reduction technique to efficiently capture the spatial and temporal correlation in the dynamic optimal power flow problem. Numerical simulations demonstrate the superiority of the proposed approach in terms of computational efficiency and accuracy compared to existing methods.

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS (2024)

Article Engineering, Electrical & Electronic

Integrated simulation of electromechanical and thermal dynamics of voltage source converters

Juan Manuel Mauricio, J. Carlos Olives-Camps, Jose Maria Maza-Ortega, Antonio Gomez-Exposito

Summary: This paper proposes a simplified thermal model of VSC, which can produce accurate results at a low computational cost. The model consists of a simple first-order thermal dynamics system and two quadratic equations to model power losses. A methodology is also provided to derive the model parameters from manufacturer data.

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS (2024)

Article Engineering, Electrical & Electronic

On the perturbation size of the finite difference method for trajectory sensitivity-based assessment of power system dynamics with non-smooth behavior

Jae-Kyeong Kim, Kyeon Hur

Summary: This paper investigates the relationship between the accuracy of finite difference-based trajectory sensitivity (FDTS) analysis and the perturbation size in non-smooth systems. The study reveals that the approximation accuracy is significantly influenced by the perturbation size, and linear approximation is the most suitable method for practical applications.

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS (2024)

Article Engineering, Electrical & Electronic

Mitigating the impacts of geomagnetic disturbances on the small signal stability

Yuan Si, Amjad Anvari-Moghaddam

Summary: This paper investigates the impact of geomagnetic disturbances on small signal stability in power systems and proposes the installation of blocking devices to mitigate the negative effects. Quantitative evaluation reveals that intense geomagnetic disturbances significantly increase the risk of small signal instability. Optimal placement of blocking devices based on sensitivity scenarios results in a significant reduction in the risk index compared to constant and varying induced geoelectric fields scenarios.

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS (2024)

Article Engineering, Electrical & Electronic

Comparison of different geoelectric field methods to calculate geomagnetically induced currents in North China

Xuejian Zhang, Wenxin Kong, Nian Yu, Huang Chen, Tianyang Li, Enci Wang

Summary: The intensity estimation of geomagnetically induced currents (GICs) varies depending on the method used. The estimation using field magnetotelluric (MT) data provides the highest accuracy, followed by the estimation using 3D conductivity models and the estimation using a 1D conductivity model. The GICs in the North China 1000-kV power grid have reached a very high-risk level, with C3 and C4 having a significant impact on the geoelectric field and GICs.

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS (2024)

Article Engineering, Electrical & Electronic

Stochastic optimal dispatch of offshore-onshore regional integrated energy system based on improved state-space approximate dynamic programming

Yue Pan, Shunjiang Lin, Weikun Liang, Xiangyong Feng, Xuan Sheng, Mingbo Liu

Summary: This paper introduces the concept and model of offshore-onshore regional integrated energy system, and proposes a stochastic optimal dispatch model and an improved state-space approximate dynamic programming algorithm to solve the model. The case study demonstrates the effectiveness and high efficiency of the proposed method in improving economic and environmental benefits.

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS (2024)

Article Engineering, Electrical & Electronic

A decentralized control method for proportional current-sharing, voltage restoration, and SOCs balancing of widespread DC microgrids

Mohammad Eydi, Reza Ghazi, Majid Oloomi Buygi

Summary: Proportional current sharing, voltage restoration, and SOCs balancing in DC microgrid control algorithms are the leading challenges. This paper proposes a novel communication-less control method using a capacitor and a DC/DC converter to stabilize the system and restore the DC bus voltage. The method includes injecting an AC signal into the DC bus, setting the current of energy storage units based on frequency and SOC, and incorporating droop control for system stability. Stability analysis and simulation results validate the effectiveness of the proposed method.

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS (2024)

Article Engineering, Electrical & Electronic

An upscaling minute-level regional photovoltaic power forecasting scheme

Xiangjian Meng, Xinyu Shi, Weiqi Wang, Yumin Zhang, Feng Gao

Summary: With the increasing penetration of photovoltaic power generation, regional power forecasting becomes critical for stable and economical operation of power systems. This paper proposes a minute-level regional PV power forecasting scheme using selected reference PV plants. The challenges include the lack of complete historical power data and the heavy computation burden. The proposed method incorporates a novel reference PV plant selection method and a flexible approach to decrease the accumulated error of rolling forecasting.

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS (2024)

Article Engineering, Electrical & Electronic

Stability characteristics analysis and ultra-low frequency oscillation suppression strategy for FSC-VSPSU

Huabo Shi, Yuhong Wang, Xinwei Sun, Gang Chen, Lijie Ding, Pengyu Pan, Qi Zeng

Summary: This article investigates the dynamic stability characteristics of the full size converter variable speed pumped storage unit and proposes improvements for the control strategy. The research is important for ensuring the safe and efficient operation of the unit.

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS (2024)

Article Engineering, Electrical & Electronic

Optimal scheduling of a microgrid with power quality constraints based on demand side management under grid-connected and islanding operations

Firmansyah Nur Budiman, Makbul A. M. Ramli, Houssem R. E. H. Bouchekara, Ahmad H. Milyani

Summary: This paper proposes an optimal harmonic power flow framework for the daily scheduling of a grid-connected microgrid, which addresses power quality issues and ensures effective control through demand side management.

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS (2024)

Article Engineering, Electrical & Electronic

A double decomposition based coevolutionary algorithm for distributed multi-objective OPF solution

Cong Zeng, Ziyu Chen, Jizhong Zhu, Fellew Ieee

Summary: This paper introduces a distributed solution method for the multi-objective OPF problem, using a coevolutionary multi-objective evolutionary algorithm and the idea of decomposition. The problem is alleviated by decomposing decision variables and objective functions, and a new distributed fitness evaluation method is proposed. The experimental results demonstrate the effectiveness of the method and its excellence in large-scale systems.

INTERNATIONAL JOURNAL OF ELECTRICAL POWER & ENERGY SYSTEMS (2024)