IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v18y2025i7p1837-d1628507.html
   My bibliography  Save this article

A Review of Smart Grid Evolution and Reinforcement Learning: Applications, Challenges and Future Directions

Author

Listed:
  • Na Xu

    (School of Electrical and Information Engineering, Tianjin University, No. 92, Weijin Road, Nankai District, Tianjin 300072, China)

  • Zhuo Tang

    (School of Electrical and Information Engineering, Tianjin University, No. 92, Weijin Road, Nankai District, Tianjin 300072, China)

  • Chenyi Si

    (School of Electrical and Information Engineering, Tianjin University, No. 92, Weijin Road, Nankai District, Tianjin 300072, China)

  • Jinshan Bian

    (School of Artificial Intelligence, Anhui University, Qingyuan Campus, 111 Jiulong Road, Hefei 230093, China)

  • Chaoxu Mu

    (School of Electrical and Information Engineering, Tianjin University, No. 92, Weijin Road, Nankai District, Tianjin 300072, China)

Abstract

In the face of the rapid development of smart grid technologies, it is increasingly difficult for traditional power system management methods to support the increasingly complex operation of modern power grids. This study systematically reviews new challenges and research trends in the field of smart grid optimization, focusing on key issues such as power flow optimization, load scheduling, and reactive power compensation. By analyzing the application of reinforcement learning in the smart grid, the impact of distributed new energy’s high penetration on the stability of the system is thoroughly discussed, and the advantages and disadvantages of the existing control strategies are systematically reviewed. This study compares the applicability, advantages, and limitations of different reinforcement learning algorithms in practical scenarios, and reveals core challenges such as state space complexity, learning stability, and computational efficiency. On this basis, a multi-agent cooperation optimization direction based on the two-layer reinforcement learning framework is proposed to improve the dynamic coordination ability of the power grid. This study provides a theoretical reference for smart grid optimization through multi-dimensional analysis and research, advancing the application of deep reinforcement learning technology in this field.

Suggested Citation

  • Na Xu & Zhuo Tang & Chenyi Si & Jinshan Bian & Chaoxu Mu, 2025. "A Review of Smart Grid Evolution and Reinforcement Learning: Applications, Challenges and Future Directions," Energies, MDPI, vol. 18(7), pages 1-19, April.
    • Handle: RePEc:gam:jeners:v:18:y:2025:i:7:p:1837-:d:1628507
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/18/7/1837/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/18/7/1837/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. de Mars, Patrick & O’Sullivan, Aidan, 2021. "Applying reinforcement learning and tree search to the unit commitment problem," Applied Energy, Elsevier, vol. 302(C).
    2. Li, Xiangyu & Luo, Fengji & Li, Chaojie, 2024. "Multi-agent deep reinforcement learning-based autonomous decision-making framework for community virtual power plants," Applied Energy, Elsevier, vol. 360(C).
    3. Dong, Lei & Lin, Hao & Qiao, Ji & Zhang, Tao & Zhang, Shiming & Pu, Tianjiao, 2024. "A coordinated active and reactive power optimization approach for multi-microgrids connected to distribution networks with multi-actor-attention-critic deep reinforcement learning," Applied Energy, Elsevier, vol. 373(C).
    4. Van Can Nguyen & Chi-Tai Wang & Ying-Jiun Hsieh, 2021. "Electrification of Highway Transportation with Solar and Wind Energy," Sustainability, MDPI, vol. 13(10), pages 1-28, May.
    5. Alexander N. Kozlov & Nikita V. Tomin & Denis N. Sidorov & Electo E. S. Lora & Victor G. Kurbatsky, 2020. "Optimal Operation Control of PV-Biomass Gasifier-Diesel-Hybrid Systems Using Reinforcement Learning Techniques," Energies, MDPI, vol. 13(10), pages 1-20, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jiang, Wei & Wang, Teng & Yuan, Dongdong & Sha, Aimin & Zhang, Shuo & Zhang, Yufei & Xiao, Jingjing & Xing, Chengwei, 2024. "Available solar resources and photovoltaic system planning strategy for highway," Renewable and Sustainable Energy Reviews, Elsevier, vol. 203(C).
    2. Louis Desportes & Inbar Fijalkow & Pierre Andry, 2021. "Deep Reinforcement Learning for Hybrid Energy Storage Systems: Balancing Lead and Hydrogen Storage," Energies, MDPI, vol. 14(15), pages 1-22, August.
    3. Awol Seid Ebrie & Chunhyun Paik & Yongjoo Chung & Young Jin Kim, 2023. "Environment-Friendly Power Scheduling Based on Deep Contextual Reinforcement Learning," Energies, MDPI, vol. 16(16), pages 1-12, August.
    4. Daniel de Cerqueira Lima e Penalva Santos & Juliana Berendt Ihler & Laene Oliveira Soares & Yana Amaral Alves & Cristina Gomez Souza & Ronney Arismel Mancebo Boloy, 2022. "Global Trends and Opportunities in Hybrid Microgrid Systems Using Renewable Energies," International Journal of Energy Economics and Policy, Econjournals, vol. 12(4), pages 263-273, July.
    5. Lee, Jechan & Kim, Soosan & You, Siming & Park, Young-Kwon, 2023. "Bioenergy generation from thermochemical conversion of lignocellulosic biomass-based integrated renewable energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 178(C).
    6. Zhao, Shihao & Li, Kang & Yang, Zhile & Xu, Xinzhi & Zhang, Ning, 2022. "A new power system active rescheduling method considering the dispatchable plug-in electric vehicles and intermittent renewable energies," Applied Energy, Elsevier, vol. 314(C).
    7. Nadia Cerone & Francesco Zimbardi, 2021. "Effects of Oxygen and Steam Equivalence Ratios on Updraft Gasification of Biomass," Energies, MDPI, vol. 14(9), pages 1-18, May.
    8. O’Malley, Cormac & de Mars, Patrick & Badesa, Luis & Strbac, Goran, 2023. "Reinforcement learning and mixed-integer programming for power plant scheduling in low carbon systems: Comparison and hybridisation," Applied Energy, Elsevier, vol. 349(C).
    9. Linxin Zhang & Zuobin Ying & Zhile Yang & Yuanjun Guo, 2024. "Dynamic Multi-Energy Optimization for Unit Commitment Integrating PEVs and Renewable Energy: A DO3LSO Algorithm," Mathematics, MDPI, vol. 12(24), pages 1-28, December.
    10. Xu, Biao & Luan, Wenpeng & Yang, Jing & Zhao, Bochao & Long, Chao & Ai, Qian & Xiang, Jiani, 2024. "Integrated three-stage decentralized scheduling for virtual power plants: A model-assisted multi-agent reinforcement learning method," Applied Energy, Elsevier, vol. 376(PA).
    11. Guozhen Ma & Ning Pang & Yunjia Wang & Shiyao Hu & Xiaobin Xu & Zeya Zhang & Changhong Wang & Liai Gao, 2024. "Two-Stage Optimal Scheduling Strategy of Microgrid Distribution Network Considering Multi-Source Agricultural Load Aggregation," Energies, MDPI, vol. 17(21), pages 1-16, October.
    12. Nandhini Kullampalayam Murugaiyan & Kumar Chandrasekaran & Magdalin Mary Devapitchai & Tomonobu Senjyu, 2024. "Parameter Estimation of Three-Diode Photovoltaic Model Using Reinforced Learning-Based Parrot Optimizer with an Adaptive Secant Method," Sustainability, MDPI, vol. 16(23), pages 1-34, December.
    13. Tomin, Nikita & Shakirov, Vladislav & Kozlov, Aleksander & Sidorov, Denis & Kurbatsky, Victor & Rehtanz, Christian & Lora, Electo E.S., 2022. "Design and optimal energy management of community microgrids with flexible renewable energy sources," Renewable Energy, Elsevier, vol. 183(C), pages 903-921.
    14. Haugen, Mari & Blaisdell-Pijuan, Paris L. & Botterud, Audun & Levin, Todd & Zhou, Zhi & Belsnes, Michael & Korpås, Magnus & Somani, Abhishek, 2024. "Power market models for the clean energy transition: State of the art and future research needs," Applied Energy, Elsevier, vol. 357(C).
    15. Song, Tiewei & Fu, Lijun & Zhong, Linlin & Fan, Yaxiang & Shang, Qianyi, 2024. "HP3O algorithm-based all electric ship energy management strategy integrating demand-side adjustment," Energy, Elsevier, vol. 295(C).
    16. Wang, Wei & Feng, Bin & Huang, Gang & Guo, Chuangxin & Liao, Wenlong & Chen, Zhe, 2023. "Conformal asymmetric multi-quantile generative transformer for day-ahead wind power interval prediction," Applied Energy, Elsevier, vol. 333(C).
    17. Wei, Lishen & Ai, Xiaomeng & Fang, Jiakun & Cui, Shichang & Gao, Liqian & Li, Kun & Wen, Jinyu, 2025. "Data-augmentation acceleration framework by graph neural network for near-optimal unit commitment," Applied Energy, Elsevier, vol. 377(PD).
    18. Wang, Siyi & Sheng, Wanxing & Shang, Yuwei & Liu, Keyan, 2024. "Distribution network voltage control considering virtual power plants cooperative optimization with transactive energy," Applied Energy, Elsevier, vol. 371(C).
    19. Ebrie, Awol Seid & Kim, Young Jin, 2024. "Reinforcement learning-based optimization for power scheduling in a renewable energy connected grid," Renewable Energy, Elsevier, vol. 230(C).
    20. Lu, Jun & Liu, Tianqi & He, Chuan & Nan, Lu & Hu, Xiaotong, 2021. "Robust day-ahead coordinated scheduling of multi-energy systems with integrated heat-electricity demand response and high penetration of renewable energy," Renewable Energy, Elsevier, vol. 178(C), pages 466-482.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:18:y:2025:i:7:p:1837-:d:1628507. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.