IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v15y2023i14p11084-d1194972.html

An Intelligent Algorithm for Solving Unit Commitments Based on Deep Reinforcement Learning

Author

Listed:
  • Guanglei Huang

    (Shenzhen Power Supply Company, China Southern Power Grid, Shenzhen 518067, China)

  • Tian Mao

    (Electric Power Research Institute, China Southern Power Grid, Guangzhou 510530, China)

  • Bin Zhang

    (Shenzhen Power Supply Company, China Southern Power Grid, Shenzhen 518067, China)

  • Renli Cheng

    (Shenzhen Power Supply Company, China Southern Power Grid, Shenzhen 518067, China)

  • Mingyu Ou

    (Shenzhen Power Supply Company, China Southern Power Grid, Shenzhen 518067, China)

Abstract

With the reform of energy structures, the high proportion of volatile new energy access makes the existing unit commitment (UC) theory unable to satisfy the development demands of day-ahead market decision-making in the new power system. Therefore, this paper proposes an intelligent algorithm for solving UC, based on deep reinforcement learning (DRL) technology. Firstly, the DRL algorithm is used to model the Markov decision process of the UC problem, and the corresponding state space, transfer function, action space and reward function are proposed. Then, the policy gradient (PG) algorithm is used to solve the problem. On this basis, Lambda iteration is used to solve the output scheme of the unit in the start–stop state, and finally a DRL-based UC intelligent solution algorithm is proposed. The applicability and effectiveness of this method are verified based on simulation examples.

Suggested Citation

  • Guanglei Huang & Tian Mao & Bin Zhang & Renli Cheng & Mingyu Ou, 2023. "An Intelligent Algorithm for Solving Unit Commitments Based on Deep Reinforcement Learning," Sustainability, MDPI, vol. 15(14), pages 1-19, July.
    • Handle: RePEc:gam:jsusta:v:15:y:2023:i:14:p:11084-:d:1194972
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/15/14/11084/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/15/14/11084/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Huiru Zhao & Yuwei Wang & Sen Guo & Mingrui Zhao & Chao Zhang, 2016. "Application of a Gradient Descent Continuous Actor-Critic Algorithm for Double-Side Day-Ahead Electricity Market Modeling," Energies, MDPI, vol. 9(9), pages 1-20, September.
    2. Fang, Ping & Fu, Wenlong & Wang, Kai & Xiong, Dongzhen & Zhang, Kai, 2022. "A compositive architecture coupling outlier correction, EWT, nonlinear Volterra multi-model fusion with multi-objective optimization for short-term wind speed forecasting," Applied Energy, Elsevier, vol. 307(C).
    3. Yu, Guangzheng & Liu, Chengquan & Tang, Bo & Chen, Rusi & Lu, Liu & Cui, Chaoyue & Hu, Yue & Shen, Lingxu & Muyeen, S.M., 2022. "Short term wind power prediction for regional wind farms based on spatial-temporal characteristic distribution," Renewable Energy, Elsevier, vol. 199(C), pages 599-612.
    4. Chen, J.J. & Qi, B.X. & Rong, Z.K. & Peng, K. & Zhao, Y.L. & Zhang, X.H., 2021. "Multi-energy coordinated microgrid scheduling with integrated demand response for flexibility improvement," Energy, Elsevier, vol. 217(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhi Zhang & Dan Xu & Xuezhen Chan & Guobin Xu, 2023. "Research on Power System Day-Ahead Generation Scheduling Method Considering Combined Operation of Wind Power and Pumped Storage Power Station," Sustainability, MDPI, vol. 15(7), pages 1-14, April.
    2. Mansour-Saatloo, Amin & Pezhmani, Yasin & Mirzaei, Mohammad Amin & Mohammadi-Ivatloo, Behnam & Zare, Kazem & Marzband, Mousa & Anvari-Moghaddam, Amjad, 2021. "Robust decentralized optimization of Multi-Microgrids integrated with Power-to-X technologies," Applied Energy, Elsevier, vol. 304(C).
    3. Gao, Yang & Ai, Qian & He, Xing & Fan, Songli, 2023. "Coordination for regional integrated energy system through target cascade optimization," Energy, Elsevier, vol. 276(C).
    4. Meng, Anbo & Zhang, Haitao & Yin, Hao & Xian, Zikang & Chen, Shu & Zhu, Zibin & Zhang, Zheng & Rong, Jiayu & Li, Chen & Wang, Chenen & Wu, Zhenbo & Deng, Weisi & Luo, Jianqiang & Wang, Xiaolin, 2023. "A novel multi-gradient evolutionary deep learning approach for few-shot wind power prediction using time-series GAN," Energy, Elsevier, vol. 283(C).
    5. Zheng, Xidong & Bai, Feifei & Zeng, Ziyang & Jin, Tao, 2024. "A new methodology to improve wind power prediction accuracy considering power quality disturbance dimension reduction and elimination," Energy, Elsevier, vol. 287(C).
    6. Ge, Haotian & Zhu, Yu & Zhong, Jiuming & Wu, Liang, 2024. "Day-ahead optimization for smart energy management of multi-microgrid using a stochastic-robust model," Energy, Elsevier, vol. 313(C).
    7. Yang, Ting & Yang, Zhenning & Li, Fei & Wang, Hengyu, 2024. "A short-term wind power forecasting method based on multivariate signal decomposition and variable selection," Applied Energy, Elsevier, vol. 360(C).
    8. Wang, Liying & Lin, Jialin & Dong, Houqi & Wang, Yuqing & Zeng, Ming, 2023. "Demand response comprehensive incentive mechanism-based multi-time scale optimization scheduling for park integrated energy system," Energy, Elsevier, vol. 270(C).
    9. Haibing Wang & Chengmin Wang & Weiqing Sun & Muhammad Qasim Khan, 2022. "Energy Pricing and Management for the Integrated Energy Service Provider: A Stochastic Stackelberg Game Approach," Energies, MDPI, vol. 15(19), pages 1-15, October.
    10. Liu, Shuhan & Sun, Wenqiang, 2025. "Knowledge- and data-driven prediction of blast furnace gas generation and consumption in iron and steel sites," Applied Energy, Elsevier, vol. 390(C).
    11. Liu, Ling & Wang, Jujie & Li, Jianping & Wei, Lu, 2023. "An online transfer learning model for wind turbine power prediction based on spatial feature construction and system-wide update," Applied Energy, Elsevier, vol. 340(C).
    12. Mohseni, Soheil & Brent, Alan C. & Kelly, Scott & Browne, Will N., 2022. "Demand response-integrated investment and operational planning of renewable and sustainable energy systems considering forecast uncertainties: A systematic review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 158(C).
    13. Ren, Lina & Zhang, Kunpeng & Mehran, Kamyar & Ma, Kai, 2025. "Low-carbon economic dispatch of a hydrogen-based integrated energy system considering the coordinated operation of CHP-ORC-CSP and P2G-CCS," Energy, Elsevier, vol. 340(C).
    14. Li, Ling-Ling & Miao, Yan & Lim, Ming K. & Sethanan, Kanchana & Tseng, Ming-Lang, 2024. "Integrated energy system for low-carbon economic operation optimization: Pareto compromise programming and master-slave game," Renewable Energy, Elsevier, vol. 222(C).
    15. Yongkang Liu & Yi Gu & Yuwei Long & Qinyu Zhang & Yonggang Zhang & Xu Zhou, 2025. "Research on Physically Constrained VMD-CNN-BiLSTM Wind Power Prediction," Sustainability, MDPI, vol. 17(3), pages 1-21, January.
    16. Yang, Yakai & Fan, Shuanglong & Liu, Zhenqing & Yu, Zhongze, 2025. "WD-SGformer: high-precision wind power forecasting via dual-attention dynamic spatio-temporal learning," Energy, Elsevier, vol. 337(C).
    17. Yang, Mao & Han, Chao & Zhang, Wei & Wang, Bo, 2024. "A short-term power prediction method for wind farm cluster based on the fusion of multi-source spatiotemporal feature information," Energy, Elsevier, vol. 294(C).
    18. Huiru Zhao & Yuwei Wang & Mingrui Zhao & Qingkun Tan & Sen Guo, 2017. "Day-Ahead Market Modeling for Strategic Wind Power Producers under Robust Market Clearing," Energies, MDPI, vol. 10(7), pages 1-27, July.
    19. Qingquan Lv & Jialin Zhang & Jianmei Zhang & Zhenzhen Zhang & Qiang Zhou & Pengfei Gao & Haozhe Zhang, 2025. "Short-Term Wind Power Prediction Model Based on PSO-CNN-LSTM," Energies, MDPI, vol. 18(13), pages 1-18, June.
    20. Alexander Kell, 2021. "Modelling the transition to a low-carbon energy supply," Papers 2111.00987, arXiv.org.

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:15:y:2023:i:14:p:11084-:d:1194972. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.