IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v355y2024ics0306261923016100.html
   My bibliography  Save this article

Decomposition prediction fractional-order PID reinforcement learning for short-term smart generation control of integrated energy systems

Author

Listed:
  • Yin, Linfei
  • Zheng, Da

Abstract

With the continuous development of integrated energy systems (IESs), various distributed power is continuously connected to IESs. Uncertainty and volatility of renewable energy could increase power deviations of power systems and put pressure on the frequency control of the grid. A decomposition prediction fractional-order PID reinforcement learning (DPFOPIDRL) algorithm is proposed to reduce frequency deviations of IES and improve power quality in this study. The DPFOPIDRL-based unified timescale intelligent generation controller, which sends regulation commands to the automatic generation control every four seconds, continuously collects time series signals of frequency deviations; and then, the time series signals are predicted after modal decomposition. The DPFOPIDRL applies state-action-reward-state-action to control prediction signals with intense fluctuations, and fractional order proportional-integral-derivative to control prediction signals with gentle fluctuations. The DPFOPIDRL, proportional-integral-derivative, Q learning, and sliding mode are compared in four cases in an equivalent simplified and complex IES based on IEEE 39-node-Belgium 20-node system. Results under complex IES show that the frequency deviation and total generation cost of the DPFOPIDRL are at least 35.66% and 16.13% smaller than PID, Q learning, and sliding mode, respectively.

Suggested Citation

  • Yin, Linfei & Zheng, Da, 2024. "Decomposition prediction fractional-order PID reinforcement learning for short-term smart generation control of integrated energy systems," Applied Energy, Elsevier, vol. 355(C).
    • Handle: RePEc:eee:appene:v:355:y:2024:i:c:s0306261923016100
    DOI: 10.1016/j.apenergy.2023.122246
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261923016100
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2023.122246?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Liu, Yunfei & Liu, Yan & Cai, Hanhu & Zhang, Junran, 2023. "An innovative short-term multihorizon photovoltaic power output forecasting method based on variational mode decomposition and a capsule convolutional neural network," Applied Energy, Elsevier, vol. 343(C).
    2. Zhang, Chu & Hua, Lei & Ji, Chunlei & Shahzad Nazir, Muhammad & Peng, Tian, 2022. "An evolutionary robust solar radiation prediction model based on WT-CEEMDAN and IASO-optimized outlier robust extreme learning machine," Applied Energy, Elsevier, vol. 322(C).
    3. Mughees, Neelam & Jaffery, Mujtaba Hussain & Mughees, Anam & Ansari, Ejaz Ahmad & Mughees, Abdullah, 2023. "Reinforcement learning-based composite differential evolution for integrated demand response scheme in industrial microgrids," Applied Energy, Elsevier, vol. 342(C).
    4. Cheng, Gong & Wang, Xinzhi & He, Yurong, 2021. "Remaining useful life and state of health prediction for lithium batteries based on empirical mode decomposition and a long and short memory neural network," Energy, Elsevier, vol. 232(C).
    5. Han, Kunlun & Yang, Kai & Yin, Linfei, 2022. "Lightweight actor-critic generative adversarial networks for real-time smart generation control of microgrids," Applied Energy, Elsevier, vol. 317(C).
    6. Zhao, Chunyang & Andersen, Peter Bach & Træholt, Chresten & Hashemi, Seyedmostafa, 2023. "Grid-connected battery energy storage system: a review on application and integration," Renewable and Sustainable Energy Reviews, Elsevier, vol. 182(C).
    7. Andrade, Carlos & Selosse, Sandrine & Maïzi, Nadia, 2022. "The role of power-to-gas in the integration of variable renewables," Applied Energy, Elsevier, vol. 313(C).
    8. Wu, Long & Yin, Xunyuan & Pan, Lei & Liu, Jinfeng, 2022. "Economic model predictive control of integrated energy systems: A multi-time-scale framework," Applied Energy, Elsevier, vol. 328(C).
    9. Daraz, Amil, 2023. "Optimized cascaded controller for frequency stabilization of marine microgrid system," Applied Energy, Elsevier, vol. 350(C).
    10. Li, Hao & Misra, Siddharth, 2021. "Reinforcement learning based automated history matching for improved hydrocarbon production forecast," Applied Energy, Elsevier, vol. 284(C).
    11. Yin, Linfei & Luo, Shikui & Ma, Chenxiao, 2021. "Expandable depth and width adaptive dynamic programming for economic smart generation control of smart grids," Energy, Elsevier, vol. 232(C).
    12. Peng, Tian & Zhang, Chu & Zhou, Jianzhong & Nazir, Muhammad Shahzad, 2021. "An integrated framework of Bi-directional long-short term memory (BiLSTM) based on sine cosine algorithm for hourly solar radiation forecasting," Energy, Elsevier, vol. 221(C).
    13. Cao, Maosen & Shao, Changzheng & Hu, Bo & Xie, Kaigui & Zhou, Jiahao & Leng, Haimo & Zhang, Weixin, 2022. "Reliability tracing of the integrated energy system using the improved shapley value," Energy, Elsevier, vol. 260(C).
    14. Xiao, Zenan & Huang, Xiaoqiao & Liu, Jun & Li, Chengli & Tai, Yonghang, 2023. "A novel method based on time series ensemble model for hourly photovoltaic power prediction," Energy, Elsevier, vol. 276(C).
    15. Bu, Chujie & Cui, Xueqin & Li, Ruiyao & Li, Jin & Zhang, Yaxin & Wang, Can & Cai, Wenjia, 2021. "Achieving net-zero emissions in China’s passenger transport sector through regionally tailored mitigation strategies," Applied Energy, Elsevier, vol. 284(C).
    16. Makolo, Peter & Zamora, Ramon & Lie, Tek-Tjing, 2021. "The role of inertia for grid flexibility under high penetration of variable renewables - A review of challenges and solutions," Renewable and Sustainable Energy Reviews, Elsevier, vol. 147(C).
    17. Chen, Xi & Wang, Chunxi & Xu, Jianghai & Long, Shichun & Chai, Fasen & Li, Wenbin & Song, Xingxing & Wang, Xuepeng & Wan, Zhongmin, 2023. "Membrane humidity control of proton exchange membrane fuel cell system using fractional-order PID strategy," Applied Energy, Elsevier, vol. 343(C).
    18. Song, Yixuan & He, Mengxi & Wang, Yudong & Zhang, Yaojie, 2023. "Forecasting crude oil prices: A reduced-rank approach," International Review of Economics & Finance, Elsevier, vol. 88(C), pages 698-711.
    19. Yin, Linfei & Wu, Yunzhi, 2022. "Mode-decomposition memory reinforcement network strategy for smart generation control in multi-area power systems containing renewable energy," Applied Energy, Elsevier, vol. 307(C).
    20. Yin, Linfei & Zhang, Bin, 2021. "Time series generative adversarial network controller for long-term smart generation control of microgrids," Applied Energy, Elsevier, vol. 281(C).
    21. Yin, Linfei & He, Xiaoyu, 2023. "Artificial emotional deep Q learning for real-time smart voltage control of cyber-physical social power systems," Energy, Elsevier, vol. 273(C).
    22. Meng, Huixing & Geng, Mengyao & Xing, Jinduo & Zio, Enrico, 2022. "A hybrid method for prognostics of lithium-ion batteries capacity considering regeneration phenomena," Energy, Elsevier, vol. 261(PB).
    23. Ahajjam, Mohamed Aymane & Bonilla Licea, Daniel & Ghogho, Mounir & Kobbane, Abdellatif, 2022. "Experimental investigation of variational mode decomposition and deep learning for short-term multi-horizon residential electric load forecasting," Applied Energy, Elsevier, vol. 326(C).
    24. Acikgoz, Hakan, 2022. "A novel approach based on integration of convolutional neural networks and deep feature selection for short-term solar radiation forecasting," Applied Energy, Elsevier, vol. 305(C).
    25. Wang, Liying & Lin, Jialin & Dong, Houqi & Wang, Yuqing & Zeng, Ming, 2023. "Demand response comprehensive incentive mechanism-based multi-time scale optimization scheduling for park integrated energy system," Energy, Elsevier, vol. 270(C).
    26. Zhang, Yituo & Li, Chaolin & Jiang, Yiqi & Zhao, Ruobin & Yan, Kefen & Wang, Wenhui, 2023. "A hybrid model combining mode decomposition and deep learning algorithms for detecting TP in urban sewer networks," Applied Energy, Elsevier, vol. 333(C).
    27. Li, Bo & Li, Xu & Su, Qingyu, 2022. "A system and game strategy for the isolated island electric-gas deeply coupled energy network," Applied Energy, Elsevier, vol. 306(PA).
    28. Zheng, Yi & Wang, Jiawei & You, Shi & Li, Ximei & Bindner, Henrik W. & Münster, Marie, 2023. "Data-driven scheme for optimal day-ahead operation of a wind/hydrogen system under multiple uncertainties," Applied Energy, Elsevier, vol. 329(C).
    29. Yin, Linfei & Zhang, Bin, 2023. "Relaxed deep generative adversarial networks for real-time economic smart generation dispatch and control of integrated energy systems," Applied Energy, Elsevier, vol. 330(PA).
    30. Yin, Linfei & Zhao, Lulin, 2021. "Rejectable deep differential dynamic programming for real-time integrated generation dispatch and control of micro-grids," Energy, Elsevier, vol. 225(C).
    31. Han, Yan & Mi, Lihua & Shen, Lian & Cai, C.S. & Liu, Yuchen & Li, Kai & Xu, Guoji, 2022. "A short-term wind speed prediction method utilizing novel hybrid deep learning algorithms to correct numerical weather forecasting," Applied Energy, Elsevier, vol. 312(C).
    32. Nondy, J. & Gogoi, T.K., 2022. "Tri-objective optimization of two recuperative gas turbine-based CCHP systems and 4E analyses at optimal conditions," Applied Energy, Elsevier, vol. 323(C).
    33. Ding, Shixing & Gu, Wei & Lu, Shuai & Yu, Ruizhi & Sheng, Lina, 2022. "Cyber-attack against heating system in integrated energy systems: Model and propagation mechanism," Applied Energy, Elsevier, vol. 311(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Han, Kunlun & Yang, Kai & Yin, Linfei, 2022. "Lightweight actor-critic generative adversarial networks for real-time smart generation control of microgrids," Applied Energy, Elsevier, vol. 317(C).
    2. Yin, Linfei & He, Xiaoyu, 2023. "Artificial emotional deep Q learning for real-time smart voltage control of cyber-physical social power systems," Energy, Elsevier, vol. 273(C).
    3. Xuliang Tang & Heng Wan & Weiwen Wang & Mengxu Gu & Linfeng Wang & Linfeng Gan, 2023. "Lithium-Ion Battery Remaining Useful Life Prediction Based on Hybrid Model," Sustainability, MDPI, vol. 15(7), pages 1-18, April.
    4. Duan, Jikai & Zuo, Hongchao & Bai, Yulong & Chang, Mingheng & Chen, Xiangyue & Wang, Wenpeng & Ma, Lei & Chen, Bolong, 2023. "A multistep short-term solar radiation forecasting model using fully convolutional neural networks and chaotic aquila optimization combining WRF-Solar model results," Energy, Elsevier, vol. 271(C).
    5. Liu, Jingxuan & Zang, Haixiang & Cheng, Lilin & Ding, Tao & Wei, Zhinong & Sun, Guoqiang, 2023. "A Transformer-based multimodal-learning framework using sky images for ultra-short-term solar irradiance forecasting," Applied Energy, Elsevier, vol. 342(C).
    6. Yin, Linfei & Li, Yu, 2022. "Hybrid multi-agent emotional deep Q network for generation control of multi-area integrated energy systems," Applied Energy, Elsevier, vol. 324(C).
    7. Wang, Yuwei & Song, Minghao & Jia, Mengyao & Shi, Lin & Li, Bingkang, 2023. "TimeGAN based distributionally robust optimization for biomass-photovoltaic-hydrogen scheduling under source-load-market uncertainties," Energy, Elsevier, vol. 284(C).
    8. Xiong, Jinlin & Peng, Tian & Tao, Zihan & Zhang, Chu & Song, Shihao & Nazir, Muhammad Shahzad, 2023. "A dual-scale deep learning model based on ELM-BiLSTM and improved reptile search algorithm for wind power prediction," Energy, Elsevier, vol. 266(C).
    9. Yin, Linfei & Luo, Shikui & Ma, Chenxiao, 2021. "Expandable depth and width adaptive dynamic programming for economic smart generation control of smart grids," Energy, Elsevier, vol. 232(C).
    10. Jiawei Wang & Aidong Zeng & Yaheng Wan, 2023. "Multi-Time-Scale Optimal Scheduling of Integrated Energy System Considering Transmission Delay and Heat Storage of Heating Network," Sustainability, MDPI, vol. 15(19), pages 1-26, September.
    11. Yin, Linfei & Wu, Yunzhi, 2022. "Mode-decomposition memory reinforcement network strategy for smart generation control in multi-area power systems containing renewable energy," Applied Energy, Elsevier, vol. 307(C).
    12. Yin, Linfei & Zhang, Bin, 2023. "Relaxed deep generative adversarial networks for real-time economic smart generation dispatch and control of integrated energy systems," Applied Energy, Elsevier, vol. 330(PA).
    13. Su, Qingyu & Chen, Cong & Huang, Xin & Li, Jian, 2022. "Interval TrendRank method for grid node importance assessment considering new energy," Applied Energy, Elsevier, vol. 324(C).
    14. Guan, Keqin & Gong, Xu, 2023. "A new hybrid deep learning model for monthly oil prices forecasting," Energy Economics, Elsevier, vol. 128(C).
    15. Cao, Yisheng & Liu, Gang & Luo, Donghua & Bavirisetti, Durga Prasad & Xiao, Gang, 2023. "Multi-timescale photovoltaic power forecasting using an improved Stacking ensemble algorithm based LSTM-Informer model," Energy, Elsevier, vol. 283(C).
    16. Li, Min & Yang, Yi & He, Zhaoshuang & Guo, Xinbo & Zhang, Ruisheng & Huang, Bingqing, 2023. "A wind speed forecasting model based on multi-objective algorithm and interpretability learning," Energy, Elsevier, vol. 269(C).
    17. Wang, Huan & Li, Yan-Fu & Zhang, Ying, 2023. "Bioinspired spiking spatiotemporal attention framework for lithium-ion batteries state-of-health estimation," Renewable and Sustainable Energy Reviews, Elsevier, vol. 188(C).
    18. Abadie, Luis Mª & Chamorro, José M., 2023. "Investment in wind-based hydrogen production under economic and physical uncertainties," Applied Energy, Elsevier, vol. 337(C).
    19. Shahjalal, Mohammad & Roy, Probir Kumar & Shams, Tamanna & Fly, Ashley & Chowdhury, Jahedul Islam & Ahmed, Md. Rishad & Liu, Kailong, 2022. "A review on second-life of Li-ion batteries: prospects, challenges, and issues," Energy, Elsevier, vol. 241(C).
    20. Fang, Lei & He, Bin, 2023. "A deep learning framework using multi-feature fusion recurrent neural networks for energy consumption forecasting," Applied Energy, Elsevier, vol. 348(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:355:y:2024:i:c:s0306261923016100. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.