IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v399y2025ics0306261925011985.html

Safe-AutoSAC: AutoML-enhanced safe deep reinforcement learning for integrated energy system scheduling with multi-channel informer forecasting and electric vehicle demand response

Author

Listed:
  • Li, Yang
  • Zhao, Bingsong
  • Li, Yuanzheng
  • Long, Chao
  • Li, Sen
  • Dong, Zhaoyang
  • Shahidehpour, Mohammad

Abstract

To coordinate renewable uncertainties and electric vehicle (EV) demand response, this paper proposes a safe deep reinforcement learning-based optimal scheduling approach for integrated energy systems (IESs). An optimal scheduling model for IESs is established to maximize net profit through an integrated demand response (IDR) strategy driven by a dynamic pricing mechanism. This strategy comprehensively coordinates conventional flexible loads and EV demand response, thereby unlocking the potential of flexible resources. The scheduling model is formulated as a Markov decision process and solved using a safety-assured adaptive delay policy network and automated machine learning-enhanced soft actor-critic (Safe-AutoSAC) algorithm, ensuring that the generated scheduling decisions comply with IES power flow constraints. To improve renewable and load prediction accuracy, a novel multi-channel Informer forecasting model is proposed with cross-channel attention and multi-scale temporal embeddings for capturing spatiotemporal dependencies and periodic patterns. Experimental results demonstrate that our approach is superior to the traditional soft actor-critic algorithm, achieving a 7.06 % improvement in net profit while maintaining operational safety. Our forecasting method achieves high accuracy and outperforms alternative methods. Moreover, the implementation of the proposed IDR strategies enhances the operational flexibility of the IES and results in a 44.35 % increase in net profit compared to the baseline scenario without demand response.

Suggested Citation

  • Li, Yang & Zhao, Bingsong & Li, Yuanzheng & Long, Chao & Li, Sen & Dong, Zhaoyang & Shahidehpour, Mohammad, 2025. "Safe-AutoSAC: AutoML-enhanced safe deep reinforcement learning for integrated energy system scheduling with multi-channel informer forecasting and electric vehicle demand response," Applied Energy, Elsevier, vol. 399(C).
  • Handle: RePEc:eee:appene:v:399:y:2025:i:c:s0306261925011985
    DOI: 10.1016/j.apenergy.2025.126468
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261925011985
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.apenergy.2025.126468?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Zhang, Bin & Hu, Weihao & Xu, Xiao & Li, Tao & Zhang, Zhenyuan & Chen, Zhe, 2022. "Physical-model-free intelligent energy management for a grid-connected hybrid wind-microturbine-PV-EV energy system via deep reinforcement learning approach," Renewable Energy, Elsevier, vol. 200(C), pages 433-448.
    2. Xiaoshuang Huang & Yinbao Zhang & Jianzhong Liu & Xinjia Zhang & Sicong Liu, 2023. "A Short-Term Wind Power Forecasting Model Based on 3D Convolutional Neural Network–Gated Recurrent Unit," Sustainability, MDPI, vol. 15(19), pages 1-13, September.
    3. Li, Yang & Yang, Zhen & Li, Guoqing & Mu, Yunfei & Zhao, Dongbo & Chen, Chen & Shen, Bo, 2018. "Optimal scheduling of isolated microgrid with an electric vehicle battery swapping station in multi-stakeholder scenarios: A bi-level programming approach via real-time pricing," Applied Energy, Elsevier, vol. 232(C), pages 54-68.
    4. Zheng, Yanchong & Yu, Hang & Shao, Ziyun & Jian, Linni, 2020. "Day-ahead bidding strategy for electric vehicle aggregator enabling multiple agent modes in uncertain electricity markets," Applied Energy, Elsevier, vol. 280(C).
    5. Lv, Si & Wei, Zhinong & Chen, Sheng & Sun, Guoqiang & Wang, Dan, 2021. "Integrated demand response for congestion alleviation in coupled power and transportation networks," Applied Energy, Elsevier, vol. 283(C).
    6. Li, Yang & Bu, Fanjin & Li, Yuanzheng & Long, Chao, 2023. "Optimal scheduling of island integrated energy systems considering multi-uncertainties and hydrothermal simultaneous transmission: A deep reinforcement learning approach," Applied Energy, Elsevier, vol. 333(C).
    7. Wang, Kang & Wang, Haixin & Yang, Zihao & Feng, Jiawei & Li, Yanzhen & Yang, Junyou & Chen, Zhe, 2023. "A transfer learning method for electric vehicles charging strategy based on deep reinforcement learning," Applied Energy, Elsevier, vol. 343(C).
    8. Zhou, Yanting & Ma, Zhongjing & Shi, Xingyu & Zou, Suli, 2024. "Multi-agent optimal scheduling for integrated energy system considering the global carbon emission constraint," Energy, Elsevier, vol. 288(C).
    9. Li, Yang & Ma, Wenjie & Li, Yuanzheng & Li, Sen & Chen, Zhe & Shahidehpour, Mohammad, 2025. "Enhancing cyber-resilience in integrated energy system scheduling with demand response using deep reinforcement learning," Applied Energy, Elsevier, vol. 379(C).
    10. Mao, Ning & Hao, Jingyu & He, Tianbiao & Song, Mengjie & Xu, Yingjie & Deng, Shiming, 2019. "PMV-based dynamic optimization of energy consumption for a residential task/ambient air conditioning system in different climate zones," Renewable Energy, Elsevier, vol. 142(C), pages 41-54.
    11. Guo, Chenyu & Wang, Xin & Zheng, Yihui & Zhang, Feng, 2022. "Real-time optimal energy management of microgrid with uncertainties based on deep reinforcement learning," Energy, Elsevier, vol. 238(PC).
    12. Soleimanzade, Mohammad Amin & Kumar, Amit & Sadrzadeh, Mohtada, 2022. "Novel data-driven energy management of a hybrid photovoltaic-reverse osmosis desalination system using deep reinforcement learning," Applied Energy, Elsevier, vol. 317(C).
    13. Yang, Ting & Zhao, Liyuan & Li, Wei & Zomaya, Albert Y., 2021. "Dynamic energy dispatch strategy for integrated energy system based on improved deep reinforcement learning," Energy, Elsevier, vol. 235(C).
    14. Gong, Mingju & Yan, Changcheng & Xu, Wei & Zhao, Zhixuan & Li, Wenxiang & Liu, Yan & Li, Sheng, 2023. "Short-term wind power forecasting model based on temporal convolutional network and Informer," Energy, Elsevier, vol. 283(C).
    15. Liu, Lei & Liu, Jicheng & Ye, Yu & Liu, Hui & Chen, Kun & Li, Dong & Dong, Xue & Sun, Mingzhai, 2023. "Ultra-short-term wind power forecasting based on deep Bayesian model with uncertainty," Renewable Energy, Elsevier, vol. 205(C), pages 598-607.
    16. Alabi, Tobi Michael & Aghimien, Emmanuel I. & Agbajor, Favour D. & Yang, Zaiyue & Lu, Lin & Adeoye, Adebusola R. & Gopaluni, Bhushan, 2022. "A review on the integrated optimization techniques and machine learning approaches for modeling, prediction, and decision making on integrated energy systems," Renewable Energy, Elsevier, vol. 194(C), pages 822-849.
    17. Li, Yang & Wang, Ruinong & Li, Yuanzheng & Zhang, Meng & Long, Chao, 2023. "Wind power forecasting considering data privacy protection: A federated deep reinforcement learning approach," Applied Energy, Elsevier, vol. 329(C).
    18. Fan, Guozhu & Peng, Chunhua & Wang, Xuekui & Wu, Peng & Yang, Yifan & Sun, Huijuan, 2024. "Optimal scheduling of integrated energy system considering renewable energy uncertainties based on distributionally robust adaptive MPC," Renewable Energy, Elsevier, vol. 226(C).
    19. Zhang, Tengxi & Xin, Li & Wang, Shunjiang & Guo, Ren & Wang, Wentao & Cui, Jia & Wang, Peng, 2024. "A novel approach of energy and reserve scheduling for hybrid power systems: Frequency security constraints," Applied Energy, Elsevier, vol. 361(C).
    20. Cai, Qiran & Xu, Qingyang & Qing, Jing & Shi, Gang & Liang, Qiao-Mei, 2022. "Promoting wind and photovoltaics renewable energy integration through demand response: Dynamic pricing mechanism design and economic analysis for smart residential communities," Energy, Elsevier, vol. 261(PB).
    21. Gutsch, Moritz & Leker, Jens, 2024. "Costs, carbon footprint, and environmental impacts of lithium-ion batteries – From cathode active material synthesis to cell manufacturing and recycling," Applied Energy, Elsevier, vol. 353(PB).
    22. Zhang, Bin & Hu, Weihao & Cao, Di & Ghias, Amer M.Y.M. & Chen, Zhe, 2023. "Novel Data-Driven decentralized coordination model for electric vehicle aggregator and energy hub entities in multi-energy system using an improved multi-agent DRL approach," Applied Energy, Elsevier, vol. 339(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mahmud, Sakib & Sayed, Aya Nabil & Himeur, Yassine & Nhlabatsi, Armstrong & Bensaali, Faycal, 2026. "A comprehensive review of deep reinforcement learning applications from centralized power generation to modern energy internet frameworks," Renewable and Sustainable Energy Reviews, Elsevier, vol. 226(PE).
    2. Li, Feng & Hou, Hui & Ni, Tian & Wang, Pingfan & Wang, Yingxiang & Li, Zhengmao & Pouresmaeil, Edris, 2025. "A novel deep reinforcement learning framework for optimal scheduling of low-carbon integrated energy systems considering battery degradation," Energy, Elsevier, vol. 341(C).
    3. Cheng, Fang & Liu, Hui, 2024. "Multi-step electric vehicles charging loads forecasting: An autoformer variant with feature extraction, frequency enhancement, and error correction blocks," Applied Energy, Elsevier, vol. 376(PB).
    4. Chen, Juntao & Fu, Xueying & Zhang, Lingli & Shen, Haoye & Wu, Jibo, 2024. "A novel offshore wind power prediction model based on TCN-DANet-sparse transformer and considering spatio-temporal coupling in multiple wind farms," Energy, Elsevier, vol. 308(C).
    5. Jiankai Gao & Yang Li & Bin Wang & Haibo Wu, 2023. "Multi-Microgrid Collaborative Optimization Scheduling Using an Improved Multi-Agent Soft Actor-Critic Algorithm," Energies, MDPI, vol. 16(7), pages 1-21, April.
    6. Chen, Yuejiang & He, Yingjing & Xiao, Jiang-Wen & Wang, Yan-Wu & Li, Yuanzheng, 2024. "Hybrid model based on similar power extraction and improved temporal convolutional network for probabilistic wind power forecasting," Energy, Elsevier, vol. 304(C).
    7. Hong, Jun-Tao & Han, Shuang & Yan, Jie & Liu, Yong-Qian, 2025. "Dual-path frequency Mamba-Transformer model for wind power forecasting," Energy, Elsevier, vol. 332(C).
    8. Liu, Wei & Wang, Xian & Long, Qingcan & Zeng, Bing & Zhong, Shuai, 2025. "A sensitive and easy-to-deploy condition monitoring method for main drive chain of large wind turbines," Renewable Energy, Elsevier, vol. 254(C).
    9. Li, Yang & Bu, Fanjin & Li, Yuanzheng & Long, Chao, 2023. "Optimal scheduling of island integrated energy systems considering multi-uncertainties and hydrothermal simultaneous transmission: A deep reinforcement learning approach," Applied Energy, Elsevier, vol. 333(C).
    10. Yao, Xianshuang & Guo, Kangshuai & Lei, Jianqi & Li, Xuanyu, 2024. "Fully connected multi-reservoir echo state networks for wind power prediction," Energy, Elsevier, vol. 312(C).
    11. Gang Li & Chen Lin & Yupeng Li, 2025. "Probabilistic Forecasting of Provincial Regional Wind Power Considering Spatio-Temporal Features," Energies, MDPI, vol. 18(3), pages 1-17, January.
    12. Chen, Dongyu & Lin, Xiaojie & Qiao, Yiyuan, 2025. "Perspectives for artificial intelligence in sustainable energy systems," Energy, Elsevier, vol. 318(C).
    13. Li, Yang & Wang, Bin & Yang, Zhen & Li, Jiazheng & Chen, Chen, 2022. "Hierarchical stochastic scheduling of multi-community integrated energy systems in uncertain environments via Stackelberg game," Applied Energy, Elsevier, vol. 308(C).
    14. Wang, Qi & Huang, Chunyi & Wang, Chengmin & Li, Kangping & Xie, Ning, 2024. "Joint optimization of bidding and pricing strategy for electric vehicle aggregator considering multi-agent interactions," Applied Energy, Elsevier, vol. 360(C).
    15. Jin, Huaiping & Zhang, Kehao & Fan, Shouyuan & Jin, Huaikang & Wang, Bin, 2024. "Wind power forecasting based on ensemble deep learning with surrogate-assisted evolutionary neural architecture search and many-objective federated learning," Energy, Elsevier, vol. 308(C).
    16. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    17. Gerald Jones & Xueping Li & Yulin Sun, 2024. "Robust Energy Management Policies for Solar Microgrids via Reinforcement Learning," Energies, MDPI, vol. 17(12), pages 1-22, June.
    18. Bio Gassi, Karim & Baysal, Mustafa, 2023. "Improving real-time energy decision-making model with an actor-critic agent in modern microgrids with energy storage devices," Energy, Elsevier, vol. 263(PE).
    19. Yadav, Subhash & Kumar, Pradeep & Kumar, Ashwani, 2025. "Hybrid renewable energy systems design and techno-economic analysis for isolated rural microgrid using HOMER," Energy, Elsevier, vol. 327(C).
    20. Zhao, Wanbing & Chang, Weiguang & Yang, Qiang, 2024. "Collaborative energy management of interconnected regional integrated energy systems considering spatio-temporal characteristics," Renewable Energy, Elsevier, vol. 235(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:399:y:2025:i:c:s0306261925011985. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.