IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v324y2022ics0306261922009473.html

Multi-agent energy management optimization for integrated energy systems under the energy and carbon co-trading market

Author

Listed:
  • Sun, Qingkai
  • Wang, Xiaojun
  • Liu, Zhao
  • Mirsaeidi, Sohrab
  • He, Jinghan
  • Pei, Wei

Abstract

With the development trends of carbon neutrality, carbon trading is graduating embedded into the energy management of integrated energy systems (IESs). The dual benefices of carbon emission reduction and economics can be achieved by coordinatively optimizing the complementarity and flexibility between multiple IESs. However, this leads to the increased complexity of the market transactions, which poses significant challenges in terms of the benefits distribution among multiple entities, the convergence of trading processes, and the privacy-preserving issues. The multi-agent reinforcement learning (MARL) is capable of solving complex sequential-decision problems and acquiring the optimal strategies for each entity through the interactions between the multiple agents and the market. The MARL deployed on the local agent can provide online trading decisions for individual market entities considering their own interests, which offers new potential to solve the abovementioned difficulties. In this paper, we proposed an IESs co-trading market including electricity, natural gas and carbon trading. The multi-agent energy management coordinative optimization problem is solved by an improved Multi-agent Deep Deterministic Policy Gradient (MADDPG) algorithm to achieve fair trade and entity privacy protection. The case study results verify that the proposed optimal energy management strategy based on the improved MADDPG algorithm can efficiently guide the IESs in the energy and carbon co-trading market.

Suggested Citation

  • Sun, Qingkai & Wang, Xiaojun & Liu, Zhao & Mirsaeidi, Sohrab & He, Jinghan & Pei, Wei, 2022. "Multi-agent energy management optimization for integrated energy systems under the energy and carbon co-trading market," Applied Energy, Elsevier, vol. 324(C).
    • Handle: RePEc:eee:appene:v:324:y:2022:i:c:s0306261922009473
    DOI: 10.1016/j.apenergy.2022.119646
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261922009473
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2022.119646?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Chang, Soowon & Cho, Junyoung & Heo, Jae & Kang, Junsuk & Kobashi, Takuro, 2022. "Energy infrastructure transitions with PV and EV combined systems using techno-economic analyses for decarbonization in cities," Applied Energy, Elsevier, vol. 319(C).
    2. Perera, A.T.D. & Kamalaruban, Parameswaran, 2021. "Applications of reinforcement learning in energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    3. Gharibpour, Hassan & Aminifar, Farrokh & Rahmati, Iman & Keshavarz, Arezou, 2021. "Dual variable decomposition to discriminate the cost imposed by inflexible units in electricity markets," Applied Energy, Elsevier, vol. 287(C).
    4. Akinola, Toluleke E. & Bonilla Prado, Phebe L. & Wang, Meihong, 2022. "Experimental studies, molecular simulation and process modelling\simulation of adsorption-based post-combustion carbon capture for power plants: A state-of-the-art review," Applied Energy, Elsevier, vol. 317(C).
    5. Lin, Boqiang & Li, Zheng, 2022. "Towards world's low carbon development: The role of clean energy," Applied Energy, Elsevier, vol. 307(C).
    6. Gržanić, M. & Capuder, T. & Zhang, N. & Huang, W., 2022. "Prosumers as active market participants: A systematic review of evolution of opportunities, models and challenges," Renewable and Sustainable Energy Reviews, Elsevier, vol. 154(C).
    7. Maneesha, Ampolu & Swarup, K. Shanti, 2021. "A survey on applications of Alternating Direction Method of Multipliers in smart power grids," Renewable and Sustainable Energy Reviews, Elsevier, vol. 152(C).
    8. Zhang, Yanfang & Guo, Siyuan & Shi, Xunpeng & Qian, Xiangyan & Nie, Rui, 2021. "A market instrument to achieve carbon neutrality: Is China’s energy-consumption permit trading scheme effective?," Applied Energy, Elsevier, vol. 299(C).
    9. Farrokhifar, Meisam & Nie, Yinghui & Pozo, David, 2020. "Energy systems planning: A survey on models for integrated power and natural gas networks coordination," Applied Energy, Elsevier, vol. 262(C).
    10. Wang, Rutian & Wen, Xiangyun & Wang, Xiuyun & Fu, Yanbo & Zhang, Yu, 2022. "Low carbon optimal operation of integrated energy system based on carbon capture technology, LCA carbon emissions and ladder-type carbon trading," Applied Energy, Elsevier, vol. 311(C).
    11. Hou, Qingchun & Zhang, Ning & Du, Ershun & Miao, Miao & Peng, Fei & Kang, Chongqing, 2019. "Probabilistic duck curve in high PV penetration power system: Concept, modeling, and empirical analysis in China," Applied Energy, Elsevier, vol. 242(C), pages 205-215.
    12. Liu, Xiaojia & An, Haizhong & Wang, Lijun & Jia, Xiaoliang, 2017. "An integrated approach to optimize moving average rules in the EUA futures market based on particle swarm optimization and genetic algorithms," Applied Energy, Elsevier, vol. 185(P2), pages 1778-1787.
    13. Bürger, Adrian & Bohlayer, Markus & Hoffmann, Sarah & Altmann-Dieses, Angelika & Braun, Marco & Diehl, Moritz, 2020. "A whole-year simulation study on nonlinear mixed-integer model predictive control for a thermal energy supply system with multi-use components," Applied Energy, Elsevier, vol. 258(C).
    14. Guo, Caishan & Luo, Fengji & Cai, Zexiang & Dong, Zhao Yang, 2021. "Integrated energy systems of data centers and smart grids: State-of-the-art and future opportunities," Applied Energy, Elsevier, vol. 301(C).
    15. Berjawi, A.E.H. & Walker, S.L. & Patsios, C. & Hosseini, S.H.R., 2021. "An evaluation framework for future integrated energy systems: A whole energy systems approach," Renewable and Sustainable Energy Reviews, Elsevier, vol. 145(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Liang, Ziwen & Mu, Longhua, 2024. "Multi-agent low-carbon optimal dispatch of regional integrated energy system based on mixed game theory," Energy, Elsevier, vol. 295(C).
    2. Yang, Ting & Wang, Qiancheng & Wang, Xudong & Wang, Lin & Geng, Yinan, 2025. "Low-carbon economic distributed dispatch for district-level integrated energy system considering privacy protection and demand response," Applied Energy, Elsevier, vol. 383(C).
    3. Zhou, Jianli & Ran, Jia & Ren, Jiayi & Zhao, Ruige & Wang, Yaqi & Wu, Yunna, 2025. "Digital intelligence-driven synergistic optimization of capacity configuration for wind-solar-hydrogen multi-energy systems integrated with shared energy storage," Energy, Elsevier, vol. 341(C).
    4. Qiu, Haifeng & Vinod, Ashwin & Lu, Shuai & Gooi, Hoay Beng & Pan, Guangsheng & Zhang, Suhan & Veerasamy, Veerapandiyan, 2023. "Decentralized mixed-integer optimization for robust integrated electricity and heat scheduling," Applied Energy, Elsevier, vol. 350(C).
    5. Chen, Jie & Gu, Weiyu & Alharthi, Yahya Z. & Huang, Shoujun & Mansouri, Seyed Amir, 2025. "A decentralized framework for self-healing in hydrogen-integrated energy systems," Energy, Elsevier, vol. 331(C).
    6. Sharma, Abhimanyu & Padhy, Narayana Prasad, 2024. "Iterative convex relaxation of unbalanced power distribution system integrated multi-energy systems," Energy, Elsevier, vol. 294(C).
    7. Yao, Ganzhou & Luo, Zirong & Lu, Zhongyue & Wang, Mangkuan & Shang, Jianzhong & Guerrerob, Josep M., 2023. "Unlocking the potential of wave energy conversion: A comprehensive evaluation of advanced maximum power point tracking techniques and hybrid strategies for sustainable energy harvesting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 185(C).
    8. Debnath, Kumar Biswajit & Jenkins, David P. & Patidar, Sandhya & Peacock, Andrew D., 2024. "Remote work might unlock solar PV's potential of cracking the ‘Duck Curve’," Applied Energy, Elsevier, vol. 367(C).
    9. Østergaard, P.A. & Lund, H. & Thellufsen, J.Z. & Sorknæs, P. & Mathiesen, B.V., 2022. "Review and validation of EnergyPLAN," Renewable and Sustainable Energy Reviews, Elsevier, vol. 168(C).
    10. Deng, Jin & Li, Kuo & Fu, Xiaolan & Feng, Youneng & Yuan, Shenfu, 2025. "Effects of Na and Ca on gaseous products evolution behaviors for Fe catalyzed lignite pyrolysis volatiles reforming," Energy, Elsevier, vol. 320(C).
    11. Cong, Yingnan & Hou, Yufei & Ji, Yuan & Cai, Xiaojing, 2026. "Does innovation-driven policy optimize urban energy consumption? Evidence from China’s innovation-driven city pilot policies," The North American Journal of Economics and Finance, Elsevier, vol. 81(C).
    12. Iqra Nazir & Nermish Mushtaq & Waqas Amin, 2025. "Smart Grid Systems: Addressing Privacy Threats, Security Vulnerabilities, and Demand–Supply Balance (A Review)," Energies, MDPI, vol. 18(19), pages 1-77, September.
    13. Jiaqi Wu & Qian Zhang & Yangdong Lu & Tianxi Qin & Jianyong Bai, 2023. "Source-Load Coordinated Low-Carbon Economic Dispatch of Microgrid including Electric Vehicles," Sustainability, MDPI, vol. 15(21), pages 1-21, October.
    14. Zhang, Huacheng & Umehara, Yutaro & Horiguchi, Naoki & Yoshida, Hiroyuki & Etoh, Atsuroh & Mori, Shoji, 2025. "Experimental simulation of high-temperature and high-pressure annular two-phase flow using an HFC134a–ethanol system: Characterization of disturbance wave flow," Energy, Elsevier, vol. 335(C).
    15. Zhang, Zhonglian & Yang, Xiaohui & Li, Moxuan & Deng, Fuwei & Xiao, Riying & Mei, Linghao & Hu, Zecheng, 2023. "Optimal configuration of improved dynamic carbon neutral energy systems based on hybrid energy storage and market incentives," Energy, Elsevier, vol. 284(C).
    16. Zhang, Yanfang & Wei, Jinpeng & Gao, Qi & Shi, Xunpeng & Zhou, Dequn, 2022. "Coordination between the energy-consumption permit trading scheme and carbon emissions trading: Evidence from China," Energy Economics, Elsevier, vol. 116(C).
    17. Huang, Wenyang & Zhao, Jianyu & Wang, Xiaokang, 2024. "Model-driven multimodal LSTM-CNN for unbiased structural forecasting of European Union allowances open-high-low-close price," Energy Economics, Elsevier, vol. 132(C).
    18. Rongjiang Cai & Tao Lv & Cheng Wang & Nana Liu, 2023. "Can Environmental Information Disclosure Enhance Firm Value?—An Analysis Based on Textual Characteristics of Annual Reports," IJERPH, MDPI, vol. 20(5), pages 1-21, February.
    19. Mehar Ullah & Daniel Gutierrez-Rojas & Eero Inkeri & Tero Tynjälä & Pedro H. J. Nardelli, 2022. "Operation of Power-to-X-Related Processes Based on Advanced Data-Driven Methods: A Comprehensive Review," Energies, MDPI, vol. 15(21), pages 1-17, October.
    20. Jiyong Li & Zeyi Hua & Lin Tian & Peiwen Chen & Hao Dong, 2024. "Optimal Capacity Allocation for Life Cycle Multiobjective Integrated Energy Systems Considering Capacity Tariffs and Eco-Indicator 99," Sustainability, MDPI, vol. 16(20), pages 1-22, October.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:324:y:2022:i:c:s0306261922009473. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.