IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v324y2022ics0306261922009473.html
   My bibliography  Save this article

Multi-agent energy management optimization for integrated energy systems under the energy and carbon co-trading market

Author

Listed:
  • Sun, Qingkai
  • Wang, Xiaojun
  • Liu, Zhao
  • Mirsaeidi, Sohrab
  • He, Jinghan
  • Pei, Wei

Abstract

With the development trends of carbon neutrality, carbon trading is graduating embedded into the energy management of integrated energy systems (IESs). The dual benefices of carbon emission reduction and economics can be achieved by coordinatively optimizing the complementarity and flexibility between multiple IESs. However, this leads to the increased complexity of the market transactions, which poses significant challenges in terms of the benefits distribution among multiple entities, the convergence of trading processes, and the privacy-preserving issues. The multi-agent reinforcement learning (MARL) is capable of solving complex sequential-decision problems and acquiring the optimal strategies for each entity through the interactions between the multiple agents and the market. The MARL deployed on the local agent can provide online trading decisions for individual market entities considering their own interests, which offers new potential to solve the abovementioned difficulties. In this paper, we proposed an IESs co-trading market including electricity, natural gas and carbon trading. The multi-agent energy management coordinative optimization problem is solved by an improved Multi-agent Deep Deterministic Policy Gradient (MADDPG) algorithm to achieve fair trade and entity privacy protection. The case study results verify that the proposed optimal energy management strategy based on the improved MADDPG algorithm can efficiently guide the IESs in the energy and carbon co-trading market.

Suggested Citation

  • Sun, Qingkai & Wang, Xiaojun & Liu, Zhao & Mirsaeidi, Sohrab & He, Jinghan & Pei, Wei, 2022. "Multi-agent energy management optimization for integrated energy systems under the energy and carbon co-trading market," Applied Energy, Elsevier, vol. 324(C).
    • Handle: RePEc:eee:appene:v:324:y:2022:i:c:s0306261922009473
    DOI: 10.1016/j.apenergy.2022.119646
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261922009473
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2022.119646?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Chang, Soowon & Cho, Junyoung & Heo, Jae & Kang, Junsuk & Kobashi, Takuro, 2022. "Energy infrastructure transitions with PV and EV combined systems using techno-economic analyses for decarbonization in cities," Applied Energy, Elsevier, vol. 319(C).
    2. Lin, Boqiang & Li, Zheng, 2022. "Towards world's low carbon development: The role of clean energy," Applied Energy, Elsevier, vol. 307(C).
    3. Wang, Rutian & Wen, Xiangyun & Wang, Xiuyun & Fu, Yanbo & Zhang, Yu, 2022. "Low carbon optimal operation of integrated energy system based on carbon capture technology, LCA carbon emissions and ladder-type carbon trading," Applied Energy, Elsevier, vol. 311(C).
    4. Liu, Xiaojia & An, Haizhong & Wang, Lijun & Jia, Xiaoliang, 2017. "An integrated approach to optimize moving average rules in the EUA futures market based on particle swarm optimization and genetic algorithms," Applied Energy, Elsevier, vol. 185(P2), pages 1778-1787.
    5. Gharibpour, Hassan & Aminifar, Farrokh & Rahmati, Iman & Keshavarz, Arezou, 2021. "Dual variable decomposition to discriminate the cost imposed by inflexible units in electricity markets," Applied Energy, Elsevier, vol. 287(C).
    6. Gržanić, M. & Capuder, T. & Zhang, N. & Huang, W., 2022. "Prosumers as active market participants: A systematic review of evolution of opportunities, models and challenges," Renewable and Sustainable Energy Reviews, Elsevier, vol. 154(C).
    7. Maneesha, Ampolu & Swarup, K. Shanti, 2021. "A survey on applications of Alternating Direction Method of Multipliers in smart power grids," Renewable and Sustainable Energy Reviews, Elsevier, vol. 152(C).
    8. Bürger, Adrian & Bohlayer, Markus & Hoffmann, Sarah & Altmann-Dieses, Angelika & Braun, Marco & Diehl, Moritz, 2020. "A whole-year simulation study on nonlinear mixed-integer model predictive control for a thermal energy supply system with multi-use components," Applied Energy, Elsevier, vol. 258(C).
    9. Farrokhifar, Meisam & Nie, Yinghui & Pozo, David, 2020. "Energy systems planning: A survey on models for integrated power and natural gas networks coordination," Applied Energy, Elsevier, vol. 262(C).
    10. Guo, Caishan & Luo, Fengji & Cai, Zexiang & Dong, Zhao Yang, 2021. "Integrated energy systems of data centers and smart grids: State-of-the-art and future opportunities," Applied Energy, Elsevier, vol. 301(C).
    11. Perera, A.T.D. & Kamalaruban, Parameswaran, 2021. "Applications of reinforcement learning in energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    12. Akinola, Toluleke E. & Bonilla Prado, Phebe L. & Wang, Meihong, 2022. "Experimental studies, molecular simulation and process modelling\simulation of adsorption-based post-combustion carbon capture for power plants: A state-of-the-art review," Applied Energy, Elsevier, vol. 317(C).
    13. Zhang, Yanfang & Guo, Siyuan & Shi, Xunpeng & Qian, Xiangyan & Nie, Rui, 2021. "A market instrument to achieve carbon neutrality: Is China’s energy-consumption permit trading scheme effective?," Applied Energy, Elsevier, vol. 299(C).
    14. Hou, Qingchun & Zhang, Ning & Du, Ershun & Miao, Miao & Peng, Fei & Kang, Chongqing, 2019. "Probabilistic duck curve in high PV penetration power system: Concept, modeling, and empirical analysis in China," Applied Energy, Elsevier, vol. 242(C), pages 205-215.
    15. Berjawi, A.E.H. & Walker, S.L. & Patsios, C. & Hosseini, S.H.R., 2021. "An evaluation framework for future integrated energy systems: A whole energy systems approach," Renewable and Sustainable Energy Reviews, Elsevier, vol. 145(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Du, Minzhe & Wu, Fenger & Ye, Danfeng & Zhao, Yating & Liao, Liping, 2023. "Exploring the effects of energy quota trading policy on carbon emission efficiency: Quasi-experimental evidence from China," Energy Economics, Elsevier, vol. 124(C).
    2. Liu, Tianhao & Tian, Jun & Zhu, Hongyu & Goh, Hui Hwang & Liu, Hui & Wu, Thomas & Zhang, Dongdong, 2023. "Key technologies and developments of multi-energy system: Three-layer framework, modelling and optimisation," Energy, Elsevier, vol. 277(C).
    3. Zhou, Yanting & Ma, Zhongjing & Shi, Xingyu & Zou, Suli, 2024. "Multi-agent optimal scheduling for integrated energy system considering the global carbon emission constraint," Energy, Elsevier, vol. 288(C).
    4. Jiang, Kai & Wang, Kunyu & Wu, Chengyu & Chen, Guo & Xue, Yusheng & Dong, Zhaoyang & Liu, Nian, 2024. "Trajectory simulation and optimization for interactive electricity-carbon system evolution," Applied Energy, Elsevier, vol. 360(C).
    5. Liu, Dewen & Luo, Zhao & Qin, Jinghui & Wang, Hua & Wang, Gang & Li, Zhao & Zhao, Weijie & Shen, Xin, 2023. "Low-carbon dispatch of multi-district integrated energy systems considering carbon emission trading and green certificate trading," Renewable Energy, Elsevier, vol. 218(C).
    6. Jiang, Tao & Dong, Xinru & Zhang, Rufeng & Li, Xue, 2023. "Strategic active and reactive power scheduling of integrated community energy systems in day-ahead distribution electricity market," Applied Energy, Elsevier, vol. 336(C).
    7. Wang, Yuwei & Song, Minghao & Jia, Mengyao & Shi, Lin & Li, Bingkang, 2023. "TimeGAN based distributionally robust optimization for biomass-photovoltaic-hydrogen scheduling under source-load-market uncertainties," Energy, Elsevier, vol. 284(C).
    8. Yun, Yunyun & Zhang, Dahai & Yang, Shengchun & Li, Yaping & Yan, Jiahao, 2023. "Low-carbon optimal dispatch of integrated energy system considering the operation of oxy-fuel combustion coupled with power-to-gas and hydrogen-doped gas equipment," Energy, Elsevier, vol. 283(C).
    9. Guo, Xiaopeng & Zhang, Xinyue & Zhang, Xingping, 2024. "Incentive-oriented power‑carbon emissions trading-tradable green certificate integrated market mechanisms using multi-agent deep reinforcement learning," Applied Energy, Elsevier, vol. 357(C).
    10. Zhang, Xinyue & Guo, Xiaopeng & Zhang, Xingping, 2023. "Assessing the policy synergy among power, carbon emissions trading and tradable green certificate market mechanisms on strategic GENCOs in China," Energy, Elsevier, vol. 278(PB).
    11. Yang Li & Rongqiang Li & Linjun Shi & Feng Wu & Jianhua Zhou & Jian Liu & Keman Lin, 2023. "Adjustable Capability Evaluation of Integrated Energy Systems Considering Demand Response and Economic Constraints," Energies, MDPI, vol. 16(24), pages 1-23, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Liang, Ziwen & Mu, Longhua, 2024. "Multi-agent low-carbon optimal dispatch of regional integrated energy system based on mixed game theory," Energy, Elsevier, vol. 295(C).
    2. Qiu, Haifeng & Vinod, Ashwin & Lu, Shuai & Gooi, Hoay Beng & Pan, Guangsheng & Zhang, Suhan & Veerasamy, Veerapandiyan, 2023. "Decentralized mixed-integer optimization for robust integrated electricity and heat scheduling," Applied Energy, Elsevier, vol. 350(C).
    3. Sharma, Abhimanyu & Padhy, Narayana Prasad, 2024. "Iterative convex relaxation of unbalanced power distribution system integrated multi-energy systems," Energy, Elsevier, vol. 294(C).
    4. Yao, Ganzhou & Luo, Zirong & Lu, Zhongyue & Wang, Mangkuan & Shang, Jianzhong & Guerrerob, Josep M., 2023. "Unlocking the potential of wave energy conversion: A comprehensive evaluation of advanced maximum power point tracking techniques and hybrid strategies for sustainable energy harvesting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 185(C).
    5. Debnath, Kumar Biswajit & Jenkins, David P. & Patidar, Sandhya & Peacock, Andrew D., 2024. "Remote work might unlock solar PV's potential of cracking the ‘Duck Curve’," Applied Energy, Elsevier, vol. 367(C).
    6. Østergaard, P.A. & Lund, H. & Thellufsen, J.Z. & Sorknæs, P. & Mathiesen, B.V., 2022. "Review and validation of EnergyPLAN," Renewable and Sustainable Energy Reviews, Elsevier, vol. 168(C).
    7. Jiaqi Wu & Qian Zhang & Yangdong Lu & Tianxi Qin & Jianyong Bai, 2023. "Source-Load Coordinated Low-Carbon Economic Dispatch of Microgrid including Electric Vehicles," Sustainability, MDPI, vol. 15(21), pages 1-21, October.
    8. Zhang, Zhonglian & Yang, Xiaohui & Li, Moxuan & Deng, Fuwei & Xiao, Riying & Mei, Linghao & Hu, Zecheng, 2023. "Optimal configuration of improved dynamic carbon neutral energy systems based on hybrid energy storage and market incentives," Energy, Elsevier, vol. 284(C).
    9. Zhang, Yanfang & Wei, Jinpeng & Gao, Qi & Shi, Xunpeng & Zhou, Dequn, 2022. "Coordination between the energy-consumption permit trading scheme and carbon emissions trading: Evidence from China," Energy Economics, Elsevier, vol. 116(C).
    10. Huang, Wenyang & Zhao, Jianyu & Wang, Xiaokang, 2024. "Model-driven multimodal LSTM-CNN for unbiased structural forecasting of European Union allowances open-high-low-close price," Energy Economics, Elsevier, vol. 132(C).
    11. Rongjiang Cai & Tao Lv & Cheng Wang & Nana Liu, 2023. "Can Environmental Information Disclosure Enhance Firm Value?—An Analysis Based on Textual Characteristics of Annual Reports," IJERPH, MDPI, vol. 20(5), pages 1-21, February.
    12. Mehar Ullah & Daniel Gutierrez-Rojas & Eero Inkeri & Tero Tynjälä & Pedro H. J. Nardelli, 2022. "Operation of Power-to-X-Related Processes Based on Advanced Data-Driven Methods: A Comprehensive Review," Energies, MDPI, vol. 15(21), pages 1-17, October.
    13. Jiyong Li & Zeyi Hua & Lin Tian & Peiwen Chen & Hao Dong, 2024. "Optimal Capacity Allocation for Life Cycle Multiobjective Integrated Energy Systems Considering Capacity Tariffs and Eco-Indicator 99," Sustainability, MDPI, vol. 16(20), pages 1-22, October.
    14. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    15. Zhu, Yilin & Xu, Yujie & Chen, Haisheng & Guo, Huan & Zhang, Hualiang & Zhou, Xuezhi & Shen, Haotian, 2023. "Optimal dispatch of a novel integrated energy system combined with multi-output organic Rankine cycle and hybrid energy storage," Applied Energy, Elsevier, vol. 343(C).
    16. Guorong Chen & Shiyi Fang & Qibo Chen & Yun Zhang, 2023. "Risk Spillovers and Network Connectedness between Clean Energy Stocks, Green Bonds, and Other Financial Assets: Evidence from China," Energies, MDPI, vol. 16(20), pages 1-21, October.
    17. D'Adamo, Idiano & Mammetti, Marco & Ottaviani, Dario & Ozturk, Ilhan, 2023. "Photovoltaic systems and sustainable communities: New social models for ecological transition. The impact of incentive policies in profitability analyses," Renewable Energy, Elsevier, vol. 202(C), pages 1291-1304.
    18. Day, Min-Yuh & Ni, Yensen, 2023. "The profitability of seasonal trading timing: Insights from energy-related markets," Energy Economics, Elsevier, vol. 128(C).
    19. Montuori, Lina & Alcázar-Ortega, Manuel, 2021. "Demand response strategies for the balancing of natural gas systems: Application to a local network located in The Marches (Italy)," Energy, Elsevier, vol. 225(C).
    20. Yang, Yuyan & Xu, Xiao & Pan, Li & Liu, Junyong & Liu, Jichun & Hu, Weihao, 2024. "Distributed prosumer trading in the electricity and carbon markets considering user utility," Renewable Energy, Elsevier, vol. 228(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:324:y:2022:i:c:s0306261922009473. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.