IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i24p9633-d1008005.html
   My bibliography  Save this article

Peer-to-Peer Trading for Energy-Saving Based on Reinforcement Learning

Author

Listed:
  • Liangyi Pu

    (Chongqing Huizhi Energy Corporation Ltd., State Power Investment Corporation (SPIC), Chongqing 401127, China
    These authors contributed equally to this work.)

  • Song Wang

    (Chongqing Huizhi Energy Corporation Ltd., State Power Investment Corporation (SPIC), Chongqing 401127, China
    These authors contributed equally to this work.)

  • Xiaodong Huang

    (Chongqing Huizhi Energy Corporation Ltd., State Power Investment Corporation (SPIC), Chongqing 401127, China
    These authors contributed equally to this work.)

  • Xing Liu

    (College of Electronics and Information Engineering, Southwest University, Chongqing 400715, China
    These authors contributed equally to this work.
    Current address: Shaanxi Branch, Industrial and Commercial Bank of China (ICBC), Xi’an 710004, China.)

  • Yawei Shi

    (College of Electronics and Information Engineering, Southwest University, Chongqing 400715, China)

  • Huiwei Wang

    (Key Laboratory of Intelligent Information Processing, Chongqing Three Gorges University, Chongqing 404100, China
    Chongqing Innovation Center, Beijing Institute of Technology, Chongqing 401120, China)

Abstract

This paper proposes a new peer-to-peer (P2P) energy trading method between energy sellers and consumers in a community based on multi-agent reinforcement learning (MARL). Each user of the community is treated as a smart agent who can choose the amount and the price of the electric energy to sell/buy. There are two aspects we need to examine: the profits for the individual user and the utility for the community. For a single user, we consider that they want to realise both a comfortable living environment to enhance happiness and satisfaction by adjusting usage loads and certain economic benefits by selling the surplus electric energy. Taking the whole community into account, we care about the balance between energy sellers and consumers so that the surplus electric energy can be locally absorbed and consumed within the community. To this end, MARL is applied to solve the problem, where the decision making of each user in the community not only focuses on their own interests but also takes into account the entire community’s welfare. The experimental results prove that our method is profitable both both the sellers and buyers in the community.

Suggested Citation

  • Liangyi Pu & Song Wang & Xiaodong Huang & Xing Liu & Yawei Shi & Huiwei Wang, 2022. "Peer-to-Peer Trading for Energy-Saving Based on Reinforcement Learning," Energies, MDPI, vol. 15(24), pages 1-16, December.
    • Handle: RePEc:gam:jeners:v:15:y:2022:i:24:p:9633-:d:1008005
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/24/9633/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/15/24/9633/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Esmat, Ayman & de Vos, Martijn & Ghiassi-Farrokhfal, Yashar & Palensky, Peter & Epema, Dick, 2021. "A novel decentralized platform for peer-to-peer energy trading market with blockchain technology," Applied Energy, Elsevier, vol. 282(PA).
    2. Lu, Renzhi & Hong, Seung Ho & Zhang, Xiongfeng, 2018. "A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach," Applied Energy, Elsevier, vol. 220(C), pages 220-230.
    3. Jin-Gyeom Kim & Bowon Lee, 2020. "Automatic P2P Energy Trading Model Based on Reinforcement Learning Using Long Short-Term Delayed Reward," Energies, MDPI, vol. 13(20), pages 1-27, October.
    4. Soto, Esteban A. & Bosman, Lisa B. & Wollega, Ebisa & Leon-Salas, Walter D., 2021. "Peer-to-peer energy trading: A review of the literature," Applied Energy, Elsevier, vol. 283(C).
    5. Qiu, Dawei & Ye, Yujian & Papadaskalopoulos, Dimitrios & Strbac, Goran, 2021. "Scalable coordinated management of peer-to-peer energy trading: A multi-cluster deep reinforcement learning approach," Applied Energy, Elsevier, vol. 292(C).
    6. Daniel S. Bernstein & Robert Givan & Neil Immerman & Shlomo Zilberstein, 2002. "The Complexity of Decentralized Control of Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 27(4), pages 819-840, November.
    7. Hannie Zang & JongWon Kim, 2021. "Reinforcement Learning Based Peer-to-Peer Energy Trade Management Using Community Energy Storage in Local Energy Market," Energies, MDPI, vol. 14(14), pages 1-18, July.
    8. Ping-Huan Kuo & Chiou-Jye Huang, 2018. "An Electricity Price Forecasting Model by Hybrid Structured Deep Neural Networks," Sustainability, MDPI, vol. 10(4), pages 1-17, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Meritxell Domènech Monfort & César De Jesús & Natapon Wanapinit & Niklas Hartmann, 2022. "A Review of Peer-to-Peer Energy Trading with Standard Terminology Proposal and a Techno-Economic Characterisation Matrix," Energies, MDPI, vol. 15(23), pages 1-29, November.
    2. Zhou, Yuekuan & Lund, Peter D., 2023. "Peer-to-peer energy sharing and trading of renewable energy in smart communities ─ trading pricing models, decision-making and agent-based collaboration," Renewable Energy, Elsevier, vol. 207(C), pages 177-193.
    3. Azim, M. Imran & Tushar, Wayes & Saha, Tapan K. & Yuen, Chau & Smith, David, 2022. "Peer-to-peer kilowatt and negawatt trading: A review of challenges and recent advances in distribution networks," Renewable and Sustainable Energy Reviews, Elsevier, vol. 169(C).
    4. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    5. Samende, Cephas & Cao, Jun & Fan, Zhong, 2022. "Multi-agent deep deterministic policy gradient algorithm for peer-to-peer energy trading considering distribution network constraints," Applied Energy, Elsevier, vol. 317(C).
    6. Filipe Bandeiras & Álvaro Gomes & Mário Gomes & Paulo Coelho, 2023. "Exploring Energy Trading Markets in Smart Grid and Microgrid Systems and Their Implications for Sustainability in Smart Cities," Energies, MDPI, vol. 16(2), pages 1-41, January.
    7. Javier Parra-Domínguez & Esteban Sánchez & Ángel Ordóñez, 2023. "The Prosumer: A Systematic Review of the New Paradigm in Energy and Sustainable Development," Sustainability, MDPI, vol. 15(13), pages 1-44, July.
    8. Arnob Das & Susmita Datta Peu & Md. Abdul Mannan Akanda & Abu Reza Md. Towfiqul Islam, 2023. "Peer-to-Peer Energy Trading Pricing Mechanisms: Towards a Comprehensive Analysis of Energy and Network Service Pricing (NSP) Mechanisms to Get Sustainable Enviro-Economical Energy Sector," Energies, MDPI, vol. 16(5), pages 1-27, February.
    9. Tsaousoglou, Georgios & Ellinas, Petros & Varvarigos, Emmanouel, 2023. "Operating peer-to-peer electricity markets under uncertainty via learning-based, distributed optimal control," Applied Energy, Elsevier, vol. 343(C).
    10. Seyed Amin Sedgh & Hossein Aghamohammadloo & Hassan Khazaei & Mehdi Mehdinejad & Somayeh Asadi, 2023. "A New Design for the Peer-to-Peer Electricity and Gas Markets Based on Robust Probabilistic Programming," Energies, MDPI, vol. 16(8), pages 1-19, April.
    11. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    12. Shengcheng Fu & Yaxin Tan & Zhiyu Xu, 2023. "Blockchain-Based Renewable Energy Certificate Trade for Low-Carbon Community of Active Energy Agents," Sustainability, MDPI, vol. 15(23), pages 1-22, November.
    13. Chathurangi Edussuriya & Umar Marikkar & Subash Wickramasinghe & Upul Jayasinghe & Janaka Alawatugoda, 2023. "Peer-to-Peer Energy Trading through Swarm Intelligent Stackelberg Game," Energies, MDPI, vol. 16(5), pages 1-17, March.
    14. Shama Naz Islam, 2024. "A Review of Peer-to-Peer Energy Trading Markets: Enabling Models and Technologies," Energies, MDPI, vol. 17(7), pages 1-18, April.
    15. Lei, Yu-Tian & Ma, Chao-Qun & Mirza, Nawazish & Ren, Yi-Shuai & Narayan, Seema Wati & Chen, Xun-Qi, 2022. "A renewable energy microgrids trading management platform based on permissioned blockchain," Energy Economics, Elsevier, vol. 115(C).
    16. Umer, Khalid & Huang, Qi & Khorasany, Mohsen & Afzal, Muhammad & Amin, Waqas, 2021. "A novel communication efficient peer-to-peer energy trading scheme for enhanced privacy in microgrids," Applied Energy, Elsevier, vol. 296(C).
    17. Nils Korber & Maximilian Rohrig & Andreas Ulbig, 2022. "A stakeholder-oriented multi-criteria optimization model for decentral multi-energy systems," Papers 2204.06545, arXiv.org.
    18. Jing, Rui & Hua, Weiqi & Lin, Jian & Lin, Jianyi & Zhao, Yingru & Zhou, Yue & Wu, Jianzhong, 2022. "Cost-efficient decarbonization of local energy systems by whole-system based design optimization," Applied Energy, Elsevier, vol. 326(C).
    19. Wang, Yi & Qiu, Dawei & Strbac, Goran, 2022. "Multi-agent deep reinforcement learning for resilience-driven routing and scheduling of mobile energy storage systems," Applied Energy, Elsevier, vol. 310(C).
    20. Barja-Martinez, Sara & Aragüés-Peñalba, Mònica & Munné-Collado, Íngrid & Lloret-Gallego, Pau & Bullich-Massagué, Eduard & Villafafila-Robles, Roberto, 2021. "Artificial intelligence techniques for enabling Big Data services in distribution networks: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 150(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:24:p:9633-:d:1008005. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.