IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v14y2021i14p4131-d591038.html
   My bibliography  Save this article

Reinforcement Learning Based Peer-to-Peer Energy Trade Management Using Community Energy Storage in Local Energy Market

Author

Listed:
  • Hannie Zang

    (School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology (GIST), 123 Cheomdangwagi-ro, Buk-gu, Gwangju 61005, Korea)

  • JongWon Kim

    (AI Graduate School, Gwangju Institute of Science and Technology (GIST), Gwangju 61005, Korea)

Abstract

Many studies have proposed a peer-to-peer energy market where the prosumers’ actions, including energy consumption, charge and discharge schedule of energy storage systems, and transactions in local energy markets, are controlled by a central operator. In this paper, prosumers’ actions are not controlled by an operator, and the prosumers freely participate in the local energy market to trade energy with other prosumers. We designed and modeled a local energy market with a management algorithm that uses community energy storage for prosumers who competitively participate in trade in the real-time energy market. We propose an energy-trade management algorithm that manages the trades of prosumers in two phases based on bids and offers submitted by prosumers. The first phase is to manage the trade of prosumers who have submitted fair prices to trade with other prosumers in the real-time energy market. The second phase is managing the trade of prosumers that could not trade in the first phase. Community energy storage is employed in the second phase and controlled by a reinforcement learning-based trading algorithm to decide whether to buy, sell, or do nothing with the prosumers. The action of buying and selling means charging and discharging the community energy storage, respectively. Numerical results show that the proposed trading algorithm gains a near-maximum profit. Besides, we verified that community energy storage yields more profit than the battery wear-out cost.

Suggested Citation

  • Hannie Zang & JongWon Kim, 2021. "Reinforcement Learning Based Peer-to-Peer Energy Trade Management Using Community Energy Storage in Local Energy Market," Energies, MDPI, vol. 14(14), pages 1-18, July.
    • Handle: RePEc:gam:jeners:v:14:y:2021:i:14:p:4131-:d:591038
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/14/14/4131/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/14/14/4131/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Umer, Khalid & Huang, Qi & Khorasany, Mohsen & Afzal, Muhammad & Amin, Waqas, 2021. "A novel communication efficient peer-to-peer energy trading scheme for enhanced privacy in microgrids," Applied Energy, Elsevier, vol. 296(C).
    2. Jin-Gyeom Kim & Bowon Lee, 2020. "Automatic P2P Energy Trading Model Based on Reinforcement Learning Using Long Short-Term Delayed Reward," Energies, MDPI, vol. 13(20), pages 1-27, October.
    3. Bucciarelli, Martina & Paoletti, Simone & Vicino, Antonio, 2018. "Optimal sizing of energy storage systems under uncertain demand and generation," Applied Energy, Elsevier, vol. 225(C), pages 611-621.
    4. Koirala, Binod Prasad & van Oost, Ellen & van der Windt, Henny, 2018. "Community energy storage: A responsible innovation towards a sustainable energy system?," Applied Energy, Elsevier, vol. 231(C), pages 570-585.
    5. Yael Parag & Benjamin K. Sovacool, 2016. "Electricity market design for the prosumer era," Nature Energy, Nature, vol. 1(4), pages 1-6, April.
    6. Anvari-Moghaddam, Amjad & Rahimi-Kian, Ashkan & Mirian, Maryam S. & Guerrero, Josep M., 2017. "A multi-agent based energy management solution for integrated buildings and microgrid system," Applied Energy, Elsevier, vol. 203(C), pages 41-56.
    7. Han, Sekyung & Han, Soohee & Aki, Hirohisa, 2014. "A practical battery wear model for electric vehicle charging applications," Applied Energy, Elsevier, vol. 113(C), pages 1100-1108.
    8. Björn Nykvist & Måns Nilsson, 2015. "Rapidly falling costs of battery packs for electric vehicles," Nature Climate Change, Nature, vol. 5(4), pages 329-332, April.
    9. Akter, M.N. & Mahmud, M.A. & Haque, M.E. & Oo, Amanullah M.T., 2020. "An optimal distributed energy management scheme for solving transactive energy sharing problems in residential microgrids," Applied Energy, Elsevier, vol. 270(C).
    10. Roberts, Mike B. & Bruce, Anna & MacGill, Iain, 2019. "Impact of shared battery energy storage systems on photovoltaic self-consumption and electricity bills in apartment buildings," Applied Energy, Elsevier, vol. 245(C), pages 78-95.
    11. Nikolaos Koltsaklis & Ioannis P. Panapakidis & David Pozo & Georgios C. Christoforidis, 2021. "A Prosumer Model Based on Smart Home Energy Management and Forecasting Techniques," Energies, MDPI, vol. 14(6), pages 1-32, March.
    12. Jing, Rui & Xie, Mei Na & Wang, Feng Xiang & Chen, Long Xiang, 2020. "Fair P2P energy trading between residential and commercial multi-energy systems enabling integrated demand-side management," Applied Energy, Elsevier, vol. 262(C).
    13. Lüth, Alexandra & Zepter, Jan Martin & Crespo del Granado, Pedro & Egging, Ruud, 2018. "Local electricity market designs for peer-to-peer trading: The role of battery flexibility," Applied Energy, Elsevier, vol. 229(C), pages 1233-1243.
    14. Zhang, Chenghua & Wu, Jianzhong & Zhou, Yue & Cheng, Meng & Long, Chao, 2018. "Peer-to-Peer energy trading in a Microgrid," Applied Energy, Elsevier, vol. 220(C), pages 1-12.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shida Zhang & Petr Musilek, 2023. "The Impact of Battery Storage on Power Flow and Economy in an Automated Transactive Energy Market," Energies, MDPI, vol. 16(5), pages 1-13, February.
    2. Yuzhe Xie & Yan Yao & Yawu Wang & Weiqiang Cha & Sheng Zhou & Yue Wu & Chunyi Huang, 2022. "A Cooperative Game-Based Sizing and Configuration of Community-Shared Energy Storage," Energies, MDPI, vol. 15(22), pages 1-17, November.
    3. Mohammad Alipour & Rodney A. Stewart & Oz Sahin, 2021. "Beyond the Diffusion of Residential Solar Photovoltaic Systems at Scale: Allegorising the Battery Energy Storage Adoption Behaviour," Energies, MDPI, vol. 14(16), pages 1-12, August.
    4. Liangyi Pu & Song Wang & Xiaodong Huang & Xing Liu & Yawei Shi & Huiwei Wang, 2022. "Peer-to-Peer Trading for Energy-Saving Based on Reinforcement Learning," Energies, MDPI, vol. 15(24), pages 1-16, December.
    5. Peter Klement & Tobias Brandt & Lucas Schmeling & Antonieta Alcorta de Bronstein & Steffen Wehkamp & Fernando Andres Penaherrera Vaca & Mathias Lanezki & Patrik Schönfeldt & Alexander Hill & Nemanja K, 2022. "Local Energy Markets in Action: Smart Integration of National Markets, Distributed Energy Resources and Incentivisation to Promote Citizen Participation," Energies, MDPI, vol. 15(8), pages 1-24, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Andreolli, Francesca & D'Alpaos, Chiara & Kort, Peter, 2023. "Does P2P Trading Favor Investments in PV-Battery Systems?," FEEM Working Papers 330498, Fondazione Eni Enrico Mattei (FEEM).
    2. Francesca Andreolli & Chiara D'Alpaos & Peter Kort, 2023. "Does P2P Trading Favor Investments in PV-Battery Systems?," Working Papers 2023.02, Fondazione Eni Enrico Mattei.
    3. Meritxell Domènech Monfort & César De Jesús & Natapon Wanapinit & Niklas Hartmann, 2022. "A Review of Peer-to-Peer Energy Trading with Standard Terminology Proposal and a Techno-Economic Characterisation Matrix," Energies, MDPI, vol. 15(23), pages 1-29, November.
    4. Tushar, Wayes & Yuen, Chau & Saha, Tapan K. & Morstyn, Thomas & Chapman, Archie C. & Alam, M. Jan E. & Hanif, Sarmad & Poor, H. Vincent, 2021. "Peer-to-peer energy systems for connected communities: A review of recent advances and emerging challenges," Applied Energy, Elsevier, vol. 282(PA).
    5. Ableitner, Liliane & Tiefenbeck, Verena & Meeuw, Arne & Wörner, Anselma & Fleisch, Elgar & Wortmann, Felix, 2020. "User behavior in a real-world peer-to-peer electricity market," Applied Energy, Elsevier, vol. 270(C).
    6. Azim, M. Imran & Tushar, Wayes & Saha, Tapan K. & Yuen, Chau & Smith, David, 2022. "Peer-to-peer kilowatt and negawatt trading: A review of challenges and recent advances in distribution networks," Renewable and Sustainable Energy Reviews, Elsevier, vol. 169(C).
    7. Capper, Timothy & Gorbatcheva, Anna & Mustafa, Mustafa A. & Bahloul, Mohamed & Schwidtal, Jan Marc & Chitchyan, Ruzanna & Andoni, Merlinda & Robu, Valentin & Montakhabi, Mehdi & Scott, Ian J. & Franci, 2022. "Peer-to-peer, community self-consumption, and transactive energy: A systematic literature review of local energy market models," Renewable and Sustainable Energy Reviews, Elsevier, vol. 162(C).
    8. Filipe Bandeiras & Álvaro Gomes & Mário Gomes & Paulo Coelho, 2023. "Exploring Energy Trading Markets in Smart Grid and Microgrid Systems and Their Implications for Sustainability in Smart Cities," Energies, MDPI, vol. 16(2), pages 1-41, January.
    9. Nizami, Sohrab & Tushar, Wayes & Hossain, M.J. & Yuen, Chau & Saha, Tapan & Poor, H. Vincent, 2022. "Transactive energy for low voltage residential networks: A review," Applied Energy, Elsevier, vol. 323(C).
    10. Karami, Mahdi & Madlener, Reinhard, 2022. "Business models for peer-to-peer energy trading in Germany based on households’ beliefs and preferences," Applied Energy, Elsevier, vol. 306(PB).
    11. Qiu, Dawei & Ye, Yujian & Papadaskalopoulos, Dimitrios & Strbac, Goran, 2021. "Scalable coordinated management of peer-to-peer energy trading: A multi-cluster deep reinforcement learning approach," Applied Energy, Elsevier, vol. 292(C).
    12. Lüth, Alexandra & Zepter, Jan Martin & Crespo del Granado, Pedro & Egging, Ruud, 2018. "Local electricity market designs for peer-to-peer trading: The role of battery flexibility," Applied Energy, Elsevier, vol. 229(C), pages 1233-1243.
    13. Javier Parra-Domínguez & Esteban Sánchez & Ángel Ordóñez, 2023. "The Prosumer: A Systematic Review of the New Paradigm in Energy and Sustainable Development," Sustainability, MDPI, vol. 15(13), pages 1-44, July.
    14. Schwidtal, J.M. & Piccini, P. & Troncia, M. & Chitchyan, R. & Montakhabi, M. & Francis, C. & Gorbatcheva, A. & Capper, T. & Mustafa, M.A. & Andoni, M. & Robu, V. & Bahloul, M. & Scott, I.J. & Mbavarir, 2023. "Emerging business models in local energy markets: A systematic review of peer-to-peer, community self-consumption, and transactive energy models," Renewable and Sustainable Energy Reviews, Elsevier, vol. 179(C).
    15. Rodrigues, Stefane Dias & Garcia, Vinicius Jacques, 2023. "Transactive energy in microgrid communities: A systematic review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 171(C).
    16. Chen, Kaixuan & Lin, Jin & Song, Yonghua, 2019. "Trading strategy optimization for a prosumer in continuous double auction-based peer-to-peer market: A prediction-integration model," Applied Energy, Elsevier, vol. 242(C), pages 1121-1133.
    17. Arnob Das & Susmita Datta Peu & Md. Abdul Mannan Akanda & Abu Reza Md. Towfiqul Islam, 2023. "Peer-to-Peer Energy Trading Pricing Mechanisms: Towards a Comprehensive Analysis of Energy and Network Service Pricing (NSP) Mechanisms to Get Sustainable Enviro-Economical Energy Sector," Energies, MDPI, vol. 16(5), pages 1-27, February.
    18. Gržanić, M. & Capuder, T. & Zhang, N. & Huang, W., 2022. "Prosumers as active market participants: A systematic review of evolution of opportunities, models and challenges," Renewable and Sustainable Energy Reviews, Elsevier, vol. 154(C).
    19. Kobashi, Takuro & Yoshida, Takahiro & Yamagata, Yoshiki & Naito, Katsuhiko & Pfenninger, Stefan & Say, Kelvin & Takeda, Yasuhiro & Ahl, Amanda & Yarime, Masaru & Hara, Keishiro, 2020. "On the potential of “Photovoltaics + Electric vehicles” for deep decarbonization of Kyoto’s power systems: Techno-economic-social considerations," Applied Energy, Elsevier, vol. 275(C).
    20. Alexandra Lüth & Jens Weibezahn & Jan Martin Zepter, 2020. "On Distributional Effects in Local Electricity Market Designs—Evidence from a German Case Study," Energies, MDPI, vol. 13(8), pages 1-26, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:14:y:2021:i:14:p:4131-:d:591038. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.