IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v14y2021i17p5515-d628715.html
   My bibliography  Save this article

Novel Energy Trading System Based on Deep-Reinforcement Learning in Microgrids

Author

Listed:
  • Seongwoo Lee

    (Department of Electronic Convergence Engineering, University of Kwangwoon, Seoul 01897, Korea)

  • Joonho Seon

    (Department of Electronic Convergence Engineering, University of Kwangwoon, Seoul 01897, Korea)

  • Chanuk Kyeong

    (Department of Electronic Convergence Engineering, University of Kwangwoon, Seoul 01897, Korea)

  • Soohyun Kim

    (Department of Electronic Convergence Engineering, University of Kwangwoon, Seoul 01897, Korea)

  • Youngghyu Sun

    (Department of Electronic Convergence Engineering, University of Kwangwoon, Seoul 01897, Korea)

  • Jinyoung Kim

    (Department of Electronic Convergence Engineering, University of Kwangwoon, Seoul 01897, Korea)

Abstract

Inefficiencies in energy trading systems of microgrids are mainly caused by uncertainty in non-stationary operating environments. The problem of uncertainty can be mitigated by analyzing patterns of primary operation parameters and their corresponding actions. In this paper, a novel energy trading system based on a double deep Q-networks (DDQN) algorithm and a double Kelly strategy is proposed for improving profits while reducing dependence on the main grid in the microgrid systems. The DDQN algorithm is proposed in order to select optimized action for improving energy transactions. Additionally, the double Kelly strategy is employed to control the microgrid’s energy trading quantity for producing long-term profits. From the simulation results, it is confirmed that the proposed strategies can achieve a significant improvement in the total profits and independence from the main grid via optimized energy transactions.

Suggested Citation

  • Seongwoo Lee & Joonho Seon & Chanuk Kyeong & Soohyun Kim & Youngghyu Sun & Jinyoung Kim, 2021. "Novel Energy Trading System Based on Deep-Reinforcement Learning in Microgrids," Energies, MDPI, vol. 14(17), pages 1-14, September.
    • Handle: RePEc:gam:jeners:v:14:y:2021:i:17:p:5515-:d:628715
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/14/17/5515/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/14/17/5515/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Hirsch, Adam & Parag, Yael & Guerrero, Josep, 2018. "Microgrids: A review of technologies, key drivers, and outstanding issues," Renewable and Sustainable Energy Reviews, Elsevier, vol. 90(C), pages 402-411.
    2. Shahin Bayramov & Iurii Prokazov & Sergey Kondrashev & Jan Kowalik, 2021. "Household Electricity Generation as a Way of Energy Independence of States—Social Context of Energy Management," Energies, MDPI, vol. 14(12), pages 1-19, June.
    3. Duarte Kazacos Winter & Rahul Khatri & Michael Schmidt, 2021. "Decentralized Prosumer-Centric P2P Electricity Market Coordination with Grid Security," Energies, MDPI, vol. 14(15), pages 1-17, August.
    4. Yingshu Liu & Yue Fang & Jun Li, 2017. "Interconnecting Microgrids via the Energy Router with Smart Energy Management," Energies, MDPI, vol. 10(9), pages 1-19, August.
    5. Boram Kim & Sunghwan Bae & Hongseok Kim, 2017. "Optimal Energy Scheduling and Transaction Mechanism for Multiple Microgrids," Energies, MDPI, vol. 10(4), pages 1-17, April.
    6. Ying Ji & Jianhui Wang & Jiacan Xu & Xiaoke Fang & Huaguang Zhang, 2019. "Real-Time Energy Management of a Microgrid Using Deep Reinforcement Learning," Energies, MDPI, vol. 12(12), pages 1-21, June.
    7. Milad Afzalan & Farrokh Jazizadeh, 2021. "Quantification of Demand-Supply Balancing Capacity among Prosumers and Consumers: Community Self-Sufficiency Assessment for Energy Trading," Energies, MDPI, vol. 14(14), pages 1-21, July.
    8. Koo-Hyung Chung & Don Hur, 2020. "Towards the Design of P2P Energy Trading Scheme Based on Optimal Energy Scheduling for Prosumers," Energies, MDPI, vol. 13(19), pages 1-15, October.
    9. Abdullah M. Alabdullatif & Enrico H. Gerding & Alvaro Perez-Diaz, 2020. "Market Design and Trading Strategies for Community Energy Markets with Storage and Renewable Supply," Energies, MDPI, vol. 13(4), pages 1-31, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    2. Bio Gassi, Karim & Baysal, Mustafa, 2023. "Improving real-time energy decision-making model with an actor-critic agent in modern microgrids with energy storage devices," Energy, Elsevier, vol. 263(PE).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gerlach, Lisa & Bocklisch, Thilo & Verweij, Marco, 2023. "Selfish batteries vs. benevolent optimizers," Renewable and Sustainable Energy Reviews, Elsevier, vol. 177(C).
    2. Arnob Das & Susmita Datta Peu & Md. Abdul Mannan Akanda & Abu Reza Md. Towfiqul Islam, 2023. "Peer-to-Peer Energy Trading Pricing Mechanisms: Towards a Comprehensive Analysis of Energy and Network Service Pricing (NSP) Mechanisms to Get Sustainable Enviro-Economical Energy Sector," Energies, MDPI, vol. 16(5), pages 1-27, February.
    3. Amrutha Raju Battula & Sandeep Vuddanti & Surender Reddy Salkuti, 2021. "Review of Energy Management System Approaches in Microgrids," Energies, MDPI, vol. 14(17), pages 1-32, September.
    4. Azim, M. Imran & Tushar, Wayes & Saha, Tapan K. & Yuen, Chau & Smith, David, 2022. "Peer-to-peer kilowatt and negawatt trading: A review of challenges and recent advances in distribution networks," Renewable and Sustainable Energy Reviews, Elsevier, vol. 169(C).
    5. Tri-Hai Nguyen & Luong Vuong Nguyen & Jason J. Jung & Israel Edem Agbehadji & Samuel Ofori Frimpong & Richard C. Millham, 2020. "Bio-Inspired Approaches for Smart Energy Management: State of the Art and Challenges," Sustainability, MDPI, vol. 12(20), pages 1-24, October.
    6. Yingpei Liu & Yan Li & Haiping Liang & Jia He & Hanyang Cui, 2019. "Energy Routing Control Strategy for Integrated Microgrids Including Photovoltaic, Battery-Energy Storage and Electric Vehicles," Energies, MDPI, vol. 12(2), pages 1-16, January.
    7. Àlex Alonso-Travesset & Helena Martín & Sergio Coronas & Jordi de la Hoz, 2022. "Optimization Models under Uncertainty in Distributed Generation Systems: A Review," Energies, MDPI, vol. 15(5), pages 1-40, March.
    8. Farhat Afzah Samoon & Ikhlaq Hussain & Sheikh Javed Iqbal, 2023. "ILA Optimisation Based Control for Enhancing DC Link Voltage with Seamless and Adaptive VSC Control in a PV-BES Based AC Microgrid," Energies, MDPI, vol. 16(21), pages 1-23, October.
    9. Emrani-Rahaghi, Pouria & Hashemi-Dezaki, Hamed & Ketabi, Abbas, 2023. "Efficient voltage control of low voltage distribution networks using integrated optimized energy management of networked residential multi-energy microgrids," Applied Energy, Elsevier, vol. 349(C).
    10. Dimitrios Trigkas & Chrysovalantou Ziogou & Spyros Voutetakis & Simira Papadopoulou, 2021. "Virtual Energy Storage in RES-Powered Smart Grids with Nonlinear Model Predictive Control," Energies, MDPI, vol. 14(4), pages 1-22, February.
    11. Jihed Hmad & Azeddine Houari & Allal El Moubarek Bouzid & Abdelhakim Saim & Hafedh Trabelsi, 2023. "A Review on Mode Transition Strategies between Grid-Connected and Standalone Operation of Voltage Source Inverters-Based Microgrids," Energies, MDPI, vol. 16(13), pages 1-41, June.
    12. Alqahtani, Mohammed & Hu, Mengqi, 2022. "Dynamic energy scheduling and routing of multiple electric vehicles using deep reinforcement learning," Energy, Elsevier, vol. 244(PA).
    13. Matija Kostelac & Lin Herenčić & Tomislav Capuder, 2022. "Planning and Operational Aspects of Individual and Clustered Multi-Energy Microgrid Options," Energies, MDPI, vol. 15(4), pages 1-17, February.
    14. Terlouw, Tom & AlSkaif, Tarek & Bauer, Christian & van Sark, Wilfried, 2019. "Optimal energy management in all-electric residential energy systems with heat and electricity storage," Applied Energy, Elsevier, vol. 254(C).
    15. Soheil Mohseni & Alan C. Brent & Daniel Burmester, 2020. "Community Resilience-Oriented Optimal Micro-Grid Capacity Expansion Planning: The Case of Totarabank Eco-Village, New Zealand," Energies, MDPI, vol. 13(15), pages 1-29, August.
    16. Yuhong Wang & Lei Chen & Hong Zhou & Xu Zhou & Zongsheng Zheng & Qi Zeng & Li Jiang & Liang Lu, 2021. "Flexible Transmission Network Expansion Planning Based on DQN Algorithm," Energies, MDPI, vol. 14(7), pages 1-21, April.
    17. Yang, Ting & Zhao, Liyuan & Li, Wei & Zomaya, Albert Y., 2021. "Dynamic energy dispatch strategy for integrated energy system based on improved deep reinforcement learning," Energy, Elsevier, vol. 235(C).
    18. Wang, Yi & Qiu, Dawei & Sun, Mingyang & Strbac, Goran & Gao, Zhiwei, 2023. "Secure energy management of multi-energy microgrid: A physical-informed safe reinforcement learning approach," Applied Energy, Elsevier, vol. 335(C).
    19. Roth, Jonathan & Martin, Amory & Miller, Clayton & Jain, Rishee K., 2020. "SynCity: Using open data to create a synthetic city of hourly building energy estimates by integrating data-driven and physics-based methods," Applied Energy, Elsevier, vol. 280(C).
    20. Hammad Alnuman & Kuo-Hsien Hsia & Mohammadreza Askari Sepestanaki & Emad M. Ahmed & Saleh Mobayen & Ammar Armghan, 2023. "Design of Continuous Finite-Time Controller Based on Adaptive Tuning Approach for Disturbed Boost Converters," Mathematics, MDPI, vol. 11(7), pages 1-23, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:14:y:2021:i:17:p:5515-:d:628715. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.