IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v16y2023i5p2357-d1084595.html
   My bibliography  Save this article

Real-Time Multi-Home Energy Management with EV Charging Scheduling Using Multi-Agent Deep Reinforcement Learning Optimization

Author

Listed:
  • Niphon Kaewdornhan

    (Department of Electrical Engineering, Khon Kaen University, Khon Kaen 40002, Thailand)

  • Chitchai Srithapon

    (Department of Electrical Engineering, KTH Royal Institute of Technology, SE-100 44 Stockholm, Sweden)

  • Rittichai Liemthong

    (Business Development Engineer, Sermsang Power Corporation Public Company Limited, Bangkok 10300, Thailand)

  • Rongrit Chatthaworn

    (Department of Electrical Engineering, Khon Kaen University, Khon Kaen 40002, Thailand
    Center for Alternative Energy Research and Development, Khon Kaen University, Khon Kaen 40002, Thailand)

Abstract

Energy management for multi-home installation of solar PhotoVoltaics (solar PVs) combined with Electric Vehicles’ (EVs) charging scheduling has a rich complexity due to the uncertainties of solar PV generation and EV usage. Changing clients from multi-consumers to multi-prosumers with real-time energy trading supervised by the aggregator is an efficient way to solve undesired demand problems due to disorderly EV scheduling. Therefore, this paper proposes real-time multi-home energy management with EV charging scheduling using multi-agent deep reinforcement learning optimization. The aggregator and prosumers are developed as smart agents to interact with each other to find the best decision. This paper aims to reduce the electricity expense of prosumers through EV battery scheduling. The aggregator calculates the revenue from energy trading with multi-prosumers by using a real-time pricing concept which can facilitate the proper behavior of prosumers. Simulation results show that the proposed method can reduce mean power consumption by 9.04% and 39.57% compared with consumption using the system without EV usage and the system that applies the conventional energy price, respectively. Also, it can decrease the costs of the prosumer by between 1.67% and 24.57%, and the aggregator can generate revenue by 0.065 USD per day, which is higher than that generated when employing conventional energy prices.

Suggested Citation

  • Niphon Kaewdornhan & Chitchai Srithapon & Rittichai Liemthong & Rongrit Chatthaworn, 2023. "Real-Time Multi-Home Energy Management with EV Charging Scheduling Using Multi-Agent Deep Reinforcement Learning Optimization," Energies, MDPI, vol. 16(5), pages 1-25, March.
    • Handle: RePEc:gam:jeners:v:16:y:2023:i:5:p:2357-:d:1084595
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/16/5/2357/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/16/5/2357/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Sheraz Aslam & Zafar Iqbal & Nadeem Javaid & Zahoor Ali Khan & Khursheed Aurangzeb & Syed Irtaza Haider, 2017. "Towards Efficient Energy Management of Smart Buildings Exploiting Heuristic Optimization with Real Time and Critical Peak Pricing Schemes," Energies, MDPI, vol. 10(12), pages 1-25, December.
    2. Gonçalves, Ivo & Gomes, Álvaro & Henggeler Antunes, Carlos, 2019. "Optimizing the management of smart home energy resources under different power cost scenarios," Applied Energy, Elsevier, vol. 242(C), pages 351-363.
    3. Park, Keonwoo & Moon, Ilkyeong, 2022. "Multi-agent deep reinforcement learning approach for EV charging scheduling in a smart grid," Applied Energy, Elsevier, vol. 328(C).
    4. Chitchai Srithapon & Prasanta Ghosh & Apirat Siritaratiwat & Rongrit Chatthaworn, 2020. "Optimization of Electric Vehicle Charging Scheduling in Urban Village Networks Considering Energy Arbitrage and Distribution Cost," Energies, MDPI, vol. 13(2), pages 1-20, January.
    5. Sanchari Deb & Kari Tammi & Karuna Kalita & Pinakeshwar Mahanta, 2018. "Impact of Electric Vehicle Charging Station Load on Distribution Network," Energies, MDPI, vol. 11(1), pages 1-25, January.
    6. Ying Ji & Jianhui Wang & Jiacan Xu & Xiaoke Fang & Huaguang Zhang, 2019. "Real-Time Energy Management of a Microgrid Using Deep Reinforcement Learning," Energies, MDPI, vol. 12(12), pages 1-21, June.
    7. Amrutha Raju Battula & Sandeep Vuddanti & Surender Reddy Salkuti, 2021. "Review of Energy Management System Approaches in Microgrids," Energies, MDPI, vol. 14(17), pages 1-32, September.
    8. Killian, M. & Zauner, M. & Kozek, M., 2018. "Comprehensive smart home energy management system using mixed-integer quadratic-programming," Applied Energy, Elsevier, vol. 222(C), pages 662-672.
    9. Mak, Davye & Choi, Dae-Hyun, 2020. "Optimization framework for coordinated operation of home energy management system and Volt-VAR optimization in unbalanced active distribution networks considering uncertainties," Applied Energy, Elsevier, vol. 276(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jeddi, Babak & Mishra, Yateendra & Ledwich, Gerard, 2021. "Distributed load scheduling in residential neighborhoods for coordinated operation of multiple home energy management systems," Applied Energy, Elsevier, vol. 300(C).
    2. Mak, Davye & Choi, Dae-Hyun, 2020. "Optimization framework for coordinated operation of home energy management system and Volt-VAR optimization in unbalanced active distribution networks considering uncertainties," Applied Energy, Elsevier, vol. 276(C).
    3. Ebrahimi, Seyyed Reza & Rahimiyan, Morteza & Assili, Mohsen & Hajizadeh, Amin, 2022. "Home energy management under correlated uncertainties: A statistical analysis through Copula," Applied Energy, Elsevier, vol. 305(C).
    4. Lilia Tightiz & Joon Yoo, 2022. "A Review on a Data-Driven Microgrid Management System Integrating an Active Distribution Network: Challenges, Issues, and New Trends," Energies, MDPI, vol. 15(22), pages 1-24, November.
    5. Rocha, Helder R.O. & Honorato, Icaro H. & Fiorotti, Rodrigo & Celeste, Wanderley C. & Silvestre, Leonardo J. & Silva, Jair A.L., 2021. "An Artificial Intelligence based scheduling algorithm for demand-side energy management in Smart Homes," Applied Energy, Elsevier, vol. 282(PA).
    6. Lorenzo Bartolucci & Stefano Cordiner & Vincenzo Mulone & Marina Santarelli, 2019. "Ancillary Services Provided by Hybrid Residential Renewable Energy Systems through Thermal and Electrochemical Storage Systems," Energies, MDPI, vol. 12(12), pages 1-18, June.
    7. Ana Cabrera-Tobar & Alessandro Massi Pavan & Giovanni Petrone & Giovanni Spagnuolo, 2022. "A Review of the Optimization and Control Techniques in the Presence of Uncertainties for the Energy Management of Microgrids," Energies, MDPI, vol. 15(23), pages 1-38, December.
    8. Rittichai Liemthong & Chitchai Srithapon & Prasanta K. Ghosh & Rongrit Chatthaworn, 2022. "Home Energy Management Strategy-Based Meta-Heuristic Optimization for Electrical Energy Cost Minimization Considering TOU Tariffs," Energies, MDPI, vol. 15(2), pages 1-22, January.
    9. Alqahtani, Mohammed & Hu, Mengqi, 2022. "Dynamic energy scheduling and routing of multiple electric vehicles using deep reinforcement learning," Energy, Elsevier, vol. 244(PA).
    10. Yuhong Wang & Lei Chen & Hong Zhou & Xu Zhou & Zongsheng Zheng & Qi Zeng & Li Jiang & Liang Lu, 2021. "Flexible Transmission Network Expansion Planning Based on DQN Algorithm," Energies, MDPI, vol. 14(7), pages 1-21, April.
    11. Sulman Shahzad & Muhammad Abbas Abbasi & Hassan Ali & Muhammad Iqbal & Rania Munir & Heybet Kilic, 2023. "Possibilities, Challenges, and Future Opportunities of Microgrids: A Review," Sustainability, MDPI, vol. 15(8), pages 1-28, April.
    12. Yang, Ting & Zhao, Liyuan & Li, Wei & Zomaya, Albert Y., 2021. "Dynamic energy dispatch strategy for integrated energy system based on improved deep reinforcement learning," Energy, Elsevier, vol. 235(C).
    13. Wang, Yi & Qiu, Dawei & Sun, Mingyang & Strbac, Goran & Gao, Zhiwei, 2023. "Secure energy management of multi-energy microgrid: A physical-informed safe reinforcement learning approach," Applied Energy, Elsevier, vol. 335(C).
    14. Jia, Kunqi & Guo, Ge & Xiao, Jucheng & Zhou, Huan & Wang, Zhihua & He, Guangyu, 2019. "Data compression approach for the home energy management system," Applied Energy, Elsevier, vol. 247(C), pages 643-656.
    15. Joanna Kott & Marek Kott, 2019. "Generic Ontology of Energy Consumption Households," Energies, MDPI, vol. 12(19), pages 1-19, September.
    16. Kabir, Farzana & Yu, Nanpeng & Gao, Yuanqi & Wang, Wenyu, 2023. "Deep reinforcement learning-based two-timescale Volt-VAR control with degradation-aware smart inverters in power distribution systems," Applied Energy, Elsevier, vol. 335(C).
    17. Hang Yi & Wenjun Peng & Xiuchun Xiao & Shaojin Feng & Hengde Zhu & Yudong Zhang, 2023. "An Adaptive Zeroing Neural Network with Non-Convex Activation for Time-Varying Quadratic Minimization," Mathematics, MDPI, vol. 11(11), pages 1-15, June.
    18. Paulo M. De Oliveira-De Jesus & Mario A. Rios & Gustavo A. Ramos, 2018. "Energy Loss Allocation in Smart Distribution Systems with Electric Vehicle Integration," Energies, MDPI, vol. 11(8), pages 1-19, July.
    19. Guilherme Henrique Alves & Geraldo Caixeta Guimarães & Fabricio Augusto Matheus Moura, 2023. "Battery Storage Systems Control Strategies with Intelligent Algorithms in Microgrids with Dynamic Pricing," Energies, MDPI, vol. 16(14), pages 1-30, July.
    20. Imed Khabbouchi & Dhaou Said & Aziz Oukaira & Idir Mellal & Lyes Khoukhi, 2023. "Machine Learning and Game-Theoretic Model for Advanced Wind Energy Management Protocol (AWEMP)," Energies, MDPI, vol. 16(5), pages 1-15, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:16:y:2023:i:5:p:2357-:d:1084595. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.