IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v16y2023i5p2357-d1084595.html

Real-Time Multi-Home Energy Management with EV Charging Scheduling Using Multi-Agent Deep Reinforcement Learning Optimization

Author

Listed:
  • Niphon Kaewdornhan

    (Department of Electrical Engineering, Khon Kaen University, Khon Kaen 40002, Thailand)

  • Chitchai Srithapon

    (Department of Electrical Engineering, KTH Royal Institute of Technology, SE-100 44 Stockholm, Sweden)

  • Rittichai Liemthong

    (Business Development Engineer, Sermsang Power Corporation Public Company Limited, Bangkok 10300, Thailand)

  • Rongrit Chatthaworn

    (Department of Electrical Engineering, Khon Kaen University, Khon Kaen 40002, Thailand
    Center for Alternative Energy Research and Development, Khon Kaen University, Khon Kaen 40002, Thailand)

Abstract

Energy management for multi-home installation of solar PhotoVoltaics (solar PVs) combined with Electric Vehicles’ (EVs) charging scheduling has a rich complexity due to the uncertainties of solar PV generation and EV usage. Changing clients from multi-consumers to multi-prosumers with real-time energy trading supervised by the aggregator is an efficient way to solve undesired demand problems due to disorderly EV scheduling. Therefore, this paper proposes real-time multi-home energy management with EV charging scheduling using multi-agent deep reinforcement learning optimization. The aggregator and prosumers are developed as smart agents to interact with each other to find the best decision. This paper aims to reduce the electricity expense of prosumers through EV battery scheduling. The aggregator calculates the revenue from energy trading with multi-prosumers by using a real-time pricing concept which can facilitate the proper behavior of prosumers. Simulation results show that the proposed method can reduce mean power consumption by 9.04% and 39.57% compared with consumption using the system without EV usage and the system that applies the conventional energy price, respectively. Also, it can decrease the costs of the prosumer by between 1.67% and 24.57%, and the aggregator can generate revenue by 0.065 USD per day, which is higher than that generated when employing conventional energy prices.

Suggested Citation

  • Niphon Kaewdornhan & Chitchai Srithapon & Rittichai Liemthong & Rongrit Chatthaworn, 2023. "Real-Time Multi-Home Energy Management with EV Charging Scheduling Using Multi-Agent Deep Reinforcement Learning Optimization," Energies, MDPI, vol. 16(5), pages 1-25, March.
    • Handle: RePEc:gam:jeners:v:16:y:2023:i:5:p:2357-:d:1084595
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/16/5/2357/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/16/5/2357/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Sheraz Aslam & Zafar Iqbal & Nadeem Javaid & Zahoor Ali Khan & Khursheed Aurangzeb & Syed Irtaza Haider, 2017. "Towards Efficient Energy Management of Smart Buildings Exploiting Heuristic Optimization with Real Time and Critical Peak Pricing Schemes," Energies, MDPI, vol. 10(12), pages 1-25, December.
    2. Sanchari Deb & Kari Tammi & Karuna Kalita & Pinakeshwar Mahanta, 2018. "Impact of Electric Vehicle Charging Station Load on Distribution Network," Energies, MDPI, vol. 11(1), pages 1-25, January.
    3. Gonçalves, Ivo & Gomes, Álvaro & Henggeler Antunes, Carlos, 2019. "Optimizing the management of smart home energy resources under different power cost scenarios," Applied Energy, Elsevier, vol. 242(C), pages 351-363.
    4. Ying Ji & Jianhui Wang & Jiacan Xu & Xiaoke Fang & Huaguang Zhang, 2019. "Real-Time Energy Management of a Microgrid Using Deep Reinforcement Learning," Energies, MDPI, vol. 12(12), pages 1-21, June.
    5. Park, Keonwoo & Moon, Ilkyeong, 2022. "Multi-agent deep reinforcement learning approach for EV charging scheduling in a smart grid," Applied Energy, Elsevier, vol. 328(C).
    6. Amrutha Raju Battula & Sandeep Vuddanti & Surender Reddy Salkuti, 2021. "Review of Energy Management System Approaches in Microgrids," Energies, MDPI, vol. 14(17), pages 1-32, September.
    7. Killian, M. & Zauner, M. & Kozek, M., 2018. "Comprehensive smart home energy management system using mixed-integer quadratic-programming," Applied Energy, Elsevier, vol. 222(C), pages 662-672.
    8. Mak, Davye & Choi, Dae-Hyun, 2020. "Optimization framework for coordinated operation of home energy management system and Volt-VAR optimization in unbalanced active distribution networks considering uncertainties," Applied Energy, Elsevier, vol. 276(C).
    9. Chitchai Srithapon & Prasanta Ghosh & Apirat Siritaratiwat & Rongrit Chatthaworn, 2020. "Optimization of Electric Vehicle Charging Scheduling in Urban Village Networks Considering Energy Arbitrage and Distribution Cost," Energies, MDPI, vol. 13(2), pages 1-20, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Deng, Zhipeng & Li, Yuewei & Wang, Xuezheng & Jiang, Zixin & Dong, Bing, 2025. "Quantum computing-enhanced large-scale residential electric vehicle charging management," Applied Energy, Elsevier, vol. 401(PC).
    2. Mohammadreza Ganjian & Mohammad Tabrizian & Nasser Khodabakhshi & Meqdad Ansarian, 2026. "Optimizing energy management in smart homes with Q-deep networks: a deep reinforcement learning approach for sustainable consumption," Quality & Quantity: International Journal of Methodology, Springer, vol. 60(1), pages 1617-1640, February.
    3. Song, Ge & Xie, Hongbin & Zhang, Jingyuan & Fu, Hongdi & Shi, Zhuoran & Feng, Defan & Song, Xuan & Zhang, Haoran, 2025. "Long-term efficient energy management for multi-station collaborative electric vehicle charging: A transformer-based multi-agent reinforcement learning approach," Applied Energy, Elsevier, vol. 397(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jeddi, Babak & Mishra, Yateendra & Ledwich, Gerard, 2021. "Distributed load scheduling in residential neighborhoods for coordinated operation of multiple home energy management systems," Applied Energy, Elsevier, vol. 300(C).
    2. Mak, Davye & Choi, Dae-Hyun, 2020. "Optimization framework for coordinated operation of home energy management system and Volt-VAR optimization in unbalanced active distribution networks considering uncertainties," Applied Energy, Elsevier, vol. 276(C).
    3. Ebrahimi, Seyyed Reza & Rahimiyan, Morteza & Assili, Mohsen & Hajizadeh, Amin, 2022. "Home energy management under correlated uncertainties: A statistical analysis through Copula," Applied Energy, Elsevier, vol. 305(C).
    4. Ana Cabrera-Tobar & Alessandro Massi Pavan & Giovanni Petrone & Giovanni Spagnuolo, 2022. "A Review of the Optimization and Control Techniques in the Presence of Uncertainties for the Energy Management of Microgrids," Energies, MDPI, vol. 15(23), pages 1-38, December.
    5. Lilia Tightiz & Joon Yoo, 2022. "A Review on a Data-Driven Microgrid Management System Integrating an Active Distribution Network: Challenges, Issues, and New Trends," Energies, MDPI, vol. 15(22), pages 1-24, November.
    6. Abid, Md. Shadman & Apon, Hasan Jamil & Hossain, Salman & Ahmed, Ashik & Ahshan, Razzaqul & Lipu, M.S. Hossain, 2024. "A novel multi-objective optimization based multi-agent deep reinforcement learning approach for microgrid resources planning," Applied Energy, Elsevier, vol. 353(PA).
    7. Habibnia, S. & Faraji, M. & Alizadeh, M.H. & Mollayousefi Zadeh, M. & Caire, R. & Gharehpetian, G.B. & Guerrero, J.M., 2026. "Data-driven solutions for microgrids energy management systems: A state-of-the-art survey on current trends and future directions," Renewable and Sustainable Energy Reviews, Elsevier, vol. 228(C).
    8. Rocha, Helder R.O. & Honorato, Icaro H. & Fiorotti, Rodrigo & Celeste, Wanderley C. & Silvestre, Leonardo J. & Silva, Jair A.L., 2021. "An Artificial Intelligence based scheduling algorithm for demand-side energy management in Smart Homes," Applied Energy, Elsevier, vol. 282(PA).
    9. Lorenzo Bartolucci & Stefano Cordiner & Vincenzo Mulone & Marina Santarelli, 2019. "Ancillary Services Provided by Hybrid Residential Renewable Energy Systems through Thermal and Electrochemical Storage Systems," Energies, MDPI, vol. 12(12), pages 1-18, June.
    10. Majdalani, Naim & Aelenei, Daniel & Lopes, Rui Amaral & Silva, Carlos Augusto Santo, 2020. "The potential of energy flexibility of space heating and cooling in Portugal," Utilities Policy, Elsevier, vol. 66(C).
    11. Rittichai Liemthong & Chitchai Srithapon & Prasanta K. Ghosh & Rongrit Chatthaworn, 2022. "Home Energy Management Strategy-Based Meta-Heuristic Optimization for Electrical Energy Cost Minimization Considering TOU Tariffs," Energies, MDPI, vol. 15(2), pages 1-22, January.
    12. Panagiotis Michailidis & Iakovos Michailidis & Elias Kosmatopoulos, 2025. "Reinforcement Learning for Electric Vehicle Charging Management: Theory and Applications," Energies, MDPI, vol. 18(19), pages 1-50, October.
    13. Alqahtani, Mohammed & Hu, Mengqi, 2022. "Dynamic energy scheduling and routing of multiple electric vehicles using deep reinforcement learning," Energy, Elsevier, vol. 244(PA).
    14. Md. Mosaraf Hossain Khan & Amran Hossain & Aasim Ullah & Molla Shahadat Hossain Lipu & S. M. Shahnewaz Siddiquee & M. Shafiul Alam & Taskin Jamal & Hafiz Ahmed, 2021. "Integration of Large-Scale Electric Vehicles into Utility Grid: An Efficient Approach for Impact Analysis and Power Quality Assessment," Sustainability, MDPI, vol. 13(19), pages 1-18, October.
    15. Yuhong Wang & Lei Chen & Hong Zhou & Xu Zhou & Zongsheng Zheng & Qi Zeng & Li Jiang & Liang Lu, 2021. "Flexible Transmission Network Expansion Planning Based on DQN Algorithm," Energies, MDPI, vol. 14(7), pages 1-21, April.
    16. Sulman Shahzad & Muhammad Abbas Abbasi & Hassan Ali & Muhammad Iqbal & Rania Munir & Heybet Kilic, 2023. "Possibilities, Challenges, and Future Opportunities of Microgrids: A Review," Sustainability, MDPI, vol. 15(8), pages 1-28, April.
    17. Tuğba Yeğin & Muhammad Ikram, 2022. "Analysis of Consumers’ Electric Vehicle Purchase Intentions: An Expansion of the Theory of Planned Behavior," Sustainability, MDPI, vol. 14(19), pages 1-27, September.
    18. Yang, Ting & Zhao, Liyuan & Li, Wei & Zomaya, Albert Y., 2021. "Dynamic energy dispatch strategy for integrated energy system based on improved deep reinforcement learning," Energy, Elsevier, vol. 235(C).
    19. Wang, Xuan & Shu, Gequn & Tian, Hua & Wang, Rui & Cai, Jinwen, 2020. "Operation performance comparison of CCHP systems with cascade waste heat recovery systems by simulation and operation optimisation," Energy, Elsevier, vol. 206(C).
    20. Wang, Yi & Qiu, Dawei & Sun, Mingyang & Strbac, Goran & Gao, Zhiwei, 2023. "Secure energy management of multi-energy microgrid: A physical-informed safe reinforcement learning approach," Applied Energy, Elsevier, vol. 335(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:16:y:2023:i:5:p:2357-:d:1084595. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager The email address of this maintainer does not seem to be valid anymore. Please ask MDPI Indexing Manager to update the entry or send us the correct address (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.