IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v316y2025ics0360544225000623.html
   My bibliography  Save this article

Deep reinforcement learning-based plug-in electric vehicle charging/discharging scheduling in a home energy management system

Author

Listed:
  • Mansour, Shaza H.
  • Azzam, Sarah M.
  • Hasanien, Hany M.
  • Tostado-Véliz, Marcos
  • Alkuhayli, Abdulaziz
  • Jurado, Francisco

Abstract

With the emergence of plug-in electric vehicles (PEVs) in smart grids (SGs) that helps in SG decarbonization, it has become crucial to harness these PEVs by optimizing their charging and discharging schedules in a smart home setting. However, uncertainties in arrival time, departure time, and state of charge (SOC) make scheduling tasks challenging. This paper proposes a two-stage approach whose objectives are minimizing both the PEV charging cost and the electricity bill of the smart home. In the first level, a deep reinforcement learning (DRL) method, a soft actor-critic (SAC)-based algorithm, is presented for a smart home PEV charging/discharging (C/D) scheduling under real-time pricing (RTP) tariff. In the second level, the obtained schedule is provided as an input to a home energy management system (HEMS) problem. The HEMS problem is formulated as a mixed integer linear programming (MILP) problem for scheduling home appliances and a battery energy storage system (BESS). SAC is compared to different reinforcement learning (RL) algorithms and disorderly PEV C/D for four samples from different seasons. The results show that SAC achieves the highest average rewards, lowest charging cost, and reaches the required SOC upon departure. Compared with other RL algorithms, SAC can achieve a PEV charging cost saving of up to 51.45 % during the summer season. It is also illustrated that SAC causes a huge cost reduction compared to the disorderly C/D schedule. The HEMS appliances and BESS schedules with the SAC-scheduled PEV are shown for four samples. These schedules are compared with the HEMS schedules with disorderly PEV scheduling, and without PEVs. HEMS schedules with SAC-scheduled PEVs can save costs up to 83.29 %, and 15.69 % compared with HEMS with disorderly PEV scheduling, and to the HEMS schedule without PEVs, respectively.

Suggested Citation

  • Mansour, Shaza H. & Azzam, Sarah M. & Hasanien, Hany M. & Tostado-Véliz, Marcos & Alkuhayli, Abdulaziz & Jurado, Francisco, 2025. "Deep reinforcement learning-based plug-in electric vehicle charging/discharging scheduling in a home energy management system," Energy, Elsevier, vol. 316(C).
    • Handle: RePEc:eee:energy:v:316:y:2025:i:c:s0360544225000623
    DOI: 10.1016/j.energy.2025.134420
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544225000623
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2025.134420?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Makeen, Peter & Ghali, Hani A. & Memon, Saim & Duan, Fang, 2023. "Smart techno-economic operation of electric vehicle charging station in Egypt," Energy, Elsevier, vol. 264(C).
    2. Zhang, Gang & Wen, Jiaxing & Xie, Tuo & Zhang, Kaoshe & Jia, Rong, 2023. "Bi-layer economic scheduling for integrated energy system based on source-load coordinated carbon reduction," Energy, Elsevier, vol. 280(C).
    3. Das, Ridoy & Wang, Yue & Putrus, Ghanim & Kotter, Richard & Marzband, Mousa & Herteleer, Bert & Warmerdam, Jos, 2020. "Multi-objective techno-economic-environmental optimisation of electric vehicle for energy services," Applied Energy, Elsevier, vol. 257(C).
    4. Amit Shewale & Anil Mokhade & Nitesh Funde & Neeraj Dhanraj Bokde, 2020. "An Overview of Demand Response in Smart Grid and Optimization Techniques for Efficient Residential Appliance Scheduling Problem," Energies, MDPI, vol. 13(16), pages 1-31, August.
    5. Zhang, Shulei & Jia, Runda & Pan, Hengxin & Cao, Yankai, 2023. "A safe reinforcement learning-based charging strategy for electric vehicles in residential microgrid," Applied Energy, Elsevier, vol. 348(C).
    6. Yao, Zhaosheng & Wang, Zhiyuan & Ran, Lun, 2023. "Smart charging and discharging of electric vehicles based on multi-objective robust optimization in smart cities," Applied Energy, Elsevier, vol. 343(C).
    7. Dimitrios Vamvakas & Panagiotis Michailidis & Christos Korkas & Elias Kosmatopoulos, 2023. "Review and Evaluation of Reinforcement Learning Frameworks on Smart Grid Applications," Energies, MDPI, vol. 16(14), pages 1-38, July.
    8. Liang, Zhaowen & Ruan, Jiageng & Wang, Zhenpo & Liu, Kai & Li, Bin, 2024. "Soft actor-critic-based EMS design for dual motor battery electric bus," Energy, Elsevier, vol. 288(C).
    9. Ren, Kezheng & Liu, Jun & Wu, Zeyang & Liu, Xinglei & Nie, Yongxin & Xu, Haitao, 2024. "A data-driven DRL-based home energy management system optimization framework considering uncertain household parameters," Applied Energy, Elsevier, vol. 355(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Panagiotis Michailidis & Iakovos Michailidis & Dimitrios Vamvakas & Elias Kosmatopoulos, 2023. "Model-Free HVAC Control in Buildings: A Review," Energies, MDPI, vol. 16(20), pages 1-45, October.
    2. Yossi Hadad & Baruch Keren & Dima Alberg, 2023. "An Expert System for Ranking and Matching Electric Vehicles to Customer Specifications and Requirements," Energies, MDPI, vol. 16(11), pages 1-18, May.
    3. Nguyen, Hai-Tra & Safder, Usman & Loy-Benitez, Jorge & Yoo, ChangKyoo, 2022. "Optimal demand side management scheduling-based bidirectional regulation of energy distribution network for multi-residential demand response with self-produced renewable energy," Applied Energy, Elsevier, vol. 322(C).
    4. Yao, Zhaosheng & Ran, Lun & Wang, Zhiyuan & Guo, Xian, 2024. "Integrated management of electric vehicle sharing system operations and Internet of Vehicles energy scheduling," Energy, Elsevier, vol. 309(C).
    5. Zhang, Yiwen & Lin, Rui & Mei, Zhen & Lyu, Minghao & Jiang, Huaiguang & Xue, Ying & Zhang, Jun & Gao, David Wenzhong, 2024. "Interior-point policy optimization based multi-agent deep reinforcement learning method for secure home energy management under various uncertainties," Applied Energy, Elsevier, vol. 376(PA).
    6. Thamer Alquthami & Ahmad H. Milyani & Muhammad Awais & Muhammad B. Rasheed, 2021. "An Incentive Based Dynamic Pricing in Smart Grid: A Customer’s Perspective," Sustainability, MDPI, vol. 13(11), pages 1-17, May.
    7. Mingyu Kang & Bosung Lee & Younsoo Lee, 2025. "A Robust Optimization Approach for E-Bus Charging and Discharging Scheduling with Vehicle-to-Grid Integration," Mathematics, MDPI, vol. 13(9), pages 1-25, April.
    8. Andre Leippi & Markus Fleschutz & Michael D. Murphy, 2022. "A Review of EV Battery Utilization in Demand Response Considering Battery Degradation in Non-Residential Vehicle-to-Grid Scenarios," Energies, MDPI, vol. 15(9), pages 1-22, April.
    9. Amit Shewale & Anil Mokhade & Nitesh Funde & Neeraj Dhanraj Bokde, 2022. "A Survey of Efficient Demand-Side Management Techniques for the Residential Appliance Scheduling Problem in Smart Homes," Energies, MDPI, vol. 15(8), pages 1-34, April.
    10. Zhang, Yachao & Liu, Wei & Huang, Zhanghao & Zheng, Feng & Le, Jian & Zhu, Shu, 2021. "Distributionally robust coordination optimization scheduling for electricity-gas-transportation coupled system considering multiple uncertainties," Renewable Energy, Elsevier, vol. 163(C), pages 2037-2052.
    11. Neaimeh, Myriam & Crozier, Constance & Deakin, Matthew, 2025. "Learning by charging: Understanding consumers’ changing attitudes towards vehicle-to-grid," Applied Energy, Elsevier, vol. 382(C).
    12. Kakkar, Riya & Agrawal, Smita & Tanwar, Sudeep, 2024. "A systematic survey on demand response management schemes for electric vehicles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 203(C).
    13. Al-Bahrani, Loau Tawfak & Horan, Ben & Seyedmahmoudian, Mehdi & Stojcevski, Alex, 2020. "Dynamic economic emission dispatch with load dema nd management for the load demand of electric vehicles during crest shaving and valley filling in smart cities environment," Energy, Elsevier, vol. 195(C).
    14. Nelson, James & Johnson, Nathan G. & Fahy, Kelsey & Hansen, Timothy A., 2020. "Statistical development of microgrid resilience during islanding operations," Applied Energy, Elsevier, vol. 279(C).
    15. Yun, Yunyun & Zhang, Dahai & Yang, Shengchun & Li, Yaping & Yan, Jiahao, 2023. "Low-carbon optimal dispatch of integrated energy system considering the operation of oxy-fuel combustion coupled with power-to-gas and hydrogen-doped gas equipment," Energy, Elsevier, vol. 283(C).
    16. Chen, Qi & Kuang, Zhonghong & Liu, Xiaohua & Zhang, Tao, 2024. "Application-oriented assessment of grid-connected PV-battery system with deep reinforcement learning in buildings considering electricity price dynamics," Applied Energy, Elsevier, vol. 364(C).
    17. Ibham Veza & Mohd Syaifuddin & Muhammad Idris & Safarudin Gazali Herawan & Abdulfatah Abdu Yusuf & Islam Md Rizwanul Fattah, 2024. "Electric Vehicle (EV) Review: Bibliometric Analysis of Electric Vehicle Trend, Policy, Lithium-Ion Battery, Battery Management, Charging Infrastructure, Smart Charging, and Electric Vehicle-to-Everyth," Energies, MDPI, vol. 17(15), pages 1-43, July.
    18. Moslem Dehghani & Seyyed Mohammad Bornapour & Ehsan Sheybani, 2025. "Enhanced Energy Management System in Smart Homes Considering Economic, Technical, and Environmental Aspects: A Novel Modification-Based Grey Wolf Optimizer," Energies, MDPI, vol. 18(5), pages 1-30, February.
    19. Tian, Zhe & Li, Xiaoyuan & Niu, Jide & Zhou, Ruoyu & Li, Feng, 2024. "Enhancing operation flexibility of distributed energy systems: A flexible multi-objective optimization planning method considering long-term and temporary objectives," Energy, Elsevier, vol. 288(C).
    20. Liu, Fen & Yang, Na & Li, Weihua & Wang, Fuqiang & Yang, Lizhong & Liu, Hui & Du, Qian & Romagnoli, Alessandro & Wang, Jianfeng, 2025. "Efficient and lightweight battery protection film with a bionic polar bear structure in extreme environments," Applied Energy, Elsevier, vol. 380(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:316:y:2025:i:c:s0360544225000623. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.