IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v343y2023ics0306261923005500.html
   My bibliography  Save this article

A transfer learning method for electric vehicles charging strategy based on deep reinforcement learning

Author

Listed:
  • Wang, Kang
  • Wang, Haixin
  • Yang, Zihao
  • Feng, Jiawei
  • Li, Yanzhen
  • Yang, Junyou
  • Chen, Zhe

Abstract

Reinforcement learning (RL) is popularly used for the development of an orderly charging strategy for electric vehicles (EVs). However, a new environment (e.g., charging areas and times) will cause EV users' driving behaviors and electricity prices to change, which leads to the trained RL-based charging strategy is not suitable. Besides, developing a new RL-based charging strategy for the new environment will cost too much time and data samples. In this paper, a deep transfer reinforcement learning (DTRL)-based charging method for EVs is proposed to realize the transfer of trained RL-based charging strategy to the new environment. Firstly, we formulate the uncertainty problem of EV charging behaviors as a Markov Decision Process (MDP) with an unknown state transfer function. Furthermore, an RL-based charging strategy based on deep deterministic policy gradient (DDPG) is well-trained by using massive driving and environmental data samples. Finally, an EV charging method based on transfer learning (TL) and DDPG is proposed to perform the knowledge transfer on the trained RL-based charging strategy to the new environment. The proposed method is verified by numerous simulations. The results show that the proposed approach can reduce the outliers to meet the user charging demands and shorten the EV charging strategy development time in the new environment.

Suggested Citation

  • Wang, Kang & Wang, Haixin & Yang, Zihao & Feng, Jiawei & Li, Yanzhen & Yang, Junyou & Chen, Zhe, 2023. "A transfer learning method for electric vehicles charging strategy based on deep reinforcement learning," Applied Energy, Elsevier, vol. 343(C).
    • Handle: RePEc:eee:appene:v:343:y:2023:i:c:s0306261923005500
    DOI: 10.1016/j.apenergy.2023.121186
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261923005500
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2023.121186?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Škugor, Branimir & Deur, Joško, 2016. "A bi-level optimisation framework for electric vehicle fleet charging management," Applied Energy, Elsevier, vol. 184(C), pages 1332-1342.
    2. Yagcitekin, Bunyamin & Uzunoglu, Mehmet, 2016. "A double-layer smart charging strategy of electric vehicles taking routing and charge scheduling into account," Applied Energy, Elsevier, vol. 167(C), pages 407-419.
    3. Jianxiao Wang & Haiwang Zhong & Zhifang Yang & Mu Wang & Daniel M. Kammen & Zhu Liu & Ziming Ma & Qing Xia & Chongqing Kang, 2020. "Exploring the trade-offs between electric heating policy and carbon mitigation in China," Nature Communications, Nature, vol. 11(1), pages 1-11, December.
    4. Yin, WanJun & Ming, ZhengFeng & Wen, Tao, 2021. "Scheduling strategy of electric vehicle charging considering different requirements of grid and users," Energy, Elsevier, vol. 232(C).
    5. Yang, Haolin & Schell, Kristen R., 2021. "Real-time electricity price forecasting of wind farms with deep neural network transfer learning and hybrid datasets," Applied Energy, Elsevier, vol. 299(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mehta, R. & Verma, P. & Srinivasan, D. & Yang, Jing, 2019. "Double-layered intelligent energy management for optimal integration of plug-in electric vehicles into distribution systems," Applied Energy, Elsevier, vol. 233, pages 146-155.
    2. Faramarz Saghi & Mustafa Jahangoshai Rezaee, 2023. "Integrating Wavelet Decomposition and Fuzzy Transformation for Improving the Accuracy of Forecasting Crude Oil Price," Computational Economics, Springer;Society for Computational Economics, vol. 61(2), pages 559-591, February.
    3. Kandpal, Bakul & Pareek, Parikshit & Verma, Ashu, 2022. "A robust day-ahead scheduling strategy for EV charging stations in unbalanced distribution grid," Energy, Elsevier, vol. 249(C).
    4. Wang, Jianxiao & An, Qi & Zhao, Yue & Pan, Guangsheng & Song, Jie & Hu, Qinran & Tan, Chin-Woo, 2023. "Role of electrolytic hydrogen in smart city decarbonization in China," Applied Energy, Elsevier, vol. 336(C).
    5. Ma, Sining & Guo, Siyue & Zheng, Dingqian & Chang, Shiyan & Zhang, Xiliang, 2021. "Roadmap towards clean and low carbon heating to 2035: A provincial analysis in northern China," Energy, Elsevier, vol. 225(C).
    6. Müller, Mathias & Blume, Yannic & Reinhard, Janis, 2022. "Impact of behind-the-meter optimised bidirectional electric vehicles on the distribution grid load," Energy, Elsevier, vol. 255(C).
    7. Dapeng Chen & Zhaoxia Jing & Huijuan Tan, 2019. "Optimal Bidding/Offering Strategy for EV Aggregators under a Novel Business Model," Energies, MDPI, vol. 12(7), pages 1-19, April.
    8. Ding, Tao & Sun, Yuge & Huang, Can & Mu, Chenlu & Fan, Yuqi & Lin, Jiang & Qin, Yining, 2022. "Pathways of clean energy heating electrification programs for reducing carbon emissions in Northwest China," Renewable and Sustainable Energy Reviews, Elsevier, vol. 166(C).
    9. Xydas, Erotokritos & Marmaras, Charalampos & Cipcigan, Liana M., 2016. "A multi-agent based scheduling algorithm for adaptive electric vehicles charging," Applied Energy, Elsevier, vol. 177(C), pages 354-365.
    10. Yu, Qing & Li, Weifeng & Zhang, Haoran & Chen, Jinyu, 2022. "GPS data in taxi-sharing system: Analysis of potential demand and assessment of fuel consumption based on routing probability model," Applied Energy, Elsevier, vol. 314(C).
    11. Zamar, David S. & Gopaluni, Bhushan & Sokhansanj, Shahab, 2017. "Optimization of sawmill residues collection for bioenergy production," Applied Energy, Elsevier, vol. 202(C), pages 487-495.
    12. Yang, Tianqi & Shu, Yun & Zhang, Shaohui & Wang, Hongchang & Zhu, Jinwei & Wang, Fan, 2023. "Impacts of end-use electrification on air quality and CO2 emissions in China's northern cities in 2030," Energy, Elsevier, vol. 278(PA).
    13. de Wildt, T.E. & Chappin, E.J.L. & van de Kaa, G. & Herder, P.M. & van de Poel, I.R., 2019. "Conflicting values in the smart electricity grid a comprehensive overview," Renewable and Sustainable Energy Reviews, Elsevier, vol. 111(C), pages 184-196.
    14. Liu, Ling & Wang, Jujie & Li, Jianping & Wei, Lu, 2023. "An online transfer learning model for wind turbine power prediction based on spatial feature construction and system-wide update," Applied Energy, Elsevier, vol. 340(C).
    15. Luo, Lizi & Gu, Wei & Wu, Zhi & Zhou, Suyang, 2019. "Joint planning of distributed generation and electric vehicle charging stations considering real-time charging navigation," Applied Energy, Elsevier, vol. 242(C), pages 1274-1284.
    16. Guangsheng Pan & Qinran Hu & Wei Gu & Shixing Ding & Haifeng Qiu & Yuping Lu, 2021. "Assessment of plum rain’s impact on power system emissions in Yangtze-Huaihe River basin of China," Nature Communications, Nature, vol. 12(1), pages 1-10, December.
    17. Moon, Sang-Keun & Kim, Jin-O, 2017. "Balanced charging strategies for electric vehicles on power systems," Applied Energy, Elsevier, vol. 189(C), pages 44-54.
    18. Li, Xinyu & Cao, Yue & Yan, Fei & Li, Yuzhe & Zhao, Wanlin & Wang, Yue, 2022. "Towards user-friendly energy supplement service considering battery degradation cost," Energy, Elsevier, vol. 249(C).
    19. Meng, Anbo & Wang, Peng & Zhai, Guangsong & Zeng, Cong & Chen, Shun & Yang, Xiaoyi & Yin, Hao, 2022. "Electricity price forecasting with high penetration of renewable energy using attention-based LSTM network trained by crisscross optimization," Energy, Elsevier, vol. 254(PA).
    20. Yangrui Zhang & Peng Tao & Xiangming Wu & Chenguang Yang & Guang Han & Hui Zhou & Yinlong Hu, 2022. "Hourly Electricity Price Prediction for Electricity Market with High Proportion of Wind and Solar Power," Energies, MDPI, vol. 15(4), pages 1-13, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:343:y:2023:i:c:s0306261923005500. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.