IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v14y2022i16p10351-d892811.html
   My bibliography  Save this article

An Optimal Scheduling Strategy of a Microgrid with V2G Based on Deep Q-Learning

Author

Listed:
  • Yuxin Wen

    (School of Electrical Engineering and Automation, Wuhan University, Wuhan 430072, China)

  • Peixiao Fan

    (School of Electrical Engineering and Automation, Wuhan University, Wuhan 430072, China)

  • Jia Hu

    (State Grid Hubei Electric Power Co., Ltd., Wuhan 430072, China)

  • Song Ke

    (School of Electrical Engineering and Automation, Wuhan University, Wuhan 430072, China)

  • Fuzhang Wu

    (School of Electrical Engineering and Automation, Wuhan University, Wuhan 430072, China)

  • Xu Zhu

    (School of Electrical Engineering and Automation, Wuhan University, Wuhan 430072, China)

Abstract

In recent years, the access of various distributed power sources and electric vehicles (EVs) has brought more and more randomness and uncertainty to the operation and regulation of microgrids. Therefore, an optimal scheduling strategy for microgrids with EVs based on Deep Q-learning is proposed in this paper. Firstly, a vehicle-to-grid (V2G) model considering the mobility of EVs and the randomness of user charging behavior is proposed. The charging time distribution model, charging demand model, state-of-charge (SOC) dynamic model and the model of travel location are comprehensively established, thereby realizing the construction of the mathematical model of the microgrid with EVs: it can obtain the charging/discharging situation in the EV station, so as to obtain the overall output power of the EV station. Secondly, based on Deep Q-learning, the state space and action space are set up according to the actual microgrid system, and the design of the optimal scheduling reward function is completed with the goal of economy. Finally, the calculation example results show that compared with the traditional optimization algorithm, the strategy proposed in this paper has the ability of online learning and can cope with the randomness of renewable resources better. Meanwhile, the agent with experience replay ability can be trained to complete the evolution process, so as to adapt to the nonlinear influence caused by the mobility of EVs and the periodicity of user behavior, which is feasible and superior in the field of optimal scheduling of microgrids with renewable resources and EVs.

Suggested Citation

  • Yuxin Wen & Peixiao Fan & Jia Hu & Song Ke & Fuzhang Wu & Xu Zhu, 2022. "An Optimal Scheduling Strategy of a Microgrid with V2G Based on Deep Q-Learning," Sustainability, MDPI, vol. 14(16), pages 1-18, August.
    • Handle: RePEc:gam:jsusta:v:14:y:2022:i:16:p:10351-:d:892811
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/14/16/10351/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/14/16/10351/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jicheng Liu & Fangqiu Xu & Shuaishuai Lin & Hua Cai & Suli Yan, 2018. "A Multi-Agent-Based Optimization Model for Microgrid Operation Using Dynamic Guiding Chaotic Search Particle Swarm Optimization," Energies, MDPI, vol. 11(12), pages 1-22, November.
    2. Kofinas, P. & Dounis, A.I. & Vouros, G.A., 2018. "Fuzzy Q-Learning for multi-agent decentralized energy management in microgrids," Applied Energy, Elsevier, vol. 219(C), pages 53-67.
    3. Steven Chu & Arun Majumdar, 2012. "Opportunities and challenges for a sustainable energy future," Nature, Nature, vol. 488(7411), pages 294-303, August.
    4. Li, Pengfei & Hu, Weihao & Xu, Xiao & Huang, Qi & Liu, Zhou & Chen, Zhe, 2019. "A frequency control strategy of electric vehicles in microgrid using virtual synchronous generator control," Energy, Elsevier, vol. 189(C).
    5. Zhu, Xianwen & Xia, Mingchao & Chiang, Hsiao-Dong, 2018. "Coordinated sectional droop charging control for EV aggregator enhancing frequency stability of microgrid with high penetration of renewable energy sources," Applied Energy, Elsevier, vol. 210(C), pages 936-943.
    6. Jun Yang & Zhili Zeng & Yufei Tang & Jun Yan & Haibo He & Yunliang Wu, 2015. "Load Frequency Control in Isolated Micro-Grids with Electrical Vehicles Based on Multivariable Generalized Predictive Theory," Energies, MDPI, vol. 8(3), pages 1-20, March.
    7. Eun-Kyu Lee & Wenbo Shi & Rajit Gadh & Wooseong Kim, 2016. "Design and Implementation of a Microgrid Energy Management System," Sustainability, MDPI, vol. 8(11), pages 1-19, November.
    8. Li, Qiang & Gao, Mengkai & Lin, Houfei & Chen, Ziyu & Chen, Minyou, 2019. "MAS-based distributed control method for multi-microgrids with high-penetration renewable energy," Energy, Elsevier, vol. 171(C), pages 284-295.
    9. Rao, Yingqing & Yang, Jun & Xiao, Jinxing & Xu, Bingyan & Liu, Wenjing & Li, Yonghui, 2021. "A frequency control strategy for multimicrogrids with V2G based on the improved robust model predictive control," Energy, Elsevier, vol. 222(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhang, Shulei & Jia, Runda & Pan, Hengxin & Cao, Yankai, 2023. "A safe reinforcement learning-based charging strategy for electric vehicles in residential microgrid," Applied Energy, Elsevier, vol. 348(C).
    2. Jianhong Hao & Ting Huang & Qiuming Xu & Yi Sun, 2023. "Robust Optimal Scheduling of Microgrid with Electric Vehicles Based on Stackelberg Game," Sustainability, MDPI, vol. 15(24), pages 1-15, December.
    3. Jiashun Li & Aixing Li, 2024. "Optimizing Electric Vehicle Integration with Vehicle-to-Grid Technology: The Influence of Price Difference and Battery Costs on Adoption, Profits, and Green Energy Utilization," Sustainability, MDPI, vol. 16(3), pages 1-19, January.
    4. Xiaoqing Bai & Chun Wei & Peijie Li & Dongliang Xiao, 2023. "Editorial for the Special Issue on Sustainable Power Systems and Optimization," Sustainability, MDPI, vol. 15(6), pages 1-3, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Peixiao Fan & Jia Hu & Song Ke & Yuxin Wen & Shaobo Yang & Jun Yang, 2022. "A Frequency–Pressure Cooperative Control Strategy of Multi-Microgrid with an Electric–Gas System Based on MADDPG," Sustainability, MDPI, vol. 14(14), pages 1-20, July.
    2. Rao, Yingqing & Yang, Jun & Xiao, Jinxing & Xu, Bingyan & Liu, Wenjing & Li, Yonghui, 2021. "A frequency control strategy for multimicrogrids with V2G based on the improved robust model predictive control," Energy, Elsevier, vol. 222(C).
    3. Adlan Pradana & Mejbaul Haque & Mithulanathan Nadarajah, 2023. "Control Strategies of Electric Vehicles Participating in Ancillary Services: A Comprehensive Review," Energies, MDPI, vol. 16(4), pages 1-36, February.
    4. Géremi Gilson Dranka & Paula Ferreira, 2020. "Electric Vehicles and Biofuels Synergies in the Brazilian Energy System," Energies, MDPI, vol. 13(17), pages 1-22, August.
    5. Mousavizade, Mirsaeed & Bai, Feifei & Garmabdari, Rasoul & Sanjari, Mohammad & Taghizadeh, Foad & Mahmoudian, Ali & Lu, Junwei, 2023. "Adaptive control of V2Gs in islanded microgrids incorporating EV owner expectations," Applied Energy, Elsevier, vol. 341(C).
    6. Neofytos Neofytou & Konstantinos Blazakis & Yiannis Katsigiannis & Georgios Stavrakakis, 2019. "Modeling Vehicles to Grid as a Source of Distributed Frequency Regulation in Isolated Grids with Significant RES Penetration," Energies, MDPI, vol. 12(4), pages 1-23, February.
    7. Latif, Abdul & Hussain, S.M. Suhail & Das, Dulal Chandra & Ustun, Taha Selim, 2020. "State-of-the-art of controllers and soft computing techniques for regulated load frequency management of single/multi-area traditional and renewable energy based power systems," Applied Energy, Elsevier, vol. 266(C).
    8. Chen, Long Xiang & Xie, Mei Na & Zhao, Pan Pan & Wang, Feng Xiang & Hu, Peng & Wang, Dong Xiang, 2018. "A novel isobaric adiabatic compressed air energy storage (IA-CAES) system on the base of volatile fluid," Applied Energy, Elsevier, vol. 210(C), pages 198-210.
    9. Gui, Yonghao & Wei, Baoze & Li, Mingshen & Guerrero, Josep M. & Vasquez, Juan C., 2018. "Passivity-based coordinated control for islanded AC microgrid," Applied Energy, Elsevier, vol. 229(C), pages 551-561.
    10. Chen, Xuejun & Yang, Yongming & Cui, Zhixin & Shen, Jun, 2019. "Vibration fault diagnosis of wind turbines based on variational mode decomposition and energy entropy," Energy, Elsevier, vol. 174(C), pages 1100-1109.
    11. Restrepo, Mauricio & Cañizares, Claudio A. & Simpson-Porco, John W. & Su, Peter & Taruc, John, 2021. "Optimization- and Rule-based Energy Management Systems at the Canadian Renewable Energy Laboratory microgrid facility," Applied Energy, Elsevier, vol. 290(C).
    12. Johannes Dahlke & Kristina Bogner & Matthias Mueller & Thomas Berger & Andreas Pyka & Bernd Ebersberger, 2020. "Is the Juice Worth the Squeeze? Machine Learning (ML) In and For Agent-Based Modelling (ABM)," Papers 2003.11985, arXiv.org.
    13. Muhammad Habib Ur Rehman & Luigi Coppola & Ernestino Lufrano & Isabella Nicotera & Cataldo Simari, 2023. "Enhancing Water Retention, Transport, and Conductivity Performance in Fuel Cell Applications: Nafion-Based Nanocomposite Membranes with Organomodified Graphene Oxide Nanoplatelets," Energies, MDPI, vol. 16(23), pages 1-11, November.
    14. Pin Li & Jinsuo Zhang, 2019. "Is China’s Energy Supply Sustainable? New Research Model Based on the Exponential Smoothing and GM(1,1) Methods," Energies, MDPI, vol. 12(2), pages 1-30, January.
    15. Nyong-Bassey, Bassey Etim & Giaouris, Damian & Patsios, Charalampos & Papadopoulou, Simira & Papadopoulos, Athanasios I. & Walker, Sara & Voutetakis, Spyros & Seferlis, Panos & Gadoue, Shady, 2020. "Reinforcement learning based adaptive power pinch analysis for energy management of stand-alone hybrid energy storage systems considering uncertainty," Energy, Elsevier, vol. 193(C).
    16. Sung-Fu Hung & Aoni Xu & Xue Wang & Fengwang Li & Shao-Hui Hsu & Yuhang Li & Joshua Wicks & Eduardo González Cervantes & Armin Sedighian Rasouli & Yuguang C. Li & Mingchuan Luo & Dae-Hyun Nam & Ning W, 2022. "A metal-supported single-atom catalytic site enables carbon dioxide hydrogenation," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    17. Zheng, Bobo & Xu, Jiuping & Ni, Ting & Li, Meihui, 2015. "Geothermal energy utilization trends from a technological paradigm perspective," Renewable Energy, Elsevier, vol. 77(C), pages 430-441.
    18. Mao, Guozhu & Zou, Hongyang & Chen, Guanyi & Du, Huibin & Zuo, Jian, 2015. "Past, current and future of biomass energy research: A bibliometric analysis," Renewable and Sustainable Energy Reviews, Elsevier, vol. 52(C), pages 1823-1833.
    19. Ahmadi, Seyed Ehsan & Sadeghi, Delnia & Marzband, Mousa & Abusorrah, Abdullah & Sedraoui, Khaled, 2022. "Decentralized bi-level stochastic optimization approach for multi-agent multi-energy networked micro-grids with multi-energy storage technologies," Energy, Elsevier, vol. 245(C).
    20. Luo, Rongrong & Wang, Liuwei & Yu, Wei & Shao, Feilong & Shen, Haikuo & Xie, Huaqing, 2023. "High energy storage density titanium nitride-pentaerythritol solid–solid composite phase change materials for light-thermal-electric conversion," Applied Energy, Elsevier, vol. 331(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:14:y:2022:i:16:p:10351-:d:892811. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.