IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v11y2019i14p3900-d249320.html
   My bibliography  Save this article

Congestion Control in Charging Stations Allocation with Q-Learning

Author

Listed:
  • Li Zhang

    (School of Economic and Management, Chongqing Jiaotong University, Chongqing 400074, China)

  • Ke Gong

    (School of Economic and Management, Chongqing Jiaotong University, Chongqing 400074, China)

  • Maozeng Xu

    (School of Economic and Management, Chongqing Jiaotong University, Chongqing 400074, China)

Abstract

Navigation systems can help in allocating public charging stations to electric vehicles (EVs) with the aim of minimizing EVs’ charging time by integrating sufficient data. However, the existing systems only consider their travel time and transform the allocation as a routing problem. In this paper, we involve the queuing time in stations as one part of EVs’ charging time, and another part is the travel time on roads. Roads and stations are easily congested resources, and we constructed a joint-resource congestion game to describe the interaction between vehicles and resources. With a finite number of vehicles and resources, there exists a Nash equilibrium. To realize a self-adaptive allocation work, we applied the Q-learning algorithm on systems, defining sets of states and actions in our constructed environment. After being allocated one by one, vehicles concurrently requesting to be charged will be processed properly. We collected urban road network data from Chongqing city and conducted experiments. The results illustrate the proposed method can be used to solve the problem, and its convergence performance was better than the genetic algorithm. The road capacity and the number of EVs affected the initial of Q-value, and not the convergence trends.

Suggested Citation

  • Li Zhang & Ke Gong & Maozeng Xu, 2019. "Congestion Control in Charging Stations Allocation with Q-Learning," Sustainability, MDPI, vol. 11(14), pages 1-11, July.
    • Handle: RePEc:gam:jsusta:v:11:y:2019:i:14:p:3900-:d:249320
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/11/14/3900/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/11/14/3900/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Simona Bigerna & Silvia Micheli, 2018. "Attitudes Toward Electric Vehicles: The Case of Perugia Using a Fuzzy Set Analysis," Sustainability, MDPI, vol. 10(11), pages 1-14, November.
    2. Shaohua Cui & Hui Zhao & Huijie Wen & Cuiping Zhang, 2018. "Locating Multiple Size and Multiple Type of Charging Station for Battery Electricity Vehicles," Sustainability, MDPI, vol. 10(9), pages 1-20, September.
    3. Milad Akbari & Morris Brenna & Michela Longo, 2018. "Optimal Locating of Electric Vehicle Charging Stations by Application of Genetic Algorithm," Sustainability, MDPI, vol. 10(4), pages 1-14, April.
    4. Vazifeh, Mohammad M. & Zhang, Hongmou & Santi, Paolo & Ratti, Carlo, 2019. "Optimizing the deployment of electric vehicle charging stations using pervasive mobility data," Transportation Research Part A: Policy and Practice, Elsevier, vol. 121(C), pages 75-91.
    5. Luo, Lizi & Gu, Wei & Wu, Zhi & Zhou, Suyang, 2019. "Joint planning of distributed generation and electric vehicle charging stations considering real-time charging navigation," Applied Energy, Elsevier, vol. 242(C), pages 1274-1284.
    6. Haoming Liu & Wenqian Yin & Xiaoling Yuan & Man Niu, 2018. "Reserving Charging Decision-Making Model and Route Plan for Electric Vehicles Considering Information of Traffic and Charging Station," Sustainability, MDPI, vol. 10(5), pages 1-20, April.
    7. Dan-Bi Bak & Jae-Seok Bak & Sung-Yul Kim, 2018. "Strategies for Implementing Public Service Electric Bus Lines by Charging Type in Daegu Metropolitan City, South Korea," Sustainability, MDPI, vol. 10(10), pages 1-16, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Khalil Bachiri & Ali Yahyaouy & Hamid Gualous & Maria Malek & Younes Bennani & Philippe Makany & Nicoleta Rogovschi, 2023. "Multi-Agent DDPG Based Electric Vehicles Charging Station Recommendation," Energies, MDPI, vol. 16(16), pages 1-17, August.
    2. Fescioglu-Unver, Nilgun & Yıldız Aktaş, Melike, 2023. "Electric vehicle charging service operations: A review of machine learning applications for infrastructure planning, control, pricing and routing," Renewable and Sustainable Energy Reviews, Elsevier, vol. 188(C).
    3. Kim, Hyunjung & Kim, Dae-Wook & Kim, Man-Keun, 2022. "Economics of charging infrastructure for electric vehicles in Korea," Energy Policy, Elsevier, vol. 164(C).
    4. Junchi Ma & Yuan Zhang & Zongtao Duan & Lei Tang, 2023. "PROLIFIC: Deep Reinforcement Learning for Efficient EV Fleet Scheduling and Charging," Sustainability, MDPI, vol. 15(18), pages 1-22, September.
    5. Mevan Wijewardena & Michael J. Neely, 2023. "A Two-Player Resource-Sharing Game with Asymmetric Information," Games, MDPI, vol. 14(5), pages 1-27, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Emilia M. Szumska & Rafał S. Jurecki, 2021. "Parameters Influencing on Electric Vehicle Range," Energies, MDPI, vol. 14(16), pages 1-23, August.
    2. Sayarshad, Hamid R., 2025. "Coordinated routing, charging, and power grid for electric and hydrogen vehicles with renewable energy integration," Renewable Energy, Elsevier, vol. 243(C).
    3. Mikołaj Schmidt & Paweł Zmuda-Trzebiatowski & Marcin Kiciński & Piotr Sawicki & Konrad Lasak, 2021. "Multiple-Criteria-Based Electric Vehicle Charging Infrastructure Design Problem," Energies, MDPI, vol. 14(11), pages 1-34, May.
    4. Raja S, Charles & Kumar N M, Vijaya & J, Senthil kumar & Nesamalar J, Jeslin Drusila, 2021. "Enhancing system reliability by optimally integrating PHEV charging station and renewable distributed generators: A Bi-Level programming approach," Energy, Elsevier, vol. 229(C).
    5. Woo, Hyeon & Son, Yongju & Cho, Jintae & Kim, Sung-Yul & Choi, Sungyun, 2023. "Optimal expansion planning of electric vehicle fast charging stations," Applied Energy, Elsevier, vol. 342(C).
    6. Zhou, Guangyou & Zhu, Zhiwei & Luo, Sumei, 2022. "Location optimization of electric vehicle charging stations: Based on cost model and genetic algorithm," Energy, Elsevier, vol. 247(C).
    7. Feifeng Zheng & Zhaojie Wang & Ming Liu, 2022. "Overnight charging scheduling of battery electric buses with uncertain charging time," Operational Research, Springer, vol. 22(5), pages 4865-4903, November.
    8. Wenz, Klaus-Peter & Serrano-Guerrero, Xavier & Barragán-Escandón, Antonio & González, L.G. & Clairand, Jean-Michel, 2021. "Route prioritization of urban public transportation from conventional to electric buses: A new methodology and a study of case in an intermediate city of Ecuador," Renewable and Sustainable Energy Reviews, Elsevier, vol. 148(C).
    9. Cristian Giovanni Colombo & Carola Leone & Seyed Mahdi Miraftabzadeh & Nicoletta Matera & Michela Longo, 2025. "Location of Charging Stations Considering Services and Power Losses: Case Study," Energies, MDPI, vol. 18(18), pages 1-25, September.
    10. Sugunakar Mamidala & Yellapragada Venkata Pavan Kumar & Rammohan Mallipeddi, 2025. "Revolutionizing Electric Vehicle Charging Stations with Efficient Deep Q Networks Powered by Multimodal Bioinspired Analysis for Improved Performance," Energies, MDPI, vol. 18(7), pages 1-30, March.
    11. Basso, Franco & Cox, Tomás & Pezoa, Raúl & Maldonado, Tomás & Varas, Mauricio, 2024. "Characterizing last-mile freight transportation using mobile phone data: The case of Santiago, Chile," Transportation Research Part A: Policy and Practice, Elsevier, vol. 186(C).
    12. Lin, Haiyang & Bian, Caiyun & Wang, Yu & Li, Hailong & Sun, Qie & Wallin, Fredrik, 2022. "Optimal planning of intra-city public charging stations," Energy, Elsevier, vol. 238(PC).
    13. Park, Hyunwoo & Lee, Chungmok, 2024. "An exact algorithm for maximum electric vehicle flow coverage problem with heterogeneous chargers, nonlinear charging time and route deviations," European Journal of Operational Research, Elsevier, vol. 315(3), pages 926-951.
    14. Cui, Jingshi & Cao, Yi & Wang, Bo & Wu, Jiaman, 2025. "Multi-stage adaptive expansion of EV charging stations considering impacts from the transportation network and power grid," Applied Energy, Elsevier, vol. 386(C).
    15. Scorrano, Mariangela & Danielis, Romeo & Giansoldati, Marco, 2020. "Dissecting the total cost of ownership of fully electric cars in Italy: The impact of annual distance travelled, home charging and urban driving," Research in Transportation Economics, Elsevier, vol. 80(C).
    16. Wang, Hua & Zhao, De & Cai, Yutong & Meng, Qiang & Ong, Ghim Ping, 2021. "Taxi trajectory data based fast-charging facility planning for urban electric taxi systems," Applied Energy, Elsevier, vol. 286(C).
    17. Junpeng Cai & Dewang Chen & Shixiong Jiang & Weijing Pan, 2020. "Dynamic-Area-Based Shortest-Path Algorithm for Intelligent Charging Guidance of Electric Vehicles," Sustainability, MDPI, vol. 12(18), pages 1-20, September.
    18. Bong-Gi Choi & Byeong-Chan Oh & Sungyun Choi & Sung-Yul Kim, 2020. "Selecting Locations of Electric Vehicle Charging Stations Based on the Traffic Load Eliminating Method," Energies, MDPI, vol. 13(7), pages 1-20, April.
    19. Sumeet Sahay & Saubhagya Ranjan Biswal & Gauri Shankar & Amitkumar V. Jha & Deepak Kumar Gupta & Sarita Samal & Alin-Gheorghita Mazare & Nicu Bizon, 2025. "Optimal Integration of New Technologies and Energy Sources into Radial Distribution Systems Using Fuzzy African Vulture Algorithm," Sustainability, MDPI, vol. 17(4), pages 1-25, February.
    20. Chao Wang & Zhuoqun Sun & Zhirui Ye, 2020. "On-Road Bus Emission Comparison for Diverse Locations and Fuel Types in Real-World Operation Conditions," Sustainability, MDPI, vol. 12(5), pages 1-14, February.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:11:y:2019:i:14:p:3900-:d:249320. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.