IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v257y2020ics0306261919317179.html
   My bibliography  Save this article

Jointly dampening traffic oscillations and improving energy consumption with electric, connected and automated vehicles: A reinforcement learning based approach

Author

Listed:
  • Qu, Xiaobo
  • Yu, Yang
  • Zhou, Mofan
  • Lin, Chin-Teng
  • Wang, Xiangyu

Abstract

It has been well recognized that human driver’s limits, heterogeneity, and selfishness substantially compromise the performance of our urban transport systems. In recent years, in order to deal with these deficiencies, our urban transport systems have been transforming with the blossom of key vehicle technology innovations, most notably, connected and automated vehicles. In this paper, we develop a car following model for electric, connected and automated vehicles based on reinforcement learning with the aim to dampen traffic oscillations (stop-and-go traffic waves) caused by human drivers and improve electric energy consumption. Compared to classical modelling approaches, the proposed reinforcement learning based model significantly reduces the modelling constraints and has the capability of self-learning and self-correction. Experiment results demonstrate that the proposed model is able to improve travel efficiency by reducing the negative impact of traffic oscillations, and it can also reduce the average electric energy consumption.

Suggested Citation

  • Qu, Xiaobo & Yu, Yang & Zhou, Mofan & Lin, Chin-Teng & Wang, Xiangyu, 2020. "Jointly dampening traffic oscillations and improving energy consumption with electric, connected and automated vehicles: A reinforcement learning based approach," Applied Energy, Elsevier, vol. 257(C).
    • Handle: RePEc:eee:appene:v:257:y:2020:i:c:s0306261919317179
    DOI: 10.1016/j.apenergy.2019.114030
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261919317179
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2019.114030?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Villa, Juan Luis & Sallán, Jesús & Llombart, Andrés & Sanz, José Fco, 2009. "Design of a high frequency Inductively Coupled Power Transfer system for electric vehicle battery charge," Applied Energy, Elsevier, vol. 86(3), pages 355-363, March.
    2. ., 2017. "Natural resources in international politics," Chapters, in: International Resource Politics in the Asia-Pacific, chapter 2, pages 19-44, Edward Elgar Publishing.
    3. Hu, Xiaosong & Murgovski, Nikolce & Johannesson, Lars & Egardt, Bo, 2013. "Energy efficiency analysis of a series plug-in hybrid electric bus with different energy management strategies and battery sizes," Applied Energy, Elsevier, vol. 111(C), pages 1001-1009.
    4. Zhou, Fang & Li, Xiaopeng & Ma, Jiaqi, 2017. "Parsimonious shooting heuristic for trajectory design of connected automated traffic part I: Theoretical analysis with generalized time geography," Transportation Research Part B: Methodological, Elsevier, vol. 95(C), pages 394-420.
    5. Treiber, Martin & Kesting, Arne & Helbing, Dirk, 2006. "Delays, inaccuracies and anticipation in microscopic traffic models," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 360(1), pages 71-88.
    6. Kenneth Clements & Liang Li, 2017. "Understanding resource investments," Applied Economics, Taylor & Francis Journals, vol. 49(20), pages 1950-1962, April.
    7. Hou, Cong & Ouyang, Minggao & Xu, Liangfei & Wang, Hewu, 2014. "Approximate Pontryagin’s minimum principle applied to the energy management of plug-in hybrid electric vehicles," Applied Energy, Elsevier, vol. 115(C), pages 174-189.
    8. Madhu Shree & Akhlaq Ahmed, 2017. "New Resources," Indian Journal of Gender Studies, Centre for Women's Development Studies, vol. 24(2), pages 313-315, June.
    9. Song, Ziyou & Li, Jianqiu & Han, Xuebing & Xu, Liangfei & Lu, Languang & Ouyang, Minggao & Hofmann, Heath, 2014. "Multi-objective optimization of a semi-active battery/supercapacitor energy storage system for electric vehicles," Applied Energy, Elsevier, vol. 135(C), pages 212-224.
    10. Zheng, Zuduo & Ahn, Soyoung & Chen, Danjue & Laval, Jorge, 2011. "Freeway traffic oscillations: Microscopic analysis of formations and propagations using Wavelet Transform," Transportation Research Part B: Methodological, Elsevier, vol. 45(9), pages 1378-1388.
    11. Gipps, P.G., 1981. "A behavioural car-following model for computer simulation," Transportation Research Part B: Methodological, Elsevier, vol. 15(2), pages 105-111, April.
    12. Zhang, Jian & Tang, Tie-Qiao & Yu, Shao-Wei, 2018. "An improved car-following model accounting for the preceding car’s taillight," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 492(C), pages 1831-1837.
    13. ., 2017. "Producer politics: resource nationalism," Chapters, in: International Resource Politics in the Asia-Pacific, chapter 4, pages 70-94, Edward Elgar Publishing.
    14. ., 2017. "Whistleblowers as information sources," Chapters, in: Organizational Opportunity and Deviant Behavior, chapter 7, pages 123-134, Edward Elgar Publishing.
    15. Mathew, Tom V. & Ravishankar, K.V.R., 2012. "Neural network based vehicle-following model for mixed traffic conditions," European Transport \ Trasporti Europei, ISTIEE, Institute for the Study of Transport within the European Economic Integration, issue 52, pages 1-4.
    16. Madhu Shree & Akhlaq Ahmed, 2017. "New Resources," Indian Journal of Gender Studies, Centre for Women's Development Studies, vol. 24(3), pages 482-486, October.
    17. Kim, Dong-Hyeon & Lin, Shu-Chin, 2017. "Human capital and natural resource dependence," Structural Change and Economic Dynamics, Elsevier, vol. 40(C), pages 92-102.
    18. Qu, Xiaobo & Zhang, Jin & Wang, Shuaian, 2017. "On the stochastic fundamental diagram for freeway traffic: Model development, analytical properties, validation, and extensive applications," Transportation Research Part B: Methodological, Elsevier, vol. 104(C), pages 256-271.
    19. David Silver & Julian Schrittwieser & Karen Simonyan & Ioannis Antonoglou & Aja Huang & Arthur Guez & Thomas Hubert & Lucas Baker & Matthew Lai & Adrian Bolton & Yutian Chen & Timothy Lillicrap & Fan , 2017. "Mastering the game of Go without human knowledge," Nature, Nature, vol. 550(7676), pages 354-359, October.
    20. ., 2017. "Consumer politics: resource mercantilism," Chapters, in: International Resource Politics in the Asia-Pacific, chapter 3, pages 47-69, Edward Elgar Publishing.
    21. Ma, Jiaqi & Li, Xiaopeng & Zhou, Fang & Hu, Jia & Park, B. Brian, 2017. "Parsimonious shooting heuristic for trajectory design of connected automated traffic part II: Computational issues and optimization," Transportation Research Part B: Methodological, Elsevier, vol. 95(C), pages 421-441.
    22. Chen, Danjue & Laval, Jorge & Zheng, Zuduo & Ahn, Soyoung, 2012. "A behavioral car-following model that captures traffic oscillations," Transportation Research Part B: Methodological, Elsevier, vol. 46(6), pages 744-761.
    23. Wang, Xiubin, 2007. "Modeling the process of information relay through inter-vehicle communication," Transportation Research Part B: Methodological, Elsevier, vol. 41(6), pages 684-700, July.
    24. Robert E. Chandler & Robert Herman & Elliott W. Montroll, 1958. "Traffic Dynamics: Studies in Car Following," Operations Research, INFORMS, vol. 6(2), pages 165-184, April.
    25. ., 2017. "The Northeast Asian scramble for resources," Chapters, in: International Resource Politics in the Asia-Pacific, chapter 6, pages 121-142, Edward Elgar Publishing.
    26. Zhang, Shuo & Xiong, Rui & Sun, Fengchun, 2017. "Model predictive control for power management in a plug-in hybrid electric vehicle with a hybrid energy storage system," Applied Energy, Elsevier, vol. 185(P2), pages 1654-1662.
    27. Fiori, Chiara & Ahn, Kyoungho & Rakha, Hesham A., 2016. "Power-based electric vehicle energy consumption model: Model development and validation," Applied Energy, Elsevier, vol. 168(C), pages 257-268.
    28. Yu, Shaowei & Shi, Zhongke, 2015. "The effects of vehicular gap changes with memory on traffic flow in cooperative adaptive cruise control strategy," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 428(C), pages 206-223.
    29. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    30. Chen, Zhibin & He, Fang & Yin, Yafeng, 2016. "Optimal deployment of charging lanes for electric vehicles in transportation networks," Transportation Research Part B: Methodological, Elsevier, vol. 91(C), pages 344-365.
    31. Denos C. Gazis & Robert Herman & Renfrey B. Potts, 1959. "Car-Following Theory of Steady-State Traffic Flow," Operations Research, INFORMS, vol. 7(4), pages 499-505, August.
    32. Schmidt, Johannes & Eisel, Matthias & Kolbe, Lutz M., 2014. "Assessing the potential of different charging strategies for electric vehicle fleets in closed transport systems," Energy Policy, Elsevier, vol. 74(C), pages 179-189.
    33. Li, Xiaopeng & Wang, Xin & Ouyang, Yanfeng, 2012. "Prediction and field validation of traffic oscillation propagation under nonlinear car-following laws," Transportation Research Part B: Methodological, Elsevier, vol. 46(3), pages 409-423.
    34. Laval, Jorge A. & Daganzo, Carlos F., 2006. "Lane-changing in traffic streams," Transportation Research Part B: Methodological, Elsevier, vol. 40(3), pages 251-264, March.
    35. David Silver & Aja Huang & Chris J. Maddison & Arthur Guez & Laurent Sifre & George van den Driessche & Julian Schrittwieser & Ioannis Antonoglou & Veda Panneershelvam & Marc Lanctot & Sander Dieleman, 2016. "Mastering the game of Go with deep neural networks and tree search," Nature, Nature, vol. 529(7587), pages 484-489, January.
    36. Zhou, Yang & Wang, Meng & Ahn, Soyoung, 2019. "Distributed model predictive control approach for cooperative car-following with guaranteed local and string stability," Transportation Research Part B: Methodological, Elsevier, vol. 128(C), pages 69-86.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xu, Yueru & Zheng, Yuan & Yang, Ying, 2021. "On the movement simulations of electric vehicles: A behavioral model-based approach," Applied Energy, Elsevier, vol. 283(C).
    2. Kan, S.Y. & Chen, B. & Wu, X.F. & Chen, Z.M. & Chen, G.Q., 2019. "Natural gas overview for world economy: From primary supply to final demand via global supply chains," Energy Policy, Elsevier, vol. 124(C), pages 215-225.
    3. Yao, Handong & Li, Qianwen & Li, Xiaopeng, 2020. "A study of relationships in traffic oscillation features based on field experiments," Transportation Research Part A: Policy and Practice, Elsevier, vol. 141(C), pages 339-355.
    4. He, Zhengbing & Zheng, Liang & Guan, Wei, 2015. "A simple nonparametric car-following model driven by field data," Transportation Research Part B: Methodological, Elsevier, vol. 80(C), pages 185-201.
    5. Xiao Xiao & Yunlong Zhang & Xiubin Bruce Wang & Shu Yang & Tianyi Chen, 2021. "Hierarchical Longitudinal Control for Connected and Automated Vehicles in Mixed Traffic on a Signalized Arterial," Sustainability, MDPI, vol. 13(16), pages 1-17, August.
    6. Tian, Junfang & Li, Guangyu & Treiber, Martin & Jiang, Rui & Jia, Ning & Ma, Shoufeng, 2016. "Cellular automaton model simulating spatiotemporal patterns, phase transitions and concave growth pattern of oscillations in traffic flow," Transportation Research Part B: Methodological, Elsevier, vol. 93(PA), pages 560-575.
    7. Oh, Simon & Yeo, Hwasoo, 2015. "Impact of stop-and-go waves and lane changes on discharge rate in recovery flow," Transportation Research Part B: Methodological, Elsevier, vol. 77(C), pages 88-102.
    8. Zhou, Yang & Ahn, Soyoung & Wang, Meng & Hoogendoorn, Serge, 2020. "Stabilizing mixed vehicular platoons with connected automated vehicles: An H-infinity approach," Transportation Research Part B: Methodological, Elsevier, vol. 132(C), pages 152-170.
    9. Zheng, Zuduo, 2014. "Recent developments and research needs in modeling lane changing," Transportation Research Part B: Methodological, Elsevier, vol. 60(C), pages 16-32.
    10. Li, Xiaopeng & Cui, Jianxun & An, Shi & Parsafard, Mohsen, 2014. "Stop-and-go traffic analysis: Theoretical properties, environmental impacts and oscillation mitigation," Transportation Research Part B: Methodological, Elsevier, vol. 70(C), pages 319-339.
    11. Wen, Jianghui & Hong, Lijiang & Dai, Min & Xiao, Xinping & Wu, Chaozhong, 2023. "A stochastic model for stop-and-go phenomenon in traffic oscillation: On the prospective of macro and micro traffic flow," Applied Mathematics and Computation, Elsevier, vol. 440(C).
    12. Hamdar, Samer H. & Mahmassani, Hani S. & Treiber, Martin, 2015. "From behavioral psychology to acceleration modeling: Calibration, validation, and exploration of drivers’ cognitive and safety parameters in a risk-taking environment," Transportation Research Part B: Methodological, Elsevier, vol. 78(C), pages 32-53.
    13. Laval, Jorge A. & Toth, Christopher S. & Zhou, Yi, 2014. "A parsimonious model for the formation of oscillations in car-following models," Transportation Research Part B: Methodological, Elsevier, vol. 70(C), pages 228-238.
    14. Li, Xiaopeng & Wang, Xin & Ouyang, Yanfeng, 2012. "Prediction and field validation of traffic oscillation propagation under nonlinear car-following laws," Transportation Research Part B: Methodological, Elsevier, vol. 46(3), pages 409-423.
    15. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    16. Li, Li & Li, Xiaopeng, 2019. "Parsimonious trajectory design of connected automated traffic," Transportation Research Part B: Methodological, Elsevier, vol. 119(C), pages 1-21.
    17. He, Jia & He, Zhengbing & Fan, Bo & Chen, Yanyan, 2020. "Optimal location of lane-changing warning point in a two-lane road considering different traffic flows," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 540(C).
    18. Wu, Yuankai & Tan, Huachun & Peng, Jiankun & Zhang, Hailong & He, Hongwen, 2019. "Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series-parallel plug-in hybrid electric bus," Applied Energy, Elsevier, vol. 247(C), pages 454-466.
    19. Zhang, Yihao & Chai, Zhaojie & Lykotrafitis, George, 2021. "Deep reinforcement learning with a particle dynamics environment applied to emergency evacuation of a room with obstacles," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 571(C).
    20. Weifan Long & Taixian Hou & Xiaoyi Wei & Shichao Yan & Peng Zhai & Lihua Zhang, 2023. "A Survey on Population-Based Deep Reinforcement Learning," Mathematics, MDPI, vol. 11(10), pages 1-17, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:257:y:2020:i:c:s0306261919317179. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.