IDEAS home Printed from https://ideas.repec.org/a/eee/oprepe/v15y2025ics2214716025000272.html

Deep reinforcement learning for dynamic vehicle routing with demand and traffic uncertainty

Author

Listed:
  • Kadyrov, Shirali
  • Azamov, Azamkhon
  • Abdumajitov, Yelbek
  • Turan, Cemil

Abstract

The capacitated vehicle routing problem with dynamic demand and traffic conditions presents significant challenges in logistics and supply chain optimization. Traditional methods often fail to adapt to real-time uncertainties in customer demand and traffic patterns or scale to large problem instances. In this work, we propose a deep reinforcement learning framework to learn adaptive routing policies for dynamic capacitated vehicle routing problem environments with stochastic demand and traffic. Our approach integrates graph neural networks to encode spatial problem structure and proximal policy optimization to train robust policies under both demand and traffic uncertainty. Experiments on synthetic grid-based routing environments show that our method outperforms classical heuristics and greedy baselines in minimizing travel cost while maintaining feasibility. The learned policies generalize to unseen demand and traffic scenarios and scale to larger graphs than those seen during training. Our results highlight the potential of deep reinforcement learning for real-world dynamic routing problems where both demand and traffic evolve unpredictably.

Suggested Citation

  • Kadyrov, Shirali & Azamov, Azamkhon & Abdumajitov, Yelbek & Turan, Cemil, 2025. "Deep reinforcement learning for dynamic vehicle routing with demand and traffic uncertainty," Operations Research Perspectives, Elsevier, vol. 15(C).
  • Handle: RePEc:eee:oprepe:v:15:y:2025:i:c:s2214716025000272
    DOI: 10.1016/j.orp.2025.100351
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S2214716025000272
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.orp.2025.100351?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Xiaoyu Wang & Baher Abdulhai & Scott Sanner, 2023. "A critical review of traffic signal control and a novel unified view of reinforcement learning and model predictive control approaches for adaptive traffic signal control," Chapters, in: Hussein Dia (ed.), Handbook on Artificial Intelligence and Transport, chapter 17, pages 482-532, Edward Elgar Publishing.
    2. Chrysanthos E. Gounaris & Wolfram Wiesemann & Christodoulos A. Floudas, 2013. "The Robust Capacitated Vehicle Routing Problem Under Demand Uncertainty," Operations Research, INFORMS, vol. 61(3), pages 677-693, June.
    3. Ruibin Bai & Xinan Chen & Zhi-Long Chen & Tianxiang Cui & Shuhui Gong & Wentao He & Xiaoping Jiang & Huan Jin & Jiahuan Jin & Graham Kendall & Jiawei Li & Zheng Lu & Jianfeng Ren & Paul Weng & Ning Xu, 2023. "Analytics and machine learning in vehicle routing research," International Journal of Production Research, Taylor & Francis Journals, vol. 61(1), pages 4-30, January.
    4. Zhang, Jian & Woensel, Tom Van, 2023. "Dynamic vehicle routing with random requests: A literature review," International Journal of Production Economics, Elsevier, vol. 256(C).
    5. Li, Haonan & Wu, Xu & Ribeiro, Marta & Santos, Bruno & Zheng, Pan, 2025. "Deep reinforcement learning approach for real-time airport gate assignment," Operations Research Perspectives, Elsevier, vol. 14(C).
    6. Tang, Tao & Chai, Simin & Wu, Wei & Yin, Jiateng & D’Ariano, Andrea, 2025. "A multi-task deep reinforcement learning approach to real-time railway train rescheduling," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 194(C).
    7. Meraryslan Meraliyev & Cemil Turan & Shirali Kadyrov & Ualikhan Sadyk, 2025. "A Comprehensive Survey of Methods and Challenges of Vehicle Routing Problem with Uncertainties," Mathematics, MDPI, vol. 13(23), pages 1-32, November.
    8. Alarcon Ortega, Emilio J. & Schilde, Michael & Doerner, Karl F., 2020. "Matheuristic search techniques for the consistent inventory routing problem with time windows and split deliveries," Operations Research Perspectives, Elsevier, vol. 7(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Toygar Emre & Rızvan Erol, 2025. "A Vehicle Routing Problem Based on a Long-Distance Transportation Network with an Exact Optimization Algorithm," Mathematics, MDPI, vol. 13(21), pages 1-40, October.
    2. Abouelrous, Abdo & Bliek, Laurens & Gabor, Adriana F. & Wu, Yaoxin & Zhang, Yingqian, 2025. "Reinforcement learning for solving the pricing problem in column generation for routing," Operations Research Perspectives, Elsevier, vol. 15(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Enrico Bartolini & Dominik Goeke & Michael Schneider & Mengdie Ye, 2021. "The Robust Traveling Salesman Problem with Time Windows Under Knapsack-Constrained Travel Time Uncertainty," Transportation Science, INFORMS, vol. 55(2), pages 371-394, March.
    2. Zhang, Huili & An, Xuan & Chen, Cong & Wang, Nengmin & Tong, Weitian, 2025. "Data-driven robust two-stage ferry vehicle management at airports," Omega, Elsevier, vol. 133(C).
    3. Guchhait, Rekha & Sarkar, Biswajit, 2025. "Economic evaluation of an outsourced fourth-party logistics (4PL) under a flexible production system," International Journal of Production Economics, Elsevier, vol. 279(C).
    4. Jamshidian, Fatemeh & Yaghoubi, Saeed & Sadeghi, Mohammad, 2025. "Recursive delivery multiple flying sidekicks traveling salesman problem: An enlightenment of the Covid-19 pandemic," Operations Research Perspectives, Elsevier, vol. 15(C).
    5. Zhenzhen Zhang & Yu Zhang & Roberto Baldacci, 2024. "Generalized Riskiness Index in Vehicle Routing Under Uncertain Travel Times: Formulations, Properties, and Exact Solution Framework," Transportation Science, INFORMS, vol. 58(4), pages 761-780, July.
    6. Ji, Chenlu & Mandania, Rupal & Liu, Jiyin & Liret, Anne, 2022. "Scheduling on-site service deliveries to minimise the risk of missing appointment times," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 158(C).
    7. Angelos Georghiou & Angelos Tsoukalas & Wolfram Wiesemann, 2026. "On the Optimality of Affine Decision Rules in Distributionally Robust Optimization," Management Science, INFORMS, vol. 72(2), pages 1456-1471, February.
    8. Hernan Caceres & Rajan Batta & Qing He, 2017. "School Bus Routing with Stochastic Demand and Duration Constraints," Transportation Science, INFORMS, vol. 51(4), pages 1349-1364, November.
    9. Gitae Kim, 2023. "Dynamic Vehicle Routing Problem with Fuzzy Customer Response," Sustainability, MDPI, vol. 15(5), pages 1-13, March.
    10. Makboul, Salma & Kharraja, Said & Abbassi, Abderrahman & El Hilali Alaoui, Ahmed, 2024. "A multiobjective approach for weekly Green Home Health Care routing and scheduling problem with care continuity and synchronized services," Operations Research Perspectives, Elsevier, vol. 12(C).
    11. Shubhechyya Ghosal & Wolfram Wiesemann, 2020. "The Distributionally Robust Chance-Constrained Vehicle Routing Problem," Operations Research, INFORMS, vol. 68(3), pages 716-732, May.
    12. Chen, Lijian & Chiang, Wen-Chyuan & Russell, Robert & Chen, Jun & Sun, Dengfeng, 2018. "The probabilistic vehicle routing problem with service guarantees," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 111(C), pages 149-164.
    13. Rafael Campos & Leandro C. Coelho & Pedro Munari, 2025. "New formulations for the robust vehicle routing problem with time windows under demand and travel time uncertainty," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 47(2), pages 411-453, June.
    14. Jorge Oyola & Halvard Arntzen & David L. Woodruff, 2017. "The stochastic vehicle routing problem, a literature review, Part II: solution methods," EURO Journal on Transportation and Logistics, Springer;EURO - The Association of European Operational Research Societies, vol. 6(4), pages 349-388, December.
    15. Markov, Iliya & Bierlaire, Michel & Cordeau, Jean-François & Maknoon, Yousef & Varone, Sacha, 2018. "A unified framework for rich routing problems with stochastic demands," Transportation Research Part B: Methodological, Elsevier, vol. 114(C), pages 213-240.
    16. Soraya Fatehi & Michael R. Wagner, 2022. "Crowdsourcing Last-Mile Deliveries," Manufacturing & Service Operations Management, INFORMS, vol. 24(2), pages 791-809, March.
    17. Borumand, Ali & Marandi, Ahmadreza & Nookabadi, Ali S. & Atan, Zümbül, 2024. "An oracle-based algorithm for robust planning of production routing problems in closed-loop supply chains of beverage glass bottles," Omega, Elsevier, vol. 122(C).
    18. de Moor, Danique & Wagenaar, Joris & Poos, Robert & den Hertog, Dick & Fleuren, Hein, 2024. "A robust approach to food aid supply chains," European Journal of Operational Research, Elsevier, vol. 318(1), pages 269-285.
    19. Haihao Lu & Bradley Sturt, 2026. "On the Sparsity of Optimal Linear Decision Rules for a Class of Robust Optimization Problems with Box Uncertainty Sets," Operations Research, INFORMS, vol. 74(1), pages 500-516, January.
    20. Bruck, Bruno P. & Coutinho, Walton P. & Munari, Pedro, 2025. "The Robust Bike sharing Rebalancing Problem: Formulations and a branch-and-cut algorithm," European Journal of Operational Research, Elsevier, vol. 325(1), pages 67-80.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:oprepe:v:15:y:2025:i:c:s2214716025000272. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/operations-research-perspectives .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.