IDEAS home Printed from https://ideas.repec.org/a/eee/transe/v186y2024ics136655452400142x.html
   My bibliography  Save this article

Vehicle dispatching and routing of on-demand intercity ride-pooling services: A multi-agent hierarchical reinforcement learning approach

Author

Listed:
  • Si, Jinhua
  • He, Fang
  • Lin, Xi
  • Tang, Xindi

Abstract

The integrated development of city clusters has given rise to an increasing demand for intercity travel. Intercity ride-pooling service exhibits considerable potential in upgrading traditional intercity bus services by implementing demand-responsive enhancements. Nevertheless, its online operations suffer the inherent complexities due to the coupling of vehicle resource allocation among cities and pooled-ride vehicle routing. To tackle these challenges, this study proposes a two-level framework designed to facilitate online fleet management. Specifically, a novel multi-agent feudal reinforcement learning model is proposed at the upper level of the framework to cooperatively assign idle vehicles to different intercity lines, while the lower level updates the routes of vehicles using an adaptive large neighborhood search heuristic. Numerical studies based on the realistic dataset of Xiamen and its surrounding cities in China show that the proposed framework effectively mitigates the supply and demand imbalances, and achieves significant improvement in both the average daily system profit and order fulfillment ratio.

Suggested Citation

  • Si, Jinhua & He, Fang & Lin, Xi & Tang, Xindi, 2024. "Vehicle dispatching and routing of on-demand intercity ride-pooling services: A multi-agent hierarchical reinforcement learning approach," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 186(C).
  • Handle: RePEc:eee:transe:v:186:y:2024:i:c:s136655452400142x
    DOI: 10.1016/j.tre.2024.103551
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S136655452400142X
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.tre.2024.103551?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Goeke, Dominik, 2019. "Granular tabu search for the pickup and delivery problem with time windows and electric vehicles," European Journal of Operational Research, Elsevier, vol. 278(3), pages 821-836.
    2. Stefan Ropke & David Pisinger, 2006. "An Adaptive Large Neighborhood Search Heuristic for the Pickup and Delivery Problem with Time Windows," Transportation Science, INFORMS, vol. 40(4), pages 455-472, November.
    3. Guo, Xiaotong & Caros, Nicholas S. & Zhao, Jinhua, 2021. "Robust matching-integrated vehicle rebalancing in ride-hailing system with uncertain demand," Transportation Research Part B: Methodological, Elsevier, vol. 150(C), pages 161-189.
    4. Timo Gschwind & Stefan Irnich, 2015. "Effective Handling of Dynamic Time Windows and Its Application to Solving the Dial-a-Ride Problem," Transportation Science, INFORMS, vol. 49(2), pages 335-354, May.
    5. Lee, Enoch & Cen, Xuekai & Lo, Hong K., 2021. "Zonal-based flexible bus service under elastic stochastic demand," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 152(C).
    6. Liu, Yining & Ouyang, Yanfeng, 2021. "Mobility service design via joint optimization of transit networks and demand-responsive services," Transportation Research Part B: Methodological, Elsevier, vol. 151(C), pages 22-41.
    7. Ho, Sin C. & Szeto, W.Y. & Kuo, Yong-Hong & Leung, Janny M.Y. & Petering, Matthew & Tou, Terence W.H., 2018. "A survey of dial-a-ride problems: Literature review and recent developments," Transportation Research Part B: Methodological, Elsevier, vol. 111(C), pages 395-421.
    8. Guo, Jiaqi & Long, Jiancheng & Xu, Xiaoming & Yu, Miao & Yuan, Kai, 2022. "The vehicle routing problem of intercity ride-sharing between two cities," Transportation Research Part B: Methodological, Elsevier, vol. 158(C), pages 113-139.
    9. Yang, Hai & Qin, Xiaoran & Ke, Jintao & Ye, Jieping, 2020. "Optimizing matching time interval and matching radius in on-demand ride-sourcing markets," Transportation Research Part B: Methodological, Elsevier, vol. 131(C), pages 84-105.
    10. Braekers, Kris & Kovacs, Attila A., 2016. "A multi-period dial-a-ride problem with driver consistency," Transportation Research Part B: Methodological, Elsevier, vol. 94(C), pages 355-377.
    11. Paul Czioska & Ronny Kutadinata & Aleksandar Trifunović & Stephan Winter & Monika Sester & Bernhard Friedrich, 2019. "Real-world meeting points for shared demand-responsive transportation systems," Public Transport, Springer, vol. 11(2), pages 341-377, August.
    12. Naccache, Salma & Côté, Jean-François & Coelho, Leandro C., 2018. "The multi-pickup and delivery problem with time windows," European Journal of Operational Research, Elsevier, vol. 269(1), pages 353-362.
    13. Jean-François Cordeau, 2006. "A Branch-and-Cut Algorithm for the Dial-a-Ride Problem," Operations Research, INFORMS, vol. 54(3), pages 573-586, June.
    14. Gregory A. Godfrey & Warren B. Powell, 2002. "An Adaptive Dynamic Programming Algorithm for Dynamic Fleet Management, II: Multiperiod Travel Times," Transportation Science, INFORMS, vol. 36(1), pages 40-54, February.
    15. Agatz, Niels & Erera, Alan & Savelsbergh, Martin & Wang, Xing, 2012. "Optimization for dynamic ride-sharing: A review," European Journal of Operational Research, Elsevier, vol. 223(2), pages 295-303.
    16. Timo Gschwind & Michael Drexl, 2019. "Adaptive Large Neighborhood Search with a Constant-Time Feasibility Test for the Dial-a-Ride Problem," Transportation Science, INFORMS, vol. 53(2), pages 480-491, March.
    17. Ganji, S.S. & Ahangar, A.N. & Awasthi, Anjali & Jamshidi Bandari, Smaneh, 2021. "Psychological analysis of intercity bus passenger satisfaction using Q methodology," Transportation Research Part A: Policy and Practice, Elsevier, vol. 154(C), pages 345-363.
    18. Gregory A. Godfrey & Warren B. Powell, 2002. "An Adaptive Dynamic Programming Algorithm for Dynamic Fleet Management, I: Single Period Travel Times," Transportation Science, INFORMS, vol. 36(1), pages 21-39, February.
    19. Demir, Emrah & Bektaş, Tolga & Laporte, Gilbert, 2012. "An adaptive large neighborhood search heuristic for the Pollution-Routing Problem," European Journal of Operational Research, Elsevier, vol. 223(2), pages 346-359.
    20. Wang, Hai & Yang, Hai, 2019. "Ridesourcing systems: A framework and review," Transportation Research Part B: Methodological, Elsevier, vol. 129(C), pages 122-155.
    21. Ouyang, Yanfeng & Yang, Haolin & Daganzo, Carlos F., 2021. "Performance of reservation-based carpooling services under detour and waiting time restrictions," Transportation Research Part B: Methodological, Elsevier, vol. 150(C), pages 370-385.
    22. Zhixing Luo & Mengyang Liu & Andrew Lim, 2019. "A Two-Phase Branch-and-Price-and-Cut for a Dial-a-Ride Problem in Patient Transportation," Service Science, INFORMS, vol. 53(1), pages 113-130, February.
    23. Sun, Peng & Veelenturf, Lucas P. & Hewitt, Mike & Van Woensel, Tom, 2020. "Adaptive large neighborhood search for the time-dependent profitable pickup and delivery problem with time windows," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 138(C).
    24. Lei, Chao & Jiang, Zhoutong & Ouyang, Yanfeng, 2020. "Path-based dynamic pricing for vehicle allocation in ridesharing systems with fully compliant drivers," Transportation Research Part B: Methodological, Elsevier, vol. 132(C), pages 60-75.
    25. Tafreshian, Amirmahdi & Abdolmaleki, Mojtaba & Masoud, Neda & Wang, Huizhu, 2021. "Proactive shuttle dispatching in large-scale dynamic dial-a-ride systems," Transportation Research Part B: Methodological, Elsevier, vol. 150(C), pages 227-259.
    26. Bongiovanni, Claudia & Kaspi, Mor & Cordeau, Jean-François & Geroliminis, Nikolas, 2022. "A machine learning-driven two-phase metaheuristic for autonomous ridesharing operations," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 165(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guo, Jiaqi & Long, Jiancheng & Xu, Xiaoming & Yu, Miao & Yuan, Kai, 2022. "The vehicle routing problem of intercity ride-sharing between two cities," Transportation Research Part B: Methodological, Elsevier, vol. 158(C), pages 113-139.
    2. Schulz, Arne & Pfeiffer, Christian, 2024. "Using fixed paths to improve branch-and-cut algorithms for precedence-constrained routing problems," European Journal of Operational Research, Elsevier, vol. 312(2), pages 456-472.
    3. Gaul, Daniela & Klamroth, Kathrin & Stiglmayr, Michael, 2022. "Event-based MILP models for ridepooling applications," European Journal of Operational Research, Elsevier, vol. 301(3), pages 1048-1063.
    4. Schulz, Arne & Pfeiffer, Christian, 2024. "A Branch-and-Cut algorithm for the dial-a-ride problem with incompatible customer types," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 181(C).
    5. Johnsen, Lennart C. & Meisel, Frank, 2022. "Interrelated trips in the rural dial-a-ride problem with autonomous vehicles," European Journal of Operational Research, Elsevier, vol. 303(1), pages 201-219.
    6. Zhang, Li & Liu, Zhongshan & Yu, Bin & Long, Jiancheng, 2024. "A ridesharing routing problem for airport riders with electric vehicles," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 184(C).
    7. Rahman, Md Hishamur & Chen, Shijie & Sun, Yanshuo & Siddiqui, Muhammad Imran Younus & Mohebbi, Matthew & Marković, Nikola, 2023. "Integrating dial-a-ride with transportation network companies for cost efficiency: A Maryland case study," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 175(C).
    8. Lei, Chao & Ouyang, Yanfeng, 2024. "Average minimum distance to visit a subset of random points in a compact region," Transportation Research Part B: Methodological, Elsevier, vol. 181(C).
    9. Daganzo, Carlos F. & Ouyang, Yanfeng & Yang, Haolin, 2020. "Analysis of ride-sharing with service time and detour guarantees," Transportation Research Part B: Methodological, Elsevier, vol. 140(C), pages 130-150.
    10. Li, Yuanyuan & Liu, Yang, 2021. "Optimizing flexible one-to-two matching in ride-hailing systems with boundedly rational users," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 150(C).
    11. Yining Liu & Yanfeng Ouyang, 2022. "Planning ride-pooling services with detour restrictions for spatially heterogeneous demand: A multi-zone queuing network approach," Papers 2208.02219, arXiv.org, revised Jun 2023.
    12. Ke, Jintao & Yang, Hai & Zheng, Zhengfei, 2020. "On ride-pooling and traffic congestion," Transportation Research Part B: Methodological, Elsevier, vol. 142(C), pages 213-231.
    13. Wang, Yuan & Lei, Linfei & Zhang, Dongxiang & Lee, Loo Hay, 2020. "Towards delivery-as-a-service: Effective neighborhood search strategies for integrated delivery optimization of E-commerce and static O2O parcels," Transportation Research Part B: Methodological, Elsevier, vol. 139(C), pages 38-63.
    14. Guo, Xiaotong & Caros, Nicholas S. & Zhao, Jinhua, 2021. "Robust matching-integrated vehicle rebalancing in ride-hailing system with uncertain demand," Transportation Research Part B: Methodological, Elsevier, vol. 150(C), pages 161-189.
    15. Su, Yue & Dupin, Nicolas & Puchinger, Jakob, 2023. "A deterministic annealing local search for the electric autonomous dial-a-ride problem," European Journal of Operational Research, Elsevier, vol. 309(3), pages 1091-1111.
    16. Ke, Jintao & Yang, Hai & Li, Xinwei & Wang, Hai & Ye, Jieping, 2020. "Pricing and equilibrium in on-demand ride-pooling markets," Transportation Research Part B: Methodological, Elsevier, vol. 139(C), pages 411-431.
    17. Liu, Yining & Ouyang, Yanfeng, 2023. "Planning ride-pooling services with detour restrictions for spatially heterogeneous demand: A multi-zone queuing network approach," Transportation Research Part B: Methodological, Elsevier, vol. 174(C).
    18. Zhang, Ruolin & Masoud, Neda, 2021. "A distributed algorithm for operating large-scale ridesourcing systems," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 156(C).
    19. Ho, Sin C. & Szeto, W.Y. & Kuo, Yong-Hong & Leung, Janny M.Y. & Petering, Matthew & Tou, Terence W.H., 2018. "A survey of dial-a-ride problems: Literature review and recent developments," Transportation Research Part B: Methodological, Elsevier, vol. 111(C), pages 395-421.
    20. Lee, Enoch & Cen, Xuekai & Lo, Hong K., 2021. "Zonal-based flexible bus service under elastic stochastic demand," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 152(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:transe:v:186:y:2024:i:c:s136655452400142x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/600244/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.