IDEAS home Printed from https://ideas.repec.org/a/eee/chsofr/v191y2025ics0960077924014760.html
   My bibliography  Save this article

Unbiased evacuations processes using a reinforcement learning approach

Author

Listed:
  • Encina, Nikolas N.
  • Carrasco, Sebastian C.
  • Ramirez, Max
  • Rogan, José
  • Valdivia, Juan Alejandro

Abstract

Simulations of collective phenomena require the modeling of individual choices. In evacuations, handpicked policies may produce biased results. Here, we remove that bias using reinforcement learning. This technique allows for the construction of non-optimal solutions of each agent’s trajectory but improves the performance of the whole ensemble. Our analysis reveals that evacuation time can decrease up to 12% compared to the strategy of following the shortest path, which is a more standard approach. Our simulations also show that the reinforcement algorithm causes the agents to distribute themselves in a more homogeneous way while advancing towards the exits, resulting in fewer collisions. Moreover, we found, as expected, that collisions and evacuation time are strongly correlated and discovered that such a relationship is policy-independent. Our work opens up new research venues to study evacuations and leverages the potentiality of new machine-learning techniques to study collective phenomena.

Suggested Citation

  • Encina, Nikolas N. & Carrasco, Sebastian C. & Ramirez, Max & Rogan, José & Valdivia, Juan Alejandro, 2025. "Unbiased evacuations processes using a reinforcement learning approach," Chaos, Solitons & Fractals, Elsevier, vol. 191(C).
  • Handle: RePEc:eee:chsofr:v:191:y:2025:i:c:s0960077924014760
    DOI: 10.1016/j.chaos.2024.115924
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0960077924014760
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.chaos.2024.115924?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Li, Yang & Chen, Maoyin & Dou, Zhan & Zheng, Xiaoping & Cheng, Yuan & Mebarki, Ahmed, 2019. "A review of cellular automata models for crowd evacuation," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 526(C).
    2. So, Stella K. & Daganzo, Carlos F., 2010. "Managing evacuation routes," Transportation Research Part B: Methodological, Elsevier, vol. 44(4), pages 514-520, May.
    3. Erica Chenoweth & Margherita Belgioioso, 2019. "The physics of dissent and the effects of movement momentum," Nature Human Behaviour, Nature, vol. 3(10), pages 1088-1095, October.
    4. Roberto A. Weber, 2006. "Managing Growth to Achieve Efficient Coordination in Large Groups," American Economic Review, American Economic Association, vol. 96(1), pages 114-126, March.
    5. Richard Bellman, 1954. "Some Applications of the Theory of Dynamic Programming---A Review," Operations Research, INFORMS, vol. 2(3), pages 275-288, August.
    6. Kaifeng Deng & Meng Li & Guanning Wang & Xiangmin Hu & Yan Zhang & Huijie Zheng & Koukou Tian & Tao Chen, 2022. "Experimental Study on Panic during Simulated Fire Evacuation Using Psycho- and Physiological Metrics," IJERPH, MDPI, vol. 19(11), pages 1-18, June.
    7. Varas, A. & Cornejo, M.D. & Mainemer, D. & Toledo, B. & Rogan, J. & Muñoz, V. & Valdivia, J.A., 2007. "Cellular automaton model for evacuation process with obstacles," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 382(2), pages 631-642.
    8. Bramoulle, Yann, 2007. "Anti-coordination and social interactions," Games and Economic Behavior, Elsevier, vol. 58(1), pages 30-49, January.
    9. Ellison, Glenn, 1993. "Learning, Local Interaction, and Coordination," Econometrica, Econometric Society, vol. 61(5), pages 1047-1071, September.
    10. Michael E Roberts & Robert L Goldstone, 2011. "Adaptive Group Coordination and Role Differentiation," PLOS ONE, Public Library of Science, vol. 6(7), pages 1-8, July.
    11. Richard Bellman, 1954. "On some applications of the theory of dynamic programming to logistics," Naval Research Logistics Quarterly, John Wiley & Sons, vol. 1(2), pages 141-153, June.
    12. Ramírez, M. & Torres, F. & Toledo, B.A. & Coello, M. & Correa-Burrows, P. & Rogan, J. & Valdivia, J.A., 2019. "Unpredictability in pedestrian flow: The impact of stochasticity and anxiety in the event of an emergency," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 531(C).
    13. Burstedde, C & Klauck, K & Schadschneider, A & Zittartz, J, 2001. "Simulation of pedestrian dynamics using a two-dimensional cellular automaton," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 295(3), pages 507-525.
    14. Lin Zhang & Jian Lu & Bai-bai Fu & Shu-bin Li, 2018. "A Review and Prospect for the Complexity and Resilience of Urban Public Transit Network Based on Complex Network Theory," Complexity, Hindawi, vol. 2018, pages 1-36, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhang, Wenke & Zhang, Zhichao & Ma, Yueyao & Lee, Eric Wai Ming & Shi, Meng, 2024. "Psychological impatience in pedestrian evacuation: modelling, simulations and experiments," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 635(C).
    2. Miyagawa, Daiki & Ichinose, Genki, 2020. "Cellular automaton model with turning behavior in crowd evacuation," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 549(C).
    3. Xiaoyue Li & John M. Mulvey, 2023. "Optimal Portfolio Execution in a Regime-switching Market with Non-linear Impact Costs: Combining Dynamic Program and Neural Network," Papers 2306.08809, arXiv.org.
    4. Mahmoud Mahfouz & Angelos Filos & Cyrine Chtourou & Joshua Lockhart & Samuel Assefa & Manuela Veloso & Danilo Mandic & Tucker Balch, 2019. "On the Importance of Opponent Modeling in Auction Markets," Papers 1911.12816, arXiv.org.
    5. Chen, Changkun & Sun, Huakai & Lei, Peng & Zhao, Dongyue & Shi, Congling, 2021. "An extended model for crowd evacuation considering pedestrian panic in artificial attack," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 571(C).
    6. Yue, Hao & Zhang, Junyao & Chen, Wenxin & Wu, Xinsen & Zhang, Xu & Shao, Chunfu, 2021. "Simulation of the influence of spatial obstacles on evacuation pedestrian flow in walking facilities," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 571(C).
    7. Dawei Chen & Fangxu Mo & Ye Chen & Jun Zhang & Xinyu You, 2022. "Optimization of Ramp Locations along Freeways: A Dynamic Programming Approach," Sustainability, MDPI, vol. 14(15), pages 1-13, August.
    8. Alós-Ferrer, Carlos & Weidenholzer, Simon, 2014. "Imitation and the role of information in overcoming coordination failures," Games and Economic Behavior, Elsevier, vol. 87(C), pages 397-411.
    9. Liu, Zhichen & Li, Ying & Zhang, Zhaoyi & Yu, Wenbo, 2022. "A new evacuation accessibility analysis approach based on spatial information," Reliability Engineering and System Safety, Elsevier, vol. 222(C).
    10. Liu, Jing & Jia, Yang & Mao, Tianlu & Wang, Zhaoqi, 2022. "Modeling and simulation analysis of crowd evacuation behavior under terrorist attack," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 604(C).
    11. Harrold, Daniel J.B. & Cao, Jun & Fan, Zhong, 2022. "Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning," Energy, Elsevier, vol. 238(PC).
    12. Vanvuchelen, Nathalie & De Boeck, Kim & Boute, Robert N., 2024. "Cluster-based lateral transshipments for the Zambian health supply chain," European Journal of Operational Research, Elsevier, vol. 313(1), pages 373-386.
    13. Wadi Khalid Anuar & Lai Soon Lee & Hsin-Vonn Seow & Stefan Pickl, 2021. "A Multi-Depot Vehicle Routing Problem with Stochastic Road Capacity and Reduced Two-Stage Stochastic Integer Linear Programming Models for Rollout Algorithm," Mathematics, MDPI, vol. 9(13), pages 1-44, July.
    14. Arno Riedl & Ingrid M. T. Rohde & Martin Strobel, 2021. "Free Neighborhood Choice Boosts Socially Optimal Outcomes in Stag-Hunt Coordination Problem," CESifo Working Paper Series 9012, CESifo.
    15. Hu, Xiangmin & Chen, Tao & Deng, Kaifeng & Wang, Guanning, 2023. "Effects of aggressiveness on pedestrian room evacuation using extended cellular automata model," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 619(C).
    16. Li, Jun & Fu, Siyao & He, Haibo & Jia, Hongfei & Li, Yanzhong & Guo, Yi, 2015. "Simulating large-scale pedestrian movement using CA and event driven model: Methodology and case study," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 437(C), pages 304-321.
    17. Li, Yang & Chen, Maoyin & Zheng, Xiaoping & Dou, Zhan & Cheng, Yuan, 2020. "Relationship between behavior aggressiveness and pedestrian dynamics using behavior-based cellular automata model," Applied Mathematics and Computation, Elsevier, vol. 371(C).
    18. Zheng, Xiaoping & Li, Wei & Guan, Chao, 2010. "Simulation of evacuation processes in a square with a partition wall using a cellular automaton model for pedestrian dynamics," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 389(11), pages 2177-2188.
    19. Matthias Breuer & David Windisch, 2019. "Investment Dynamics and Earnings‐Return Properties: A Structural Approach," Journal of Accounting Research, John Wiley & Sons, Ltd., vol. 57(3), pages 639-674, June.
    20. Li, Shengnan & Li, Xingang & Qu, Yunchao & Jia, Bin, 2015. "Block-based floor field model for pedestrian’s walking through corner," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 432(C), pages 337-353.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:chsofr:v:191:y:2025:i:c:s0960077924014760. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Thayer, Thomas R. (email available below). General contact details of provider: https://www.journals.elsevier.com/chaos-solitons-and-fractals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.