IDEAS home Printed from https://ideas.repec.org/a/eee/chsofr/v191y2025ics0960077924014760.html
   My bibliography  Save this article

Unbiased evacuations processes using a reinforcement learning approach

Author

Listed:
  • Encina, Nikolas N.
  • Carrasco, Sebastian C.
  • Ramirez, Max
  • Rogan, José
  • Valdivia, Juan Alejandro

Abstract

Simulations of collective phenomena require the modeling of individual choices. In evacuations, handpicked policies may produce biased results. Here, we remove that bias using reinforcement learning. This technique allows for the construction of non-optimal solutions of each agent’s trajectory but improves the performance of the whole ensemble. Our analysis reveals that evacuation time can decrease up to 12% compared to the strategy of following the shortest path, which is a more standard approach. Our simulations also show that the reinforcement algorithm causes the agents to distribute themselves in a more homogeneous way while advancing towards the exits, resulting in fewer collisions. Moreover, we found, as expected, that collisions and evacuation time are strongly correlated and discovered that such a relationship is policy-independent. Our work opens up new research venues to study evacuations and leverages the potentiality of new machine-learning techniques to study collective phenomena.

Suggested Citation

  • Encina, Nikolas N. & Carrasco, Sebastian C. & Ramirez, Max & Rogan, José & Valdivia, Juan Alejandro, 2025. "Unbiased evacuations processes using a reinforcement learning approach," Chaos, Solitons & Fractals, Elsevier, vol. 191(C).
  • Handle: RePEc:eee:chsofr:v:191:y:2025:i:c:s0960077924014760
    DOI: 10.1016/j.chaos.2024.115924
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0960077924014760
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.chaos.2024.115924?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Li, Yang & Chen, Maoyin & Dou, Zhan & Zheng, Xiaoping & Cheng, Yuan & Mebarki, Ahmed, 2019. "A review of cellular automata models for crowd evacuation," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 526(C).
    2. So, Stella K. & Daganzo, Carlos F., 2010. "Managing evacuation routes," Transportation Research Part B: Methodological, Elsevier, vol. 44(4), pages 514-520, May.
    3. Erica Chenoweth & Margherita Belgioioso, 2019. "The physics of dissent and the effects of movement momentum," Nature Human Behaviour, Nature, vol. 3(10), pages 1088-1095, October.
    4. Roberto A. Weber, 2006. "Managing Growth to Achieve Efficient Coordination in Large Groups," American Economic Review, American Economic Association, vol. 96(1), pages 114-126, March.
    5. Richard Bellman, 1954. "Some Applications of the Theory of Dynamic Programming---A Review," Operations Research, INFORMS, vol. 2(3), pages 275-288, August.
    6. Kaifeng Deng & Meng Li & Guanning Wang & Xiangmin Hu & Yan Zhang & Huijie Zheng & Koukou Tian & Tao Chen, 2022. "Experimental Study on Panic during Simulated Fire Evacuation Using Psycho- and Physiological Metrics," IJERPH, MDPI, vol. 19(11), pages 1-18, June.
    7. Varas, A. & Cornejo, M.D. & Mainemer, D. & Toledo, B. & Rogan, J. & Muñoz, V. & Valdivia, J.A., 2007. "Cellular automaton model for evacuation process with obstacles," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 382(2), pages 631-642.
    8. Bramoulle, Yann, 2007. "Anti-coordination and social interactions," Games and Economic Behavior, Elsevier, vol. 58(1), pages 30-49, January.
    9. Ellison, Glenn, 1993. "Learning, Local Interaction, and Coordination," Econometrica, Econometric Society, vol. 61(5), pages 1047-1071, September.
    10. Michael E Roberts & Robert L Goldstone, 2011. "Adaptive Group Coordination and Role Differentiation," PLOS ONE, Public Library of Science, vol. 6(7), pages 1-8, July.
    11. Richard Bellman, 1954. "On some applications of the theory of dynamic programming to logistics," Naval Research Logistics Quarterly, John Wiley & Sons, vol. 1(2), pages 141-153, June.
    12. Ramírez, M. & Torres, F. & Toledo, B.A. & Coello, M. & Correa-Burrows, P. & Rogan, J. & Valdivia, J.A., 2019. "Unpredictability in pedestrian flow: The impact of stochasticity and anxiety in the event of an emergency," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 531(C).
    13. Burstedde, C & Klauck, K & Schadschneider, A & Zittartz, J, 2001. "Simulation of pedestrian dynamics using a two-dimensional cellular automaton," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 295(3), pages 507-525.
    14. Lin Zhang & Jian Lu & Bai-bai Fu & Shu-bin Li, 2018. "A Review and Prospect for the Complexity and Resilience of Urban Public Transit Network Based on Complex Network Theory," Complexity, Hindawi, vol. 2018, pages 1-36, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhang, Wenke & Zhang, Zhichao & Ma, Yueyao & Lee, Eric Wai Ming & Shi, Meng, 2024. "Psychological impatience in pedestrian evacuation: modelling, simulations and experiments," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 635(C).
    2. Miyagawa, Daiki & Ichinose, Genki, 2020. "Cellular automaton model with turning behavior in crowd evacuation," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 549(C).
    3. Xiaoyue Li & John M. Mulvey, 2023. "Optimal Portfolio Execution in a Regime-switching Market with Non-linear Impact Costs: Combining Dynamic Program and Neural Network," Papers 2306.08809, arXiv.org.
    4. Mahmoud Mahfouz & Angelos Filos & Cyrine Chtourou & Joshua Lockhart & Samuel Assefa & Manuela Veloso & Danilo Mandic & Tucker Balch, 2019. "On the Importance of Opponent Modeling in Auction Markets," Papers 1911.12816, arXiv.org.
    5. Sun, Lishan & Yuan, Guang & Yao, Liya & Cui, Li & Kong, Dewen, 2021. "Study on strategies for alighting and boarding in subway stations," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 583(C).
    6. Chen, Changkun & Sun, Huakai & Lei, Peng & Zhao, Dongyue & Shi, Congling, 2021. "An extended model for crowd evacuation considering pedestrian panic in artificial attack," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 571(C).
    7. Huo, Feizhou & Li, Chao & Li, Yufei & Lv, Wei & Ma, Yaping, 2022. "An extended model for describing pedestrian evacuation considering the impact of obstacles on the visual view," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 604(C).
    8. Yue, Hao & Zhang, Junyao & Chen, Wenxin & Wu, Xinsen & Zhang, Xu & Shao, Chunfu, 2021. "Simulation of the influence of spatial obstacles on evacuation pedestrian flow in walking facilities," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 571(C).
    9. Boute, Robert N. & Gijsbrechts, Joren & van Jaarsveld, Willem & Vanvuchelen, Nathalie, 2022. "Deep reinforcement learning for inventory control: A roadmap," European Journal of Operational Research, Elsevier, vol. 298(2), pages 401-412.
    10. Dawei Chen & Fangxu Mo & Ye Chen & Jun Zhang & Xinyu You, 2022. "Optimization of Ramp Locations along Freeways: A Dynamic Programming Approach," Sustainability, MDPI, vol. 14(15), pages 1-13, August.
    11. Alós-Ferrer, Carlos & Weidenholzer, Simon, 2014. "Imitation and the role of information in overcoming coordination failures," Games and Economic Behavior, Elsevier, vol. 87(C), pages 397-411.
    12. Liu, Zhichen & Li, Ying & Zhang, Zhaoyi & Yu, Wenbo, 2022. "A new evacuation accessibility analysis approach based on spatial information," Reliability Engineering and System Safety, Elsevier, vol. 222(C).
    13. Huan-Huan, Tian & Li-Yun, Dong & Yu, Xue, 2015. "Influence of the exits’ configuration on evacuation process in a room without obstacle," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 420(C), pages 164-178.
    14. Liu, Jing & Jia, Yang & Mao, Tianlu & Wang, Zhaoqi, 2022. "Modeling and simulation analysis of crowd evacuation behavior under terrorist attack," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 604(C).
    15. Harrold, Daniel J.B. & Cao, Jun & Fan, Zhong, 2022. "Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning," Energy, Elsevier, vol. 238(PC).
    16. Vanvuchelen, Nathalie & De Boeck, Kim & Boute, Robert N., 2024. "Cluster-based lateral transshipments for the Zambian health supply chain," European Journal of Operational Research, Elsevier, vol. 313(1), pages 373-386.
    17. Wang, Jinhuan & Zhang, Lei & Shi, Qiongyu & Yang, Peng & Hu, Xiaoming, 2015. "Modeling and simulating for congestion pedestrian evacuation with panic," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 428(C), pages 396-409.
    18. Bartłomiej Kocot & Paweł Czarnul & Jerzy Proficz, 2023. "Energy-Aware Scheduling for High-Performance Computing Systems: A Survey," Energies, MDPI, vol. 16(2), pages 1-28, January.
    19. Wadi Khalid Anuar & Lai Soon Lee & Hsin-Vonn Seow & Stefan Pickl, 2021. "A Multi-Depot Vehicle Routing Problem with Stochastic Road Capacity and Reduced Two-Stage Stochastic Integer Linear Programming Models for Rollout Algorithm," Mathematics, MDPI, vol. 9(13), pages 1-44, July.
    20. Jackson, Matthew O. & Zenou, Yves, 2015. "Games on Networks," Handbook of Game Theory with Economic Applications,, Elsevier.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:chsofr:v:191:y:2025:i:c:s0960077924014760. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Thayer, Thomas R. (email available below). General contact details of provider: https://www.journals.elsevier.com/chaos-solitons-and-fractals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.