IDEAS home Printed from https://ideas.repec.org/a/eee/chsofr/v191y2025ics0960077924014760.html
   My bibliography  Save this article

Unbiased evacuations processes using a reinforcement learning approach

Author

Listed:
  • Encina, Nikolas N.
  • Carrasco, Sebastian C.
  • Ramirez, Max
  • Rogan, José
  • Valdivia, Juan Alejandro

Abstract

Simulations of collective phenomena require the modeling of individual choices. In evacuations, handpicked policies may produce biased results. Here, we remove that bias using reinforcement learning. This technique allows for the construction of non-optimal solutions of each agent’s trajectory but improves the performance of the whole ensemble. Our analysis reveals that evacuation time can decrease up to 12% compared to the strategy of following the shortest path, which is a more standard approach. Our simulations also show that the reinforcement algorithm causes the agents to distribute themselves in a more homogeneous way while advancing towards the exits, resulting in fewer collisions. Moreover, we found, as expected, that collisions and evacuation time are strongly correlated and discovered that such a relationship is policy-independent. Our work opens up new research venues to study evacuations and leverages the potentiality of new machine-learning techniques to study collective phenomena.

Suggested Citation

  • Encina, Nikolas N. & Carrasco, Sebastian C. & Ramirez, Max & Rogan, José & Valdivia, Juan Alejandro, 2025. "Unbiased evacuations processes using a reinforcement learning approach," Chaos, Solitons & Fractals, Elsevier, vol. 191(C).
  • Handle: RePEc:eee:chsofr:v:191:y:2025:i:c:s0960077924014760
    DOI: 10.1016/j.chaos.2024.115924
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0960077924014760
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.chaos.2024.115924?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ellison, Glenn, 1993. "Learning, Local Interaction, and Coordination," Econometrica, Econometric Society, vol. 61(5), pages 1047-1071, September.
    2. Erica Chenoweth & Margherita Belgioioso, 2019. "The physics of dissent and the effects of movement momentum," Nature Human Behaviour, Nature, vol. 3(10), pages 1088-1095, October.
    3. Ramírez, M. & Torres, F. & Toledo, B.A. & Coello, M. & Correa-Burrows, P. & Rogan, J. & Valdivia, J.A., 2019. "Unpredictability in pedestrian flow: The impact of stochasticity and anxiety in the event of an emergency," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 531(C).
    4. Roberto A. Weber, 2006. "Managing Growth to Achieve Efficient Coordination in Large Groups," American Economic Review, American Economic Association, vol. 96(1), pages 114-126, March.
    5. Richard Bellman, 1954. "Some Applications of the Theory of Dynamic Programming---A Review," Operations Research, INFORMS, vol. 2(3), pages 275-288, August.
    6. Li, Yang & Chen, Maoyin & Dou, Zhan & Zheng, Xiaoping & Cheng, Yuan & Mebarki, Ahmed, 2019. "A review of cellular automata models for crowd evacuation," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 526(C).
    7. So, Stella K. & Daganzo, Carlos F., 2010. "Managing evacuation routes," Transportation Research Part B: Methodological, Elsevier, vol. 44(4), pages 514-520, May.
    8. Burstedde, C & Klauck, K & Schadschneider, A & Zittartz, J, 2001. "Simulation of pedestrian dynamics using a two-dimensional cellular automaton," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 295(3), pages 507-525.
    9. Lin Zhang & Jian Lu & Bai-bai Fu & Shu-bin Li, 2018. "A Review and Prospect for the Complexity and Resilience of Urban Public Transit Network Based on Complex Network Theory," Complexity, Hindawi, vol. 2018, pages 1-36, December.
    10. Michael E Roberts & Robert L Goldstone, 2011. "Adaptive Group Coordination and Role Differentiation," PLOS ONE, Public Library of Science, vol. 6(7), pages 1-8, July.
    11. Kaifeng Deng & Meng Li & Guanning Wang & Xiangmin Hu & Yan Zhang & Huijie Zheng & Koukou Tian & Tao Chen, 2022. "Experimental Study on Panic during Simulated Fire Evacuation Using Psycho- and Physiological Metrics," IJERPH, MDPI, vol. 19(11), pages 1-18, June.
    12. Richard Bellman, 1954. "On some applications of the theory of dynamic programming to logistics," Naval Research Logistics Quarterly, John Wiley & Sons, vol. 1(2), pages 141-153, June.
    13. Varas, A. & Cornejo, M.D. & Mainemer, D. & Toledo, B. & Rogan, J. & Muñoz, V. & Valdivia, J.A., 2007. "Cellular automaton model for evacuation process with obstacles," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 382(2), pages 631-642.
    14. Bramoulle, Yann, 2007. "Anti-coordination and social interactions," Games and Economic Behavior, Elsevier, vol. 58(1), pages 30-49, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhang, Wenke & Zhang, Zhichao & Ma, Yueyao & Lee, Eric Wai Ming & Shi, Meng, 2024. "Psychological impatience in pedestrian evacuation: modelling, simulations and experiments," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 635(C).
    2. Miyagawa, Daiki & Ichinose, Genki, 2020. "Cellular automaton model with turning behavior in crowd evacuation," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 549(C).
    3. Yuan, Weifeng & Tan, Kang Hai, 2011. "A model for simulation of crowd behaviour in the evacuation from a smoke-filled compartment," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 390(23), pages 4210-4218.
    4. Thomas L. Magnanti, 2021. "Optimization: From Its Inception," Management Science, INFORMS, vol. 67(9), pages 5349-5363, September.
    5. Xiaoyue Li & John M. Mulvey, 2023. "Optimal Portfolio Execution in a Regime-switching Market with Non-linear Impact Costs: Combining Dynamic Program and Neural Network," Papers 2306.08809, arXiv.org.
    6. Mahmoud Mahfouz & Angelos Filos & Cyrine Chtourou & Joshua Lockhart & Samuel Assefa & Manuela Veloso & Danilo Mandic & Tucker Balch, 2019. "On the Importance of Opponent Modeling in Auction Markets," Papers 1911.12816, arXiv.org.
    7. Tianran Han & Jianming Zhao & Wenquan Li, 2020. "Smart-Guided Pedestrian Emergency Evacuation in Slender-Shape Infrastructure with Digital Twin Simulations," Sustainability, MDPI, vol. 12(22), pages 1-18, November.
    8. Sun, Lishan & Yuan, Guang & Yao, Liya & Cui, Li & Kong, Dewen, 2021. "Study on strategies for alighting and boarding in subway stations," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 583(C).
    9. Chen, Changkun & Sun, Huakai & Lei, Peng & Zhao, Dongyue & Shi, Congling, 2021. "An extended model for crowd evacuation considering pedestrian panic in artificial attack," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 571(C).
    10. Huo, Feizhou & Li, Chao & Li, Yufei & Lv, Wei & Ma, Yaping, 2022. "An extended model for describing pedestrian evacuation considering the impact of obstacles on the visual view," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 604(C).
    11. G., Mauricio Contreras & Peña, Juan Pablo, 2019. "The quantum dark side of the optimal control theory," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 515(C), pages 450-473.
    12. Mizutani, Daijiro & Nakazato, Yuto & Ikushima, Rie & Satsukawa, Koki & Kawasaki, Yosuke & Kuwahara, Masao, 2024. "Optimal intervention policy of emergency storage batteries for expressway transportation systems considering deterioration risk during lead time of replacement," Reliability Engineering and System Safety, Elsevier, vol. 242(C).
    13. Yann Disser & John Fearnley & Martin Gairing & Oliver Göbel & Max Klimm & Daniel Schmand & Alexander Skopalik & Andreas Tönnis, 2020. "Hiring Secretaries over Time: The Benefit of Concurrent Employment," Mathematics of Operations Research, INFORMS, vol. 45(1), pages 323-352, February.
    14. Sean Williams & Michael Short & Tracey Crosbie & Maryam Shadman-Pajouh, 2020. "A Decentralized Informatics, Optimization, and Control Framework for Evolving Demand Response Services," Energies, MDPI, vol. 13(16), pages 1-30, August.
    15. Yue, Hao & Zhang, Junyao & Chen, Wenxin & Wu, Xinsen & Zhang, Xu & Shao, Chunfu, 2021. "Simulation of the influence of spatial obstacles on evacuation pedestrian flow in walking facilities," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 571(C).
    16. Boute, Robert N. & Gijsbrechts, Joren & van Jaarsveld, Willem & Vanvuchelen, Nathalie, 2022. "Deep reinforcement learning for inventory control: A roadmap," European Journal of Operational Research, Elsevier, vol. 298(2), pages 401-412.
    17. Guo, Xiwei & Chen, Jianqiao & Zheng, Yaochen & Wei, Junhong, 2012. "A heterogeneous lattice gas model for simulating pedestrian evacuation," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 391(3), pages 582-592.
    18. Dawei Chen & Fangxu Mo & Ye Chen & Jun Zhang & Xinyu You, 2022. "Optimization of Ramp Locations along Freeways: A Dynamic Programming Approach," Sustainability, MDPI, vol. 14(15), pages 1-13, August.
    19. Alós-Ferrer, Carlos & Weidenholzer, Simon, 2014. "Imitation and the role of information in overcoming coordination failures," Games and Economic Behavior, Elsevier, vol. 87(C), pages 397-411.
    20. Gunaseelan Mani & Arul Joseph Gnanaprakasam & Liliana Guran & Reny George & Zoran D. Mitrović, 2023. "Some Results in Fuzzy b -Metric Space with b -Triangular Property and Applications to Fredholm Integral Equations and Dynamic Programming," Mathematics, MDPI, vol. 11(19), pages 1-17, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:chsofr:v:191:y:2025:i:c:s0960077924014760. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Thayer, Thomas R. (email available below). General contact details of provider: https://www.journals.elsevier.com/chaos-solitons-and-fractals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.