IDEAS home Printed from https://ideas.repec.org/a/zbw/espost/230635.html
   My bibliography  Save this article

The complexities of agent-based modeling output analysis

Author

Listed:
  • Lee, Ju-Sung
  • Filatova, Tatiana
  • Ligmann-Zielinska, Arika
  • Hassani-Mahmooei, Behrooz
  • Stonedahl, Forrest
  • Lorscheid, Iris
  • Voinov, Alexey
  • Polhill, J. Gareth
  • Sun, Zhanli
  • Parker, Dawn C.

Abstract

The proliferation of agent-based models (ABMs) in recent decades has motivated model practitioners to improve the transparency, replicability, and trust in results derived from ABMs. The complexity of ABMs has risen in stride with advances in computing power and resources, resulting in larger models with complex interactions and learning and whose outputs are often high-dimensional and require sophisticated analytical approaches. Similarly, the increasing use of data and dynamics in ABMs has further enhanced the complexity of their outputs. In this article, we offer an overview of the state-of-the-art approaches in analyzing and reporting ABM outputs highlighting challenges and outstanding issues. In particular, we examine issues surrounding variance stability (in connection with determination of appropriate number of runs and hypothesis testing), sensitivity analysis, spatio-temporal analysis, visualization, and effective communication of all these to non-technical audiences, such as various stakeholders.

Suggested Citation

  • Lee, Ju-Sung & Filatova, Tatiana & Ligmann-Zielinska, Arika & Hassani-Mahmooei, Behrooz & Stonedahl, Forrest & Lorscheid, Iris & Voinov, Alexey & Polhill, J. Gareth & Sun, Zhanli & Parker, Dawn C., 2015. "The complexities of agent-based modeling output analysis," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics.
  • Handle: RePEc:zbw:espost:230635
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/230635/1/Lee_2015_agent_based_modeling.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Wolfgang Radax & Bernhard Rengs, 2010. "Prospects and Pitfalls of Statistical Testing: Insights from Replicating the Demographic Prisoner's Dilemma," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 13(4), pages 1-1.
    2. Matteo Richiardi & Roberto Leombruni & Nicole J. Saam & Michele Sonnessa, 2006. "A Common Protocol for Agent-Based Social Simulation," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 9(1), pages 1-15.
    3. Chang, Myong-Hun & Harrington, Joseph Jr., 2006. "Agent-Based Models of Organizations," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 26, pages 1273-1337, Elsevier.
    4. Hassani-Mahmooei, Behrooz & Parris, Brett W., 2013. "Resource scarcity, effort allocation and environmental security: An agent-based theoretical approach," Economic Modelling, Elsevier, vol. 30(C), pages 183-192.
    5. Wenwu Tang & Meijuan Jia, 2014. "Global Sensitivity Analysis of a Large Agent-Based Model of Spatial Opinion Exchange: A Heterogeneous Multi-GPU Acceleration Approach," Annals of the American Association of Geographers, Taylor & Francis Journals, vol. 104(3), pages 485-509, May.
    6. Jan C. Thiele & Winfried Kurth & Volker Grimm, 2014. "Facilitating Parameter Estimation and Sensitivity Analysis of Agent-Based Models: A Cookbook Using NetLogo and 'R'," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 17(3), pages 1-11.
    7. Stephan Leitner & Friederike Wall (ed.), 2014. "Artificial Economics and Self Organization," Lecture Notes in Economics and Mathematical Systems, Springer, edition 127, number 978-3-319-00912-4, October.
    8. Stanislaw Raczynski, 2004. "Simulation of The Dynamic Interactions Between Terror and Anti-Terror Organizational Structures," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 7(2), pages 1-8.
    9. Lulseged Tamene & Quang Le & Paul Vlek, 2014. "A Landscape Planning and Management Tool for Land and Water Resources Management: An Example Application in Northern Ethiopia," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 28(2), pages 407-424, January.
    10. Marco A. Janssen, 2009. "Understanding Artificial Anasazi," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 12(4), pages 1-13.
    11. Fonoberova, Maria & Fonoberov, Vladimir A. & Mezić, Igor, 2013. "Global sensitivity/uncertainty analysis for agent-based models," Reliability Engineering and System Safety, Elsevier, vol. 118(C), pages 8-17.
    12. Borgonovo, E., 2007. "A new uncertainty importance measure," Reliability Engineering and System Safety, Elsevier, vol. 92(6), pages 771-784.
    13. Robert Pontius & Wideke Boersma & Jean-Christophe Castella & Keith Clarke & Ton Nijs & Charles Dietzel & Zengqiang Duan & Eric Fotsing & Noah Goldstein & Kasper Kok & Eric Koomen & Christopher Lippitt, 2008. "Comparing the input, output, and validation maps for several models of land change," The Annals of Regional Science, Springer;Western Regional Science Association, vol. 42(1), pages 11-37, March.
    14. Filatova, Tatiana & Parker, Dawn Cassandra & van der Veen, Anne, 2011. "The Implications of Skewed Risk Perception for a Dutch Coastal Land Market: Insights from an Agent-Based Computational Economics Model," Agricultural and Resource Economics Review, Northeastern Agricultural and Resource Economics Association, vol. 40(3), pages 1-19, December.
    15. Amineh Ghorbani & Pieter Bots & Virginia Dignum & Gerard Dijkema, 2013. "MAIA: a Framework for Developing Agent-Based Social Simulations," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 16(2), pages 1-9.
    16. Klaus G. Troitzsch, 2014. "Simulation Experiments and Significance Tests," Lecture Notes in Economics and Mathematical Systems, in: Stephan Leitner & Friederike Wall (ed.), Artificial Economics and Self Organization, edition 127, pages 17-28, Springer.
    17. Manel Baucells & Emanuele Borgonovo, 2013. "Invariant Probabilistic Sensitivity Analysis," Management Science, INFORMS, vol. 59(11), pages 2536-2549, November.
    18. Alan P. Kirman, 1992. "Whom or What Does the Representative Individual Represent?," Journal of Economic Perspectives, American Economic Association, vol. 6(2), pages 117-136, Spring.
    19. Riccardo Boero & Giangiacomo Bravo & Marco Castellani & Flaminio Squazzoni, 2010. "Why Bother with What Others Tell You? An Experimental Data-Driven Agent-Based Model," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 13(3), pages 1-6.
    20. Dirk Helbing & Lubos Buzna & Anders Johansson & Torsten Werner, 2005. "Self-Organized Pedestrian Crowd Dynamics: Experiments, Simulations, and Design Solutions," Transportation Science, INFORMS, vol. 39(1), pages 1-24, February.
    21. Simon Angus & Behrooz Hassani-Mahmooei, 2015. ""Anarchy" Reigns: A Quantitative Analysis of Agent-Based Modelling Publication Practices in JASSS, 2001-2012," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 18(4), pages 1-16.
    22. Schelling, Thomas C, 1969. "Models of Segregation," American Economic Review, American Economic Association, vol. 59(2), pages 488-493, May.
    23. Cara H. Kahl & Hans Hansen, 2015. "Simulating Creativity from a Systems Perspective: CRESY," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 18(1), pages 1-4.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Friederike Wall, 2016. "Agent-based modeling in managerial science: an illustrative survey and study," Review of Managerial Science, Springer, vol. 10(1), pages 135-193, January.
    2. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    3. Flaminio Squazzoni, 2010. "The impact of agent-based models in the social sciences after 15 years of incursions," History of Economic Ideas, Fabrizio Serra Editore, Pisa - Roma, vol. 18(2), pages 197-234.
    4. Emanuele Borgonovo & Marco Pangallo & Jan Rivkin & Leonardo Rizzo & Nicolaj Siggelkow, 2022. "Sensitivity analysis of agent-based models: a new protocol," Computational and Mathematical Organization Theory, Springer, vol. 28(1), pages 52-94, March.
    5. Stephan Leitner & Friederike Wall, 2015. "Simulation-based research in management accounting and control: an illustrative overview," Journal of Management Control: Zeitschrift für Planung und Unternehmenssteuerung, Springer, vol. 26(2), pages 105-129, August.
    6. S. Cucurachi & E. Borgonovo & R. Heijungs, 2016. "A Protocol for the Global Sensitivity Analysis of Impact Assessment Models in Life Cycle Assessment," Risk Analysis, John Wiley & Sons, vol. 36(2), pages 357-377, February.
    7. Joshua M. Epstein, 2007. "Agent-Based Computational Models and Generative Social Science," Introductory Chapters, in: Generative Social Science Studies in Agent-Based Computational Modeling, Princeton University Press.
    8. Matteo Coronese & Davide Luzzati, 2022. "Economic impacts of natural hazards and complexity science: a critical review," LEM Papers Series 2022/13, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    9. Tianyang Wang & James S. Dyer & Warren J. Hahn, 2017. "Sensitivity analysis of decision making under dependent uncertainties using copulas," EURO Journal on Decision Processes, Springer;EURO - The Association of European Operational Research Societies, vol. 5(1), pages 117-139, November.
    10. Stolzenburg, Ulrich, 2015. "The agent-based Solow growth model with endogenous business cycles," Economics Working Papers 2015-01, Christian-Albrechts-University of Kiel, Department of Economics.
    11. Katarzyna Ostasiewicz & Michal H. Tyc & Piotr Goliczewski & Piotr Magnuszewski & Andrzej Radosz & Jan Sendzimir, 2006. "Integrating economic and psychological insights in binary choice models with social interactions," Papers physics/0609170, arXiv.org.
    12. Robert Marks, 2007. "Validating Simulation Models: A General Framework and Four Applied Examples," Computational Economics, Springer;Society for Computational Economics, vol. 30(3), pages 265-290, October.
    13. Paola Tubaro, 2011. "Computational Economics," Chapters, in: John B. Davis & D. Wade Hands (ed.), The Elgar Companion to Recent Economic Methodology, chapter 10, Edward Elgar Publishing.
    14. Liu, Xing & Ferrario, Elisa & Zio, Enrico, 2019. "Identifying resilient-important elements in interdependent critical infrastructures by sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 189(C), pages 423-434.
    15. Lengnick, Matthias, 2013. "Agent-based macroeconomics: A baseline model," Journal of Economic Behavior & Organization, Elsevier, vol. 86(C), pages 102-120.
    16. Luis R. Izquierdo & Segismundo S. Izquierdo & José Manuel Galán & José Ignacio Santos, 2009. "Techniques to Understand Computer Simulations: Markov Chain Analysis," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 12(1), pages 1-6.
    17. Di Maio, Francesco & Nicola, Giancarlo & Borgonovo, Emanuele & Zio, Enrico, 2016. "Invariant methods for an ensemble-based sensitivity analysis of a passive containment cooling system of an AP1000 nuclear power plant," Reliability Engineering and System Safety, Elsevier, vol. 151(C), pages 12-19.
    18. Jovan Žamac & Daniel Hallberg & Thomas Lindh, 2010. "Low Fertility and Long-Run Growth in an Economy with a Large Public Sector [Fécondité basse et croissance à long terme dans une économie à secteur public très développé]," European Journal of Population, Springer;European Association for Population Studies, vol. 26(2), pages 183-205, May.
    19. Marcel Ausloos & Herbert Dawid & Ugo Merlone, 2015. "Spatial Interactions in Agent-Based Modeling," Dynamic Modeling and Econometrics in Economics and Finance, in: Pasquale Commendatore & Saime Kayam & Ingrid Kubin (ed.), Complexity and Geographical Economics, edition 127, pages 353-377, Springer.
    20. Davide Secchi & Raffaello Seri, 2017. "Controlling for false negatives in agent-based models: a review of power analysis in organizational research," Computational and Mathematical Organization Theory, Springer, vol. 23(1), pages 94-121, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:espost:230635. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/zbwkide.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/zbwkide.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.