IDEAS home Printed from https://ideas.repec.org/a/jas/jasssj/2019-139-4.html
   My bibliography  Save this article

The Use of Surrogate Models to Analyse Agent-Based Models

Author

Listed:
  • Guus ten Broeke
  • George van Voorn
  • Arend Ligtenberg
  • Jaap Molenaar

Abstract

The utility of Agent Based Models (ABMs) for decision making support as well as for scientific applications can be increased considerably by the availability and use of methodologies for thorough model behaviour analysis. In view of their intrinsic construction, ABMs have to be analysed numerically. Furthermore, ABM behaviour is often complex, featuring strong non-linearities, tipping points, and adaptation. This easily leads to high computational costs, presenting a serious practical limitation. Model developers and users alike would benefit from methodologies that can explore large parts of parameter space at limited computational costs. In this paper we present a methodology that makes this possible. The essence of our approach is to develop a cost-effective surrogate model based on ABM output using machine learning to approximate ABM simulation data. The development consists of two steps, both with iterative loops of training and cross-validation. In the first part, a Support Vector Machine (SVM) is developed to split behaviour space into regions of qualitatively different behaviour. In the second part, a Support Vector Regression (SVR) is developed to cover the quantitative behaviour within these regions. Finally, sensitivity indices are calculated to rank the importance of parameters for describing the boundaries between regions, and for the quantitative dynamics within regions. The methodology is demonstrated in three case studies, a differential equation model of predator-prey interaction, a common-pool resource ABM and an ABM representing the Philippine tuna fishery. In all cases, the model and the corresponding surrogate model show a good match. Furthermore, different parameters are shown to influence the quantitative outcomes, compared to those that influence the underlying qualitative behaviour. Thus, the method helps to distinguish which parameters determine the boundaries in parameter space between regions that are separated by tipping points, or by any criterion of interest to the user.

Suggested Citation

  • Guus ten Broeke & George van Voorn & Arend Ligtenberg & Jaap Molenaar, 2021. "The Use of Surrogate Models to Analyse Agent-Based Models," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 24(2), pages 1-3.
  • Handle: RePEc:jas:jasssj:2019-139-4
    as

    Download full text from publisher

    File URL: https://www.jasss.org/24/2/3/3.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. repec:hal:spmain:info:hdl:2441/4pa18fd9lf9h59m4vfavfcf61e is not listed on IDEAS
    2. Sylvain Barde & Sander van der Hoog, 2017. "An empirical validation protocol for large-scale agent-based models," Studies in Economics 1712, School of Economics, University of Kent.
    3. Borgonovo, E., 2007. "A new uncertainty importance measure," Reliability Engineering and System Safety, Elsevier, vol. 92(6), pages 771-784.
    4. Arika Ligmann-Zielinska & Daniel B Kramer & Kendra Spence Cheruvelil & Patricia A Soranno, 2014. "Using Uncertainty and Sensitivity Analyses in Socioecological Agent-Based Models to Improve Their Analytical Performance and Policy Relevance," PLOS ONE, Public Library of Science, vol. 9(10), pages 1-13, October.
    5. repec:spo:wpmain:info:hdl:2441/4pa18fd9lf9h59m4vfavfcf61e is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mert Edali, 2022. "Pattern‐oriented analysis of system dynamics models via random forests," System Dynamics Review, System Dynamics Society, vol. 38(2), pages 135-166, April.
    2. Karl Naumann-Woleske & Max Sina Knicker & Michael Benzaquen & Jean-Philippe Bouchaud, 2022. "Exploration of the Parameter Space in Macroeconomic Models," Post-Print hal-03797418, HAL.
    3. Bernardo Alves Furtado & Gustavo Onofre Andre~ao, 2022. "Machine Learning Simulates Agent-Based Model Towards Policy," Papers 2203.02576, arXiv.org, revised Nov 2022.
    4. Karl Naumann-Woleske & Max Sina Knicker & Michael Benzaquen & Jean-Philippe Bouchaud, 2021. "Exploration of the Parameter Space in Macroeconomic Agent-Based Models," Papers 2111.08654, arXiv.org, revised Aug 2022.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Makam, Vaishno Devi & Millossovich, Pietro & Tsanakas, Andreas, 2021. "Sensitivity analysis with χ2-divergences," Insurance: Mathematics and Economics, Elsevier, vol. 100(C), pages 372-383.
    2. S. Cucurachi & E. Borgonovo & R. Heijungs, 2016. "A Protocol for the Global Sensitivity Analysis of Impact Assessment Models in Life Cycle Assessment," Risk Analysis, John Wiley & Sons, vol. 36(2), pages 357-377, February.
    3. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    4. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    5. Tianyang Wang & James S. Dyer & Warren J. Hahn, 2017. "Sensitivity analysis of decision making under dependent uncertainties using copulas," EURO Journal on Decision Processes, Springer;EURO - The Association of European Operational Research Societies, vol. 5(1), pages 117-139, November.
    6. Yun, Wanying & Lu, Zhenzhou & Feng, Kaixuan & Li, Luyi, 2019. "An elaborate algorithm for analyzing the Borgonovo moment-independent sensitivity by replacing the probability density function estimation with the probability estimation," Reliability Engineering and System Safety, Elsevier, vol. 189(C), pages 99-108.
    7. Puppo, L. & Pedroni, N. & Maio, F. Di & Bersano, A. & Bertani, C. & Zio, E., 2021. "A Framework based on Finite Mixture Models and Adaptive Kriging for Characterizing Non-Smooth and Multimodal Failure Regions in a Nuclear Passive Safety System," Reliability Engineering and System Safety, Elsevier, vol. 216(C).
    8. Chen, Shi-Shun & Li, Xiao-Yang, 2025. "Comparison of global sensitivity analysis methods for a fire spread model with a segmented characteristic," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 229(C), pages 304-318.
    9. Zhou Changcong & Ji Mengyao & Zhao Haodong & Cao Fei, 2021. "Uncertainty analysis of motion error for mechanisms and Kriging-based solutions," Journal of Risk and Reliability, , vol. 235(5), pages 731-743, October.
    10. Torsten Heinrich & Juan Sabuco & J. Doyne Farmer, 2022. "A simulation of the insurance industry: the problem of risk model homogeneity," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 17(2), pages 535-576, April.
    11. Wei, Pengfei & Lu, Zhenzhou & Yuan, Xiukai, 2013. "Monte Carlo simulation for moment-independent sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 110(C), pages 60-67.
    12. Wenxuan Wang & Hangshan Gao & Pengfei Wei & Changcong Zhou, 2017. "Extending first-passage method to reliability sensitivity analysis of motion mechanisms," Journal of Risk and Reliability, , vol. 231(5), pages 573-586, October.
    13. Zhai, Qingqing & Yang, Jun & Zhao, Yu, 2014. "Space-partition method for the variance-based sensitivity analysis: Optimal partition scheme and comparative study," Reliability Engineering and System Safety, Elsevier, vol. 131(C), pages 66-82.
    14. Ju-Sung Lee & Tatiana Filatova & Arika Ligmann-Zielinska & Behrooz Hassani-Mahmooei & Forrest Stonedahl & Iris Lorscheid & Alexey Voinov & J. Gareth Polhill & Zhanli Sun & Dawn C. Parker, 2015. "The Complexities of Agent-Based Modeling Output Analysis," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 18(4), pages 1-4.
    15. Wu, Qiong-Li & Cournède, Paul-Henry & Mathieu, Amélie, 2012. "An efficient computational method for global sensitivity analysis and its application to tree growth modelling," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 35-43.
    16. Zio, E. & Pedroni, N., 2012. "Monte Carlo simulation-based sensitivity analysis of the model of a thermal–hydraulic passive system," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 90-106.
    17. Li, Haihe & Wang, Pan & Huang, Xiaoyu & Zhang, Zheng & Zhou, Changcong & Yue, Zhufeng, 2021. "Vine copula-based parametric sensitivity analysis of failure probability-based importance measure in the presence of multidimensional dependencies," Reliability Engineering and System Safety, Elsevier, vol. 215(C).
    18. Song, Xiaodong & Bryan, Brett A. & Almeida, Auro C. & Paul, Keryn I. & Zhao, Gang & Ren, Yin, 2013. "Time-dependent sensitivity of a process-based ecological model," Ecological Modelling, Elsevier, vol. 265(C), pages 114-123.
    19. Ge, Qiao & Menendez, Monica, 2017. "Extending Morris method for qualitative global sensitivity analysis of models with dependent inputs," Reliability Engineering and System Safety, Elsevier, vol. 162(C), pages 28-39.
    20. Ziegler, Philipp & Wartzack, Sandro, 2015. "Sensitivity Analysis of features in tolerancing based on constraint function level sets," Reliability Engineering and System Safety, Elsevier, vol. 134(C), pages 324-333.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:jas:jasssj:2019-139-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Francesco Renzini (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.