IDEAS home Printed from https://ideas.repec.org/a/eee/dyncon/v113y2020ics0165188920300294.html
   My bibliography  Save this article

A comparison of economic agent-based model calibration methods

Author

Listed:
  • Platt, Donovan

Abstract

Despite significant expansion in recent years, the literature on quantitative and data-driven approaches to economic agent-based model validation and calibration consists primarily of studies that have focused on the introduction of new calibration methods that are neither benchmarked against existing alternatives nor rigorously tested in terms of the quality of the estimates they produce. In response, we compare a number of prominent agent-based model calibration methods, both established and novel, through a series of computational experiments in an attempt to determine the respective strengths and weaknesses of each approach. Overall, we find that a simple, likelihood-based approach to Bayesian estimation consistently outperforms several members of the more popular class of simulated minimum distance methods and results in reasonable parameter estimates in many contexts, with a degradation in performance observed only when considering a large-scale model and attempting to fit a substantial number of its parameters.

Suggested Citation

  • Platt, Donovan, 2020. "A comparison of economic agent-based model calibration methods," Journal of Economic Dynamics and Control, Elsevier, vol. 113(C).
  • Handle: RePEc:eee:dyncon:v:113:y:2020:i:c:s0165188920300294
    DOI: 10.1016/j.jedc.2020.103859
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0165188920300294
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jedc.2020.103859?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Barde, Sylvain, 2016. "Direct comparison of agent-based models of herding in financial markets," Journal of Economic Dynamics and Control, Elsevier, vol. 73(C), pages 329-353.
    2. S. Alfarano & T. Lux & F. Wagner, 2007. "Empirical validation of stochastic models of interacting agents," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 55(2), pages 183-187, January.
    3. Brock, William A. & Hommes, Cars H., 1998. "Heterogeneous beliefs and routes to chaos in a simple asset pricing model," Journal of Economic Dynamics and Control, Elsevier, vol. 22(8-9), pages 1235-1274, August.
    4. Cincotti, Silvano & Raberto, Marco & Teglio, Andrea, 2010. "Credit money and macroeconomic instability in the agent-based model and simulator Eurace," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 4, pages 1-32.
    5. repec:hal:spmain:info:hdl:2441/4pa18fd9lf9h59m4vfavfcf61e is not listed on IDEAS
    6. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    7. Gilli, M. & Winker, P., 2003. "A global optimization heuristic for estimating agent based models," Computational Statistics & Data Analysis, Elsevier, vol. 42(3), pages 299-312, March.
    8. Platt, Donovan & Gebbie, Tim, 2018. "Can agent-based models probe market microstructure?," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 503(C), pages 1092-1106.
    9. Sandrine Jacob Leal & Mauro Napoletano & Andrea Roventini & Giorgio Fagiolo, 2016. "Rock around the clock: An agent-based model of low- and high-frequency trading," Journal of Evolutionary Economics, Springer, vol. 26(1), pages 49-76, March.
    10. Giorgio Fagiolo & Andrea Roventini, 2017. "Macroeconomic Policy in DSGE and Agent-Based Models Redux: New Developments and Challenges Ahead," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 20(1), pages 1-1.
    11. Recchioni, Maria Cristina & Tedeschi, Gabriele & Gallegati, Mauro, 2015. "A calibration procedure for analyzing stock price dynamics in an agent-based framework," Journal of Economic Dynamics and Control, Elsevier, vol. 60(C), pages 1-25.
    12. Sylvain Barde & Sander van Der Hoog, 2017. "An empirical validation protocol for large-scale agent-based models," Working Papers hal-03458672, HAL.
    13. Annalisa Fabretti, 2013. "On the problem of calibrating an agent based model for financial markets," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 8(2), pages 277-293, October.
    14. Sylvain Barde, 2017. "A Practical, Accurate, Information Criterion for Nth Order Markov Processes," Computational Economics, Springer;Society for Computational Economics, vol. 50(2), pages 281-324, August.
    15. repec:hal:spmain:info:hdl:2441/7kr9gv74ut9ngo58gia97t83i7 is not listed on IDEAS
    16. Lars Peter Hansen & James J. Heckman, 1996. "The Empirical Foundations of Calibration," Journal of Economic Perspectives, American Economic Association, vol. 10(1), pages 87-104, Winter.
    17. John Geanakoplos & Robert Axtell & J. Doyne Farmer & Peter Howitt & Benjamin Conlee & Jonathan Goldstein & Matthew Hendrey & Nathan M. Palmer & Chun-Yi Yang, 2012. "Getting at Systemic Risk via an Agent-Based Model of the Housing Market," American Economic Review, American Economic Association, vol. 102(3), pages 53-58, May.
    18. Isabelle Salle & Murat Yıldızoğlu, 2014. "Efficient Sampling and Meta-Modeling for Computational Economic Models," Computational Economics, Springer;Society for Computational Economics, vol. 44(4), pages 507-536, December.
    19. McFadden, Daniel, 1989. "A Method of Simulated Moments for Estimation of Discrete Response Models without Numerical Integration," Econometrica, Econometric Society, vol. 57(5), pages 995-1026, September.
    20. Alfarano, Simone & Lux, Thomas & Wagner, Friedrich, 2006. "Estimation of a simple agent-based model of financial markets: An application to Australian stock and foreign exchange data," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 370(1), pages 38-42.
    21. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    22. Carlo Bianchi & Pasquale Cirillo & Mauro Gallegati & Pietro Vagliasindi, 2007. "Validating and Calibrating Agent-Based Models: A Case Study," Computational Economics, Springer;Society for Computational Economics, vol. 30(3), pages 245-264, October.
    23. Grazzini, Jakob & Richiardi, Matteo, 2015. "Estimation of ergodic agent-based models by simulated minimum distance," Journal of Economic Dynamics and Control, Elsevier, vol. 51(C), pages 148-165.
    24. Francesco Lamperti, 2018. "Empirical validation of simulated models through the GSL-div: an illustrative application," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 13(1), pages 143-171, April.
    25. Grazzini, Jakob & Richiardi, Matteo G. & Tsionas, Mike, 2017. "Bayesian estimation of agent-based models," Journal of Economic Dynamics and Control, Elsevier, vol. 77(C), pages 26-47.
    26. Franke, Reiner & Westerhoff, Frank, 2012. "Structural stochastic volatility in asset pricing dynamics: Estimation and model contest," Journal of Economic Dynamics and Control, Elsevier, vol. 36(8), pages 1193-1211.
    27. Baptista, Rafa & Farmer, J. Doyne & Hinterschweiger, Marc & Low, Katie & Tang, Daniel & Uluc, Arzu, 2016. "Macroprudential policy in an agent-based model of the UK housing market," Bank of England working papers 619, Bank of England.
    28. Kukacka, Jiri & Barunik, Jozef, 2017. "Estimation of financial agent-based models with simulated maximum likelihood," Journal of Economic Dynamics and Control, Elsevier, vol. 85(C), pages 21-45.
    29. Guerini, Mattia & Moneta, Alessio, 2017. "A method for agent-based models validation," Journal of Economic Dynamics and Control, Elsevier, vol. 82(C), pages 125-141.
    30. Rommel Regis & Christine Shoemaker, 2005. "Constrained Global Optimization of Expensive Black Box Functions Using Radial Basis Functions," Journal of Global Optimization, Springer, vol. 31(1), pages 153-171, January.
    31. J. Doyne Farmer & John Geanakoplos, 2008. "The Virtues and Vices of Equilibrium and the Future of Financial Economics," Levine's Working Paper Archive 122247000000002067, David K. Levine.
    32. Farmer, J. Doyne & Joshi, Shareen, 2002. "The price dynamics of common trading strategies," Journal of Economic Behavior & Organization, Elsevier, vol. 49(2), pages 149-171, October.
    33. repec:hal:spmain:info:hdl:2441/13thfd12aa8rmplfudlgvgahff is not listed on IDEAS
    34. J. Doyne Farmer & Duncan Foley, 2009. "The economy needs agent-based modelling," Nature, Nature, vol. 460(7256), pages 685-686, August.
    35. LeBaron, Blake, 2006. "Agent-based Computational Finance," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 24, pages 1187-1233, Elsevier.
    36. Lux, Thomas, 2018. "Estimation of agent-based models using sequential Monte Carlo methods," Journal of Economic Dynamics and Control, Elsevier, vol. 91(C), pages 391-408.
    37. Franke, Reiner, 2009. "Applying the method of simulated moments to estimate a small agent-based asset pricing model," Journal of Empirical Finance, Elsevier, vol. 16(5), pages 804-815, December.
    38. repec:hal:spmain:info:hdl:2441/5fafm6me7k8omq5jbo61urqq27 is not listed on IDEAS
    39. Jakob Grazzini, 2012. "Analysis of the Emergent Properties: Stationarity and Ergodicity," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 15(2), pages 1-7.
    40. Lamperti, Francesco, 2018. "An information theoretic criterion for empirical validation of simulation models," Econometrics and Statistics, Elsevier, vol. 5(C), pages 83-106.
    41. Isabelle Salle & Murat Yıldızoğlu, 2014. "Efficient Sampling and Meta-Modeling for Computational Economic Models," Computational Economics, Springer;Society for Computational Economics, vol. 44(4), pages 507-536, December.
    42. Simone Alfarano & Thomas Lux & Friedrich Wagner, 2005. "Estimation of Agent-Based Models: The Case of an Asymmetric Herding Model," Computational Economics, Springer;Society for Computational Economics, vol. 26(1), pages 19-49, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Donovan Platt, 2019. "A Comparison of Economic Agent-Based Model Calibration Methods," Papers 1902.05938, arXiv.org.
    2. Donovan Platt, 2022. "Bayesian Estimation of Economic Simulation Models Using Neural Networks," Computational Economics, Springer;Society for Computational Economics, vol. 59(2), pages 599-650, February.
    3. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    4. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    5. repec:hal:spmain:info:hdl:2441/13thfd12aa8rmplfudlgvgahff is not listed on IDEAS
    6. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    7. repec:hal:spmain:info:hdl:2441/20hflp7eqn97boh50no50tv67n is not listed on IDEAS
    8. Zila, Eric & Kukacka, Jiri, 2023. "Moment set selection for the SMM using simple machine learning," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 366-391.
    9. Barde, Sylvain, 2020. "Macroeconomic simulation comparison with a multivariate extension of the Markov information criterion," Journal of Economic Dynamics and Control, Elsevier, vol. 111(C).
    10. Tubbenhauer, Tobias & Fieberg, Christian & Poddig, Thorsten, 2021. "Multi-agent-based VaR forecasting," Journal of Economic Dynamics and Control, Elsevier, vol. 131(C).
    11. Francesco Lamperti, 2018. "Empirical validation of simulated models through the GSL-div: an illustrative application," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 13(1), pages 143-171, April.
    12. Siyan Chen & Saul Desiderio, 2022. "Calibration of Agent-Based Models by Means of Meta-Modeling and Nonparametric Regression," Computational Economics, Springer;Society for Computational Economics, vol. 60(4), pages 1457-1478, December.
    13. Lamperti, Francesco, 2018. "An information theoretic criterion for empirical validation of simulation models," Econometrics and Statistics, Elsevier, vol. 5(C), pages 83-106.
    14. Grazzini, Jakob & Richiardi, Matteo G. & Tsionas, Mike, 2017. "Bayesian estimation of agent-based models," Journal of Economic Dynamics and Control, Elsevier, vol. 77(C), pages 26-47.
    15. Platt, Donovan & Gebbie, Tim, 2018. "Can agent-based models probe market microstructure?," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 503(C), pages 1092-1106.
    16. Donovan Platt & Tim Gebbie, 2016. "The Problem of Calibrating an Agent-Based Model of High-Frequency Trading," Papers 1606.01495, arXiv.org, revised Mar 2017.
    17. Seri, Raffaello & Martinoli, Mario & Secchi, Davide & Centorrino, Samuele, 2021. "Model calibration and validation via confidence sets," Econometrics and Statistics, Elsevier, vol. 20(C), pages 62-86.
    18. Mario Martinoli & Alessio Moneta & Gianluca Pallante, 2022. "Calibration and Validation of Macroeconomic Simulation Models by Statistical Causal Search," LEM Papers Series 2022/33, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    19. Guerini, Mattia & Moneta, Alessio, 2017. "A method for agent-based models validation," Journal of Economic Dynamics and Control, Elsevier, vol. 82(C), pages 125-141.
    20. Kukacka, Jiri & Kristoufek, Ladislav, 2021. "Does parameterization affect the complexity of agent-based models?," Journal of Economic Behavior & Organization, Elsevier, vol. 192(C), pages 324-356.
    21. Kukacka, Jiri & Sacht, Stephen, 2023. "Estimation of heuristic switching in behavioral macroeconomic models," Journal of Economic Dynamics and Control, Elsevier, vol. 146(C).
    22. Delli Gatti, Domenico & Grazzini, Jakob, 2020. "Rising to the challenge: Bayesian estimation and forecasting techniques for macroeconomic Agent Based Models," Journal of Economic Behavior & Organization, Elsevier, vol. 178(C), pages 875-902.

    More about this item

    Keywords

    Agent-based modelling; Calibration; Simulated minimum distance; Bayesian estimation;
    All these keywords.

    JEL classification:

    • C13 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Estimation: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:dyncon:v:113:y:2020:i:c:s0165188920300294. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jedc .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.