IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1902.05938.html
   My bibliography  Save this paper

A Comparison of Economic Agent-Based Model Calibration Methods

Author

Listed:
  • Donovan Platt

Abstract

Interest in agent-based models of financial markets and the wider economy has increased consistently over the last few decades, in no small part due to their ability to reproduce a number of empirically-observed stylised facts that are not easily recovered by more traditional modelling approaches. Nevertheless, the agent-based modelling paradigm faces mounting criticism, focused particularly on the rigour of current validation and calibration practices, most of which remain qualitative and stylised fact-driven. While the literature on quantitative and data-driven approaches has seen significant expansion in recent years, most studies have focused on the introduction of new calibration methods that are neither benchmarked against existing alternatives nor rigorously tested in terms of the quality of the estimates they produce. We therefore compare a number of prominent ABM calibration methods, both established and novel, through a series of computational experiments in an attempt to determine the respective strengths and weaknesses of each approach and the overall quality of the resultant parameter estimates. We find that Bayesian estimation, though less popular in the literature, consistently outperforms frequentist, objective function-based approaches and results in reasonable parameter estimates in many contexts. Despite this, we also find that agent-based model calibration techniques require further development in order to definitively calibrate large-scale models. We therefore make suggestions for future research.

Suggested Citation

  • Donovan Platt, 2019. "A Comparison of Economic Agent-Based Model Calibration Methods," Papers 1902.05938, arXiv.org.
  • Handle: RePEc:arx:papers:1902.05938
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1902.05938
    File Function: Latest version
    Download Restriction: no

    References listed on IDEAS

    as
    1. Brock, William A. & Hommes, Cars H., 1998. "Heterogeneous beliefs and routes to chaos in a simple asset pricing model," Journal of Economic Dynamics and Control, Elsevier, vol. 22(8-9), pages 1235-1274, August.
    2. S. Alfarano & T. Lux & F. Wagner, 2007. "Empirical validation of stochastic models of interacting agents," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 55(2), pages 183-187, January.
    3. Peter Winker & Manfred Gilli & Vahidin Jeleskovic, 2007. "An objective function for simulation based inference on exchange rate data," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 2(2), pages 125-145, December.
    4. Franke, Reiner & Westerhoff, Frank, 2012. "Structural stochastic volatility in asset pricing dynamics: Estimation and model contest," Journal of Economic Dynamics and Control, Elsevier, vol. 36(8), pages 1193-1211.
    5. Zhenxi Chen & Thomas Lux, 2018. "Estimation of Sentiment Effects in Financial Markets: A Simulated Method of Moments Approach," Computational Economics, Springer;Society for Computational Economics, vol. 52(3), pages 711-744, October.
    6. Gilli, M. & Winker, P., 2003. "A global optimization heuristic for estimating agent based models," Computational Statistics & Data Analysis, Elsevier, vol. 42(3), pages 299-312, March.
    7. John Geanakoplos & Robert Axtell & J. Doyne Farmer & Peter Howitt & Benjamin Conlee & Jonathan Goldstein & Matthew Hendrey & Nathan M. Palmer & Chun-Yi Yang, 2012. "Getting at Systemic Risk via an Agent-Based Model of the Housing Market," American Economic Review, American Economic Association, vol. 102(3), pages 53-58, May.
    8. Guerini, Mattia & Moneta, Alessio, 2017. "A method for agent-based models validation," Journal of Economic Dynamics and Control, Elsevier, vol. 82(C), pages 125-141.
    9. Rommel Regis & Christine Shoemaker, 2005. "Constrained Global Optimization of Expensive Black Box Functions Using Radial Basis Functions," Journal of Global Optimization, Springer, vol. 31(1), pages 153-171, January.
    10. Isabelle Salle & Murat Yıldızoğlu, 2014. "Efficient Sampling and Meta-Modeling for Computational Economic Models," Computational Economics, Springer;Society for Computational Economics, vol. 44(4), pages 507-536, December.
    11. Sandrine Jacob Leal & Mauro Napoletano & Andrea Roventini & Giorgio Fagiolo, 2016. "Rock around the clock: An agent-based model of low- and high-frequency trading," Journal of Evolutionary Economics, Springer, vol. 26(1), pages 49-76, March.
    12. Barde, Sylvain, 2016. "Direct comparison of agent-based models of herding in financial markets," Journal of Economic Dynamics and Control, Elsevier, vol. 73(C), pages 329-353.
    13. Sylvain Barde, 2017. "A Practical, Accurate, Information Criterion for Nth Order Markov Processes," Computational Economics, Springer;Society for Computational Economics, vol. 50(2), pages 281-324, August.
    14. Giorgio Fagiolo & Andrea Roventini, 2017. "Macroeconomic Policy in DSGE and Agent-Based Models Redux: New Developments and Challenges Ahead," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 20(1), pages 1-1.
    15. Sylvain Barde & Sander Van Der Hoog, 2017. "An empirical validation protocol for large-scale agent-based models," Sciences Po publications 17/12, Sciences Po.
    16. Recchioni, Maria Cristina & Tedeschi, Gabriele & Gallegati, Mauro, 2015. "A calibration procedure for analyzing stock price dynamics in an agent-based framework," Journal of Economic Dynamics and Control, Elsevier, vol. 60(C), pages 1-25.
    17. Sandrine Jacob Leal & Mauro Napoletano & Andrea Roventini & Giorgio Fagiolo, 2016. "Rock around the clock: An agent-based model of low- and high-frequency trading," Journal of Evolutionary Economics, Springer, vol. 26(1), pages 49-76, March.
    18. Annalisa Fabretti, 2013. "On the problem of calibrating an agent based model for financial markets," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 8(2), pages 277-293, October.
    19. J. Doyne Farmer & John Geanakoplos, 2008. "The Virtues and Vices of Equilibrium and the Future of Financial Economics," Levine's Working Paper Archive 122247000000002067, David K. Levine.
    20. Farmer, J. Doyne & Joshi, Shareen, 2002. "The price dynamics of common trading strategies," Journal of Economic Behavior & Organization, Elsevier, vol. 49(2), pages 149-171, October.
    21. Grazzini, Jakob & Richiardi, Matteo, 2015. "Estimation of ergodic agent-based models by simulated minimum distance," Journal of Economic Dynamics and Control, Elsevier, vol. 51(C), pages 148-165.
    22. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    23. Lars Peter Hansen & James J. Heckman, 1996. "The Empirical Foundations of Calibration," Journal of Economic Perspectives, American Economic Association, vol. 10(1), pages 87-104, Winter.
    24. Carlo Bianchi & Pasquale Cirillo & Mauro Gallegati & Pietro Vagliasindi, 2007. "Validating and Calibrating Agent-Based Models: A Case Study," Computational Economics, Springer;Society for Computational Economics, vol. 30(3), pages 245-264, October.
    25. Dosi, Giovanni & Fagiolo, Giorgio & Roventini, Andrea, 2010. "Schumpeter meeting Keynes: A policy-friendly model of endogenous growth and business cycles," Journal of Economic Dynamics and Control, Elsevier, vol. 34(9), pages 1748-1767, September.
    26. Franke, Reiner, 2009. "Applying the method of simulated moments to estimate a small agent-based asset pricing model," Journal of Empirical Finance, Elsevier, vol. 16(5), pages 804-815, December.
    27. McFadden, Daniel, 1989. "A Method of Simulated Moments for Estimation of Discrete Response Models without Numerical Integration," Econometrica, Econometric Society, vol. 57(5), pages 995-1026, September.
    28. Grazzini, Jakob & Richiardi, Matteo G. & Tsionas, Mike, 2017. "Bayesian estimation of agent-based models," Journal of Economic Dynamics and Control, Elsevier, vol. 77(C), pages 26-47.
    29. Alfarano, Simone & Lux, Thomas & Wagner, Friedrich, 2006. "Estimation of a simple agent-based model of financial markets: An application to Australian stock and foreign exchange data," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 370(1), pages 38-42.
    30. Jakob Grazzini, 2012. "Analysis of the Emergent Properties: Stationarity and Ergodicity," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 15(2), pages 1-7.
    31. R. Cont, 2001. "Empirical properties of asset returns: stylized facts and statistical issues," Quantitative Finance, Taylor & Francis Journals, vol. 1(2), pages 223-236.
    32. Kukacka, Jiri & Barunik, Jozef, 2017. "Estimation of financial agent-based models with simulated maximum likelihood," Journal of Economic Dynamics and Control, Elsevier, vol. 85(C), pages 21-45.
    33. Baptista, Rafa & Farmer, J. Doyne & Hinterschweiger, Marc & Low, Katie & Tang, Daniel & Uluc, Arzu, 2016. "Macroprudential policy in an agent-based model of the UK housing market," Bank of England working papers 619, Bank of England.
    34. Simone Alfarano & Thomas Lux & Friedrich Wagner, 2005. "Estimation of Agent-Based Models: The Case of an Asymmetric Herding Model," Computational Economics, Springer;Society for Computational Economics, vol. 26(1), pages 19-49, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Heinrich, Torsten & Sabuco, Juan & Farmer, J. Doyne, 2019. "A simulation of the insurance industry: The problem of risk model homogeneity," MPRA Paper 95096, University Library of Munich, Germany.
    2. Bertani, Filippo & Ponta, Linda & Raberto, Marco & Teglio, Andrea & Cincotti, Silvano, 2019. "The complexity of the intangible digital economy: an agent-based model," MPRA Paper 97071, University Library of Munich, Germany.
    3. Torsten Heinrich & Juan Sabuco & J. Doyne Farmer, 2019. "A simulation of the insurance industry: The problem of risk model homogeneity," Papers 1907.05954, arXiv.org, revised Nov 2019.
    4. Domenico Delli Gatti & Jakob Grazzini, 2019. "Rising to the Challenge: Bayesian Estimation and Forecasting Techniques for Macroeconomic Agent-Based Models," CESifo Working Paper Series 7894, CESifo.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1902.05938. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (arXiv administrators). General contact details of provider: http://arxiv.org/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.