IDEAS home Printed from https://ideas.repec.org/p/spo/wpmain/infohdl2441-20hflp7eqn97boh50no50tv67n.html
   My bibliography  Save this paper

Agent-Based Model Calibration using Machine Learning Surrogates

Author

Listed:
  • Francesco Lamperti

    (Université Panthéon-Sorbonne - Paris 1 (UP1))

  • Andrea Roventini

    (Laboratory of Economics and Management (LEM))

  • Amir Sani

    (Université Paris 1 Panthéon-Sorbonne (UP1))

Abstract

Taking agent-based models (ABM) closer to the data is an open challenge. This paper explicitly tackles parameter space exploration and calibration of ABMs combining supervised machine-learning and intelligent sampling to build a surrogate meta-model. The proposed approach provides a fast and accurate approximation of model behaviour, dramatically reducing computation time. In that, our machine-learning surrogate facilitates large scale explorations of the parameter-space, while providing a powerful filter to gain insights into the complex functioning of agent-based models. The algorithm introduced in this paper merges model simulation and output analysis into a surrogate meta-model, which substantially ease ABM calibration. We successfully apply our approach to the Brock and Hommes (1998) asset pricing model and to the “Island” endogenous growth model (Fagiolo and Dosi, 2003). Performance is evaluated against a relatively large outof-sample set of parameter combinations, while employing different user-defined statistical tests for output analysis. The results demonstrate the capacity of machine learning surrogates to facilitate fast and precise exploration of agent-based models’ behaviour over their often rugged parameter spaces.

Suggested Citation

  • Francesco Lamperti & Andrea Roventini & Amir Sani, 2017. "Agent-Based Model Calibration using Machine Learning Surrogates," Sciences Po publications 2017-09, Sciences Po.
  • Handle: RePEc:spo:wpmain:info:hdl:2441/20hflp7eqn97boh50no50tv67n
    as

    Download full text from publisher

    File URL: http://spire.sciencespo.fr/hdl:/2441/20hflp7eqn97boh50no50tv67n/resources/2017-agent-based-model-calibration-using-machine-learning-surrogates-.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Chiarella, Carl & Iori, Giulia, 2009. "The impact of heterogeneous trading rules on the limit order book and order flows," Journal of Economic Dynamics and Control, Elsevier, vol. 33(3), pages 525-537.
    2. Giorgio Fagiolo & Mauro Napoletano & Andrea Roventini, 2008. "Are output growth-rate distributions fat-tailed? some evidence from OECD countries," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 23(5), pages 639-669.
    3. G. Fagiolo & C. Birchenhall & P. Windrum, 2007. "Empirical Validation in Agent-based Models: Introduction to the Special Issue," Computational Economics, Springer;Society for Computational Economics, vol. 30(3), pages 189-194, October.
    4. Sandrine Jacob Leal & Mauro Napoletano & Andrea Roventini & Giorgio Fagiolo, 2016. "Rock around the clock: An agent-based model of low- and high-frequency trading," Journal of Evolutionary Economics, Springer, vol. 26(1), pages 49-76, March.
    5. Barde, Sylvain, 2016. "Direct comparison of agent-based models of herding in financial markets," Journal of Economic Dynamics and Control, Elsevier, vol. 73(C), pages 329-353.
    6. Recchioni, Maria Cristina & Tedeschi, Gabriele & Gallegati, Mauro, 2015. "A calibration procedure for analyzing stock price dynamics in an agent-based framework," Journal of Economic Dynamics and Control, Elsevier, vol. 60(C), pages 1-25.
    7. Giorgio Fagiolo & Andrea Roventini, 2017. "Macroeconomic Policy in DSGE and Agent-Based Models Redux: New Developments and Challenges Ahead," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 20(1), pages 1-1.
    8. Annalisa Fabretti, 2013. "On the problem of calibrating an agent based model for financial markets," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 8(2), pages 277-293, October.
    9. Fagiolo, Giorgio & Dosi, Giovanni, 2003. "Exploitation, exploration and innovation in a model of endogenous growth with locally interacting agents," Structural Change and Economic Dynamics, Elsevier, vol. 14(3), pages 237-273, September.
    10. Dosi, G. & Pereira, M.C. & Roventini, A. & Virgillito, M.E., 2017. "When more flexibility yields more fragility: The microfoundations of Keynesian aggregate unemployment," Journal of Economic Dynamics and Control, Elsevier, vol. 81(C), pages 162-186.
    11. Paul Windrum & Giorgio Fagiolo & Alessio Moneta, 2007. "Empirical Validation of Agent-Based Models: Alternatives and Prospects," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 10(2), pages 1-8.
    12. Archer, Kellie J. & Kimes, Ryan V., 2008. "Empirical characterization of random forest variable importance measures," Computational Statistics & Data Analysis, Elsevier, vol. 52(4), pages 2249-2260, January.
    13. Giorgio Fagiolo & Andrea Roventini, 2012. "Macroeconomic Policy in DSGE and Agent-Based Models," Revue de l'OFCE, Presses de Sciences-Po, vol. 0(5), pages 67-116.
    14. Guus ten Broeke & George van Voorn & Arend Ligtenberg, 2016. "Which Sensitivity Analysis Method Should I Use for My Agent-Based Model?," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 19(1), pages 1-5.
    15. repec:spr:jeicoo:v:13:y:2018:i:1:d:10.1007_s11403-017-0193-4 is not listed on IDEAS
    16. Francesco Lamperti, 2016. "Empirical Validation of Simulated Models through the GSL-div: an Illustrative Application," LEM Papers Series 2016/18, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    17. Dosi, Giovanni & Fagiolo, Giorgio & Napoletano, Mauro & Roventini, Andrea, 2013. "Income distribution, credit and fiscal policies in an agent-based Keynesian model," Journal of Economic Dynamics and Control, Elsevier, vol. 37(8), pages 1598-1625.
    18. Grazzini, Jakob & Richiardi, Matteo G. & Tsionas, Mike, 2017. "Bayesian estimation of agent-based models," Journal of Economic Dynamics and Control, Elsevier, vol. 77(C), pages 26-47.
    19. Boswijk, H. Peter & Hommes, Cars H. & Manzan, Sebastiano, 2007. "Behavioral heterogeneity in stock prices," Journal of Economic Dynamics and Control, Elsevier, vol. 31(6), pages 1938-1970, June.
    20. Francesco Lamperti & Giovanni Dosi & Mauro Napoletano & Andrea Roventini & Sandro Sapio, 2017. "Faraway, so close : coupled climate and economic dynamics in an agent-based integrated assessment model," Sciences Po publications info:hdl:2441/4hs7liq1f49, Sciences Po.
    21. Dosi, Giovanni & Fagiolo, Giorgio & Roventini, Andrea, 2010. "Schumpeter meeting Keynes: A policy-friendly model of endogenous growth and business cycles," Journal of Economic Dynamics and Control, Elsevier, vol. 34(9), pages 1748-1767, September.
    22. Isabelle Salle & Murat Yıldızoğlu, 2014. "Efficient Sampling and Meta-Modeling for Computational Economic Models," Computational Economics, Springer;Society for Computational Economics, vol. 44(4), pages 507-536, December.
    23. repec:eee:dyncon:v:85:y:2017:i:c:p:21-45 is not listed on IDEAS
    24. Carolina Castaldi & Giovanni Dosi, 2009. "The patterns of output growth of firms and countries: Scale invariances and scale specificities," Empirical Economics, Springer, vol. 37(3), pages 475-495, December.
    25. Popoyan, Lilit & Napoletano, Mauro & Roventini, Andrea, 2017. "Taming macroeconomic instability: Monetary and macro-prudential policy interactions in an agent-based model," Journal of Economic Behavior & Organization, Elsevier, vol. 134(C), pages 117-140.
    26. Dosi, Giovanni & Fagiolo, Giorgio & Napoletano, Mauro & Roventini, Andrea & Treibich, Tania, 2015. "Fiscal and monetary policies in complex evolving economies," Journal of Economic Dynamics and Control, Elsevier, vol. 52(C), pages 166-189.
    27. Peter Winker & Manfred Gilli & Vahidin Jeleskovic, 2007. "An objective function for simulation based inference on exchange rate data," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 2(2), pages 125-145, December.
    28. Francesco Lamperti & Clara Elisabetta Mattei, 2016. "Going Up and Down: Rethinking the Empirics of Growth in the Developing and Newly Industrialized World," LEM Papers Series 2016/01, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    29. Dawid, H. & Harting, P. & Neugart, M., 2014. "Economic convergence: Policy implications from a heterogeneous agent model," Journal of Economic Dynamics and Control, Elsevier, vol. 44(C), pages 54-80.
    30. repec:spo:wpecon:info:hdl:2441/f6h8764enu2lskk9p4oq9ig8k is not listed on IDEAS
    31. Dosi, Giovanni & Pereira, Marcelo C. & Roventini, Andrea & Virgillito, Maria Enrica, 2017. "Causes and Consequences of Hysteresis: Aggregate Demand, Productivity and Employment," GLO Discussion Paper Series 64, Global Labor Organization (GLO).
    32. Tesfatsion, Leigh & Judd, Kenneth L., 2006. "Handbook of Computational Economics, Vol. 2: Agent-Based Computational Economics," Staff General Research Papers Archive 10368, Iowa State University, Department of Economics.
    33. Giulio Bottazzi & Angelo Secchi, 2006. "Explaining the distribution of firm growth rates," RAND Journal of Economics, RAND Corporation, vol. 37(2), pages 235-256, June.
    34. Amilon, Henrik, 2008. "Estimation of an adaptive stock market model with heterogeneous agents," Journal of Empirical Finance, Elsevier, vol. 15(2), pages 342-362, March.
    35. Giovanni Dosi & Marcelo C. Pereira & Andrea Roventini & Maria Enrica Virgillito, 2016. "The Effects of Labour Market Reforms upon Unemployment and Income Inequalities: an Agent Based Model," Sciences Po publications 2016-24, Sciences Po.
    36. Franke, Reiner, 2009. "Applying the method of simulated moments to estimate a small agent-based asset pricing model," Journal of Empirical Finance, Elsevier, vol. 16(5), pages 804-815, December.
    37. Caiani, Alessandro & Godin, Antoine & Caverzasi, Eugenio & Gallegati, Mauro & Kinsella, Stephen & Stiglitz, Joseph E., 2016. "Agent based-stock flow consistent macroeconomics: Towards a benchmark model," Journal of Economic Dynamics and Control, Elsevier, vol. 69(C), pages 375-408.
    38. William A. Brock & Cars H. Hommes, 1997. "A Rational Route to Randomness," Econometrica, Econometric Society, vol. 65(5), pages 1059-1096, September.
    39. Assenza, Tiziana & Delli Gatti, Domenico & Grazzini, Jakob, 2015. "Emergent dynamics of a macroeconomic agent based model with capital and credit," Journal of Economic Dynamics and Control, Elsevier, vol. 50(C), pages 5-28.
    40. Ju-Sung Lee & Tatiana Filatova & Arika Ligmann-Zielinska & Behrooz Hassani-Mahmooei & Forrest Stonedahl & Iris Lorscheid & Alexey Voinov & J. Gary Polhill & Zhanli Sun & Dawn C. Parker, 2015. "The Complexities of Agent-Based Modeling Output Analysis," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 18(4), pages 1-4.
    41. Guerini, Mattia & Moneta, Alessio, 2017. "A method for agent-based models validation," Journal of Economic Dynamics and Control, Elsevier, vol. 82(C), pages 125-141.
    42. Abhijit V. Banerjee, 1992. "A Simple Model of Herd Behavior," The Quarterly Journal of Economics, Oxford University Press, vol. 107(3), pages 797-817.
    43. Simone Alfarano & Thomas Lux & Friedrich Wagner, 2005. "Estimation of Agent-Based Models: The Case of an Asymmetric Herding Model," Computational Economics, Springer;Society for Computational Economics, vol. 26(1), pages 19-49, August.
    44. Brock, William A. & Hommes, Cars H., 1998. "Heterogeneous beliefs and routes to chaos in a simple asset pricing model," Journal of Economic Dynamics and Control, Elsevier, vol. 22(8-9), pages 1235-1274, August.
    45. Sander van der Hoog, 2017. "Deep Learning in (and of) Agent-Based Models: A Prospectus," Papers 1706.06302, arXiv.org.
    46. Gilli, M. & Winker, P., 2003. "A global optimization heuristic for estimating agent based models," Computational Statistics & Data Analysis, Elsevier, vol. 42(3), pages 299-312, March.
    47. Sylvain Barde, 2017. "A Practical, Accurate, Information Criterion for Nth Order Markov Processes," Computational Economics, Springer;Society for Computational Economics, vol. 50(2), pages 281-324, August.
    48. Leonardo Bargigli & Luca Riccetti & Alberto Russo & Mauro Gallegati, 2016. "Network Calibration and Metamodeling of a Financial Accelerator Agent Based Model," Working Papers - Economics wp2016_01.rdf, Universita' degli Studi di Firenze, Dipartimento di Scienze per l'Economia e l'Impresa.
    49. Francesco Lamperti, 2015. "An Information Theoretic Criterion for Empirical Validation of Time Series Models," LEM Papers Series 2015/02, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    50. Dosi, Giovanni, 1988. "Sources, Procedures, and Microeconomic Effects of Innovation," Journal of Economic Literature, American Economic Association, vol. 26(3), pages 1120-1171, September.
    51. Alfarano, Simone & Lux, Thomas & Wagner, Friedrich, 2006. "Estimation of a simple agent-based model of financial markets: An application to Australian stock and foreign exchange data," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 370(1), pages 38-42.
    52. Grazzini, Jakob & Richiardi, Matteo, 2015. "Estimation of ergodic agent-based models by simulated minimum distance," Journal of Economic Dynamics and Control, Elsevier, vol. 51(C), pages 148-165.
    53. G. Dosi & M. C. Pereira & M. E. Virgillito, 2018. "On the robustness of the fat-tailed distribution of firm growth rates: a global sensitivity analysis," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 13(1), pages 173-193, April.
    54. Flaminio Squazzoni, 2010. "The impact of agent-based models in the social sciences after 15 years of incursions," History of Economic Ideas, Fabrizio Serra Editore, Pisa - Roma, vol. 18(2), pages 197-234.
    55. Franke, Reiner & Westerhoff, Frank, 2012. "Structural stochastic volatility in asset pricing dynamics: Estimation and model contest," Journal of Economic Dynamics and Control, Elsevier, vol. 36(8), pages 1193-1211.
    56. Jan C. Thiele & Winfried Kurth & Volker Grimm, 2014. "Facilitating Parameter Estimation and Sensitivity Analysis of Agent-Based Models: A Cookbook Using NetLogo and 'R'," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 17(3), pages 1-11.
    57. Jakob Grazzini, 2012. "Analysis of the Emergent Properties: Stationarity and Ergodicity," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 15(2), pages 1-7.
    58. Sylvain Barde, 2015. "A Practical, Universal, Information Criterion over Nth Order Markov Processes," Studies in Economics 1504, School of Economics, University of Kent.
    59. Kukacka, Jiri & Barunik, Jozef, 2017. "Estimation of financial agent-based models with simulated maximum likelihood," Journal of Economic Dynamics and Control, Elsevier, vol. 85(C), pages 21-45.
    60. Weeks, Melvyn, 1995. "Circumventing the Curse of Dimensionality in Applied Work Using Computer Intensive Methods," Economic Journal, Royal Economic Society, vol. 105(429), pages 520-530, March.
    61. Sandrine Jacob Leal & Mauro Napoletano & Andrea Roventini & Giorgio Fagiolo, 2016. "Rock around the clock: An agent-based model of low- and high-frequency trading," Journal of Evolutionary Economics, Springer, vol. 26(1), pages 49-76, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Francesco Lamperti & Giovanni Dosi & Mauro Napoletano & Andrea Roventini & Sandro Sapio, 2017. "Faraway, so close : coupled climate and economic dynamics in an agent-based integrated assessment model," Sciences Po publications info:hdl:2441/4hs7liq1f49, Sciences Po.
    2. Lux, Thomas, 2017. "Estimation of agent-based models using sequential Monte Carlo methods," Economics Working Papers 2017-07, Christian-Albrechts-University of Kiel, Department of Economics.
    3. repec:eee:ecolec:v:150:y:2018:i:c:p:315-339 is not listed on IDEAS
    4. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.

    More about this item

    Keywords

    Agent-based model; Calibration; Machine learning; Surrogate; Meta-model;

    JEL classification:

    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spo:wpmain:info:hdl:2441/20hflp7eqn97boh50no50tv67n. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Spire @ Sciences Po Library). General contact details of provider: http://edirc.repec.org/data/ecspofr.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.