IDEAS home Printed from https://ideas.repec.org/a/eee/dyncon/v111y2020ics0165188919301927.html
   My bibliography  Save this article

Macroeconomic simulation comparison with a multivariate extension of the Markov information criterion

Author

Listed:
  • Barde, Sylvain

Abstract

The paper aims to address the issue of comparing agent-based models (ABMs) with more traditional VAR and DSGE models by developing a multivariate extension of the Markov Information Criterion (MIC) of Barde (2017). The univariate MIC measures the informational distance between a simulation model and some empirical data by mapping the simulated data to a Markov transition matrix, and is proven to provide an unbiased measurement for all models reducible to a Markov process. As a result, the MIC can accurately measure distance using only simulated data, for a wide class of data generating processes. The paper first presents the multivariate extension of the MIC and its validation on VAR and DGSE models before carrying the first direct comparison between a macroeconomic ABM and a DGSE model, namely the benchmark ABM of Caiani et al. (2016) and Smets and Wouters (2007).

Suggested Citation

  • Barde, Sylvain, 2020. "Macroeconomic simulation comparison with a multivariate extension of the Markov information criterion," Journal of Economic Dynamics and Control, Elsevier, vol. 111(C).
  • Handle: RePEc:eee:dyncon:v:111:y:2020:i:c:s0165188919301927
    DOI: 10.1016/j.jedc.2019.103795
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0165188919301927
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Brock, William A. & Hommes, Cars H., 1998. "Heterogeneous beliefs and routes to chaos in a simple asset pricing model," Journal of Economic Dynamics and Control, Elsevier, vol. 22(8-9), pages 1235-1274, August.
    2. Popoyan, Lilit & Napoletano, Mauro & Roventini, Andrea, 2017. "Taming macroeconomic instability: Monetary and macro-prudential policy interactions in an agent-based model," Journal of Economic Behavior & Organization, Elsevier, vol. 134(C), pages 117-140.
    3. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    4. Gilli, M. & Winker, P., 2003. "A global optimization heuristic for estimating agent based models," Computational Statistics & Data Analysis, Elsevier, vol. 42(3), pages 299-312, March.
    5. Ashraf, Quamrul & Gershman, Boris & Howitt, Peter, 2017. "Banks, market organization, and macroeconomic performance: An agent-based computational analysis," Journal of Economic Behavior & Organization, Elsevier, vol. 135(C), pages 143-180.
    6. Marco Del Negro & Frank Schorfheide, 2006. "How good is what you've got? DSGE-VAR as a toolkit for evaluating DSGE models," Economic Review, Federal Reserve Bank of Atlanta, vol. 91(Q 2), pages 21-37.
    7. Lindé, Jesper & Smets, Frank & Wouters, Rafael, 2016. "Challenges for Central Banks´ Macro Models," Working Paper Series 323, Sveriges Riksbank (Central Bank of Sweden).
    8. Barde, Sylvain, 2016. "Direct comparison of agent-based models of herding in financial markets," Journal of Economic Dynamics and Control, Elsevier, vol. 73(C), pages 329-353.
    9. Sylvain Barde, 2017. "A Practical, Accurate, Information Criterion for Nth Order Markov Processes," Computational Economics, Springer;Society for Computational Economics, vol. 50(2), pages 281-324, August.
    10. Giorgio Fagiolo & Andrea Roventini, 2017. "Macroeconomic Policy in DSGE and Agent-Based Models Redux: New Developments and Challenges Ahead," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 20(1), pages 1-1.
    11. Alessandro Caiani & Alberto Russo & Mauro Gallegati, 2019. "Does inequality hamper innovation and growth? An AB-SFC analysis," Journal of Evolutionary Economics, Springer, vol. 29(1), pages 177-228, March.
    12. Frank Smets & Rafael Wouters, 2007. "Shocks and Frictions in US Business Cycles: A Bayesian DSGE Approach," American Economic Review, American Economic Association, vol. 97(3), pages 586-606, June.
    13. Kimball, Miles S, 1995. "The Quantitative Analytics of the Basic Neomonetarist Model," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 27(4), pages 1241-1277, November.
    14. Negro, Marco Del & Schorfheide, Frank, 2013. "DSGE Model-Based Forecasting," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 57-140, Elsevier.
    15. Joeri Schasfoort & Antoine Godin & Dirk Bezemer & Alessandro Caiani & Stephen Kinsella, 2017. "Monetary Policy Transmission In A Macroeconomic Agent-Based Model," Advances in Complex Systems (ACS), World Scientific Publishing Co. Pte. Ltd., vol. 20(08), pages 1-35, December.
    16. Dosi, Giovanni & Fagiolo, Giorgio & Napoletano, Mauro & Roventini, Andrea, 2013. "Income distribution, credit and fiscal policies in an agent-based Keynesian model," Journal of Economic Dynamics and Control, Elsevier, vol. 37(8), pages 1598-1625.
    17. Giorgio Fagiolo & Andrea Roventini, 2012. "Macroeconomic Policy in DSGE and Agent-Based Models," Revue de l'OFCE, Presses de Sciences-Po, vol. 0(5), pages 67-116.
    18. Del Negro, Marco & Hasegawa, Raiden B. & Schorfheide, Frank, 2016. "Dynamic prediction pools: An investigation of financial frictions and forecasting performance," Journal of Econometrics, Elsevier, vol. 192(2), pages 391-405.
    19. Lindé, J. & Smets, F. & Wouters, R., 2016. "Challenges for Central Banks’ Macro Models," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 2185-2262, Elsevier.
    20. Dawid, Herbert & Gemkow, Simon & Harting, Philipp & van der Hoog, Sander & Neugart, Michael, 2018. "Agent-based macroeconomic modeling and policy analyses: The Eurace@Unibi Model," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 110863, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    21. Herbert Dawid & Philipp Harting & Sander Hoog & Michael Neugart, 2019. "Macroeconomics with heterogeneous agent models: fostering transparency, reproducibility and replication," Journal of Evolutionary Economics, Springer, vol. 29(1), pages 467-538, March.
    22. Grazzini, Jakob & Richiardi, Matteo, 2015. "Estimation of ergodic agent-based models by simulated minimum distance," Journal of Economic Dynamics and Control, Elsevier, vol. 51(C), pages 148-165.
    23. Francesco Lamperti, 2018. "Empirical validation of simulated models through the GSL-div: an illustrative application," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 13(1), pages 143-171, April.
    24. Grazzini, Jakob & Richiardi, Matteo G. & Tsionas, Mike, 2017. "Bayesian estimation of agent-based models," Journal of Economic Dynamics and Control, Elsevier, vol. 77(C), pages 26-47.
    25. Dosi, Giovanni & Fagiolo, Giorgio & Napoletano, Mauro & Roventini, Andrea & Treibich, Tania, 2015. "Fiscal and monetary policies in complex evolving economies," Journal of Economic Dynamics and Control, Elsevier, vol. 52(C), pages 166-189.
    26. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, March.
    27. Dosi, Giovanni & Fagiolo, Giorgio & Roventini, Andrea, 2010. "Schumpeter meeting Keynes: A policy-friendly model of endogenous growth and business cycles," Journal of Economic Dynamics and Control, Elsevier, vol. 34(9), pages 1748-1767, September.
    28. Baptista, Rafa & Farmer, J. Doyne & Hinterschweiger, Marc & Low, Katie & Tang, Daniel & Uluc, Arzu, 2016. "Macroprudential policy in an agent-based model of the UK housing market," Bank of England working papers 619, Bank of England.
    29. Kukacka, Jiri & Barunik, Jozef, 2017. "Estimation of financial agent-based models with simulated maximum likelihood," Journal of Economic Dynamics and Control, Elsevier, vol. 85(C), pages 21-45.
    30. Guerini, Mattia & Moneta, Alessio, 2017. "A method for agent-based models validation," Journal of Economic Dynamics and Control, Elsevier, vol. 82(C), pages 125-141.
    31. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    32. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    33. Dosi, Giovanni & Fagiolo, Giorgio & Napoletano, Mauro & Roventini, Andrea & Treibich, Tania, 2015. "Fiscal and monetary policies in complex evolving economies," Journal of Economic Dynamics and Control, Elsevier, vol. 52(C), pages 166-189.
    34. Christophe Deissenberg & Sander van der Hoog & Herbert Dawid, 2008. "EURACE: A Massively Parallel Agent-Based Model of the European Economy," Working Papers halshs-00339756, HAL.
    35. Lux, Thomas, 2018. "Estimation of agent-based models using sequential Monte Carlo methods," Journal of Economic Dynamics and Control, Elsevier, vol. 91(C), pages 391-408.
    36. Caiani, Alessandro & Godin, Antoine & Caverzasi, Eugenio & Gallegati, Mauro & Kinsella, Stephen & Stiglitz, Joseph E., 2016. "Agent based-stock flow consistent macroeconomics: Towards a benchmark model," Journal of Economic Dynamics and Control, Elsevier, vol. 69(C), pages 375-408.
    37. Lamperti, Francesco, 2018. "An information theoretic criterion for empirical validation of simulation models," Econometrics and Statistics, Elsevier, vol. 5(C), pages 83-106.
    38. Giorgio Fagiolo & Alessio Moneta & Paul Windrum, 2007. "A Critical Guide to Empirical Validation of Agent-Based Models in Economics: Methodologies, Procedures, and Open Problems," Computational Economics, Springer;Society for Computational Economics, vol. 30(3), pages 195-226, October.
    39. Simone Alfarano & Thomas Lux & Friedrich Wagner, 2005. "Estimation of Agent-Based Models: The Case of an Asymmetric Herding Model," Computational Economics, Springer;Society for Computational Economics, vol. 26(1), pages 19-49, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kukacka, Jiri & Sacht, Stephen, 2021. "Estimation of Heuristic Switching in Behavioral Macroeconomic Models," Economics Working Papers 2021-01, Christian-Albrechts-University of Kiel, Department of Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    2. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    3. Platt, Donovan, 2020. "A comparison of economic agent-based model calibration methods," Journal of Economic Dynamics and Control, Elsevier, vol. 113(C).
    4. Lamperti, Francesco & Roventini, Andrea & Sani, Amir, 2018. "Agent-based model calibration using machine learning surrogates," Journal of Economic Dynamics and Control, Elsevier, vol. 90(C), pages 366-389.
    5. Giovanni Dosi & Andrea Roventini, 2019. "More is different ... and complex! the case for agent-based macroeconomics," Journal of Evolutionary Economics, Springer, vol. 29(1), pages 1-37, March.
    6. Andrea Vandin & Daniele Giachini & Francesco Lamperti & Francesca Chiaromonte, 2021. "Automated and Distributed Statistical Analysis of Economic Agent-Based Models," Papers 2102.05405, arXiv.org.
    7. Lamperti, Francesco, 2018. "An information theoretic criterion for empirical validation of simulation models," Econometrics and Statistics, Elsevier, vol. 5(C), pages 83-106.
    8. Donovan Platt, 2019. "A Comparison of Economic Agent-Based Model Calibration Methods," Papers 1902.05938, arXiv.org.
    9. Guerini, Mattia & Moneta, Alessio, 2017. "A method for agent-based models validation," Journal of Economic Dynamics and Control, Elsevier, vol. 82(C), pages 125-141.
    10. Andrea Vandin & Daniele Giachini & Francesco Lamperti & Francesca Chiaromonte, 2020. "Automated and Distributed Statistical Analysis of Economic Agent-Based Models," LEM Papers Series 2020/31, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    11. Francesco Lamperti, 2018. "Empirical validation of simulated models through the GSL-div: an illustrative application," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 13(1), pages 143-171, April.
    12. Guerini, Mattia & Napoletano, Mauro & Roventini, Andrea, 2018. "No man is an Island: The impact of heterogeneity and local interactions on macroeconomic dynamics," Economic Modelling, Elsevier, vol. 68(C), pages 82-95.
    13. Lamperti, F. & Dosi, G. & Napoletano, M. & Roventini, A. & Sapio, A., 2018. "Faraway, So Close: Coupled Climate and Economic Dynamics in an Agent-based Integrated Assessment Model," Ecological Economics, Elsevier, vol. 150(C), pages 315-339.
    14. Giorgio Fagiolo & Andrea Roventini, 2017. "Macroeconomic Policy in DSGE and Agent-Based Models Redux: New Developments and Challenges Ahead," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 20(1), pages 1-1.
    15. Kukacka, Jiri & Sacht, Stephen, 2021. "Estimation of Heuristic Switching in Behavioral Macroeconomic Models," Economics Working Papers 2021-01, Christian-Albrechts-University of Kiel, Department of Economics.
    16. Popoyan, Lilit & Napoletano, Mauro & Roventini, Andrea, 2020. "Winter is possibly not coming: Mitigating financial instability in an agent-based model with interbank market," Journal of Economic Dynamics and Control, Elsevier, vol. 117(C).
    17. Lamperti, F. & Dosi, G. & Napoletano, M. & Roventini, A. & Sapio, A., 2020. "Climate change and green transitions in an agent-based integrated assessment model," Technological Forecasting and Social Change, Elsevier, vol. 153(C).
    18. Francesco Lamperti, 2016. "Empirical Validation of Simulated Models through the GSL-div: an Illustrative Application," LEM Papers Series 2016/18, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    19. Francesco Lamperti & Valentina Bosetti & Andrea Roventini & Massimo Tavoni & Tania Treibich, 2021. "Three green financial policies to address climate risks," LEM Papers Series 2021/05, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    20. Delli Gatti, Domenico & Grazzini, Jakob, 2020. "Rising to the challenge: Bayesian estimation and forecasting techniques for macroeconomic Agent Based Models," Journal of Economic Behavior & Organization, Elsevier, vol. 178(C), pages 875-902.

    More about this item

    Keywords

    Model comparison; Agent-based models; Validation methods;
    All these keywords.

    JEL classification:

    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:dyncon:v:111:y:2020:i:c:s0165188919301927. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Nithya Sathishkumar). General contact details of provider: http://www.elsevier.com/locate/jedc .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.