IDEAS home Printed from https://ideas.repec.org/p/thk/wpaper/42.html
   My bibliography  Save this paper

A Method for Agent-Based Models Validation

Author

Listed:
  • Mattia Guerini

    () (Institute of Economics, Scuola Superiore Sant'Anna, Pisa, Italy)

  • Alessio Moneta

    () (Institute of Economics, Scuola Superiore Sant'Anna, Pisa, Italy)

Abstract

This paper proposes a new method to empirically validate simulation models that generate artificial time series data comparable with real-world data. The approach is based on comparing structures of vector autoregression models that are estimated from both artificial and real-world data by means of causal search algorithms. This relatively simple procedure is able to tackle both the problem of confronting theoretical simulation models with the data and the problem of comparing different models in terms of their empirical reliability. The paper also provides an application of the validation procedure to the Dosi et al. (2015) macro-model.

Suggested Citation

  • Mattia Guerini & Alessio Moneta, "undated". "A Method for Agent-Based Models Validation," Working Papers Series 42, Institute for New Economic Thinking.
  • Handle: RePEc:thk:wpaper:42
    as

    Download full text from publisher

    File URL: http://ineteconomics.org/uploads/papers/WP_42-Moneta.pdf
    File Function: First version, 2016
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. S. Alfarano & T. Lux & F. Wagner, 2007. "Empirical validation of stochastic models of interacting agents," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 55(2), pages 183-187, January.
    2. Blanchard, Olivier Jean & Quah, Danny, 1989. "The Dynamic Effects of Aggregate Demand and Supply Disturbances," American Economic Review, American Economic Association, vol. 79(4), pages 655-673, September.
    3. Marco Del Negro & Frank Schorfheide, 2006. "How good is what you've got? DSGE-VAR as a toolkit for evaluating DSGE models," Economic Review, Federal Reserve Bank of Atlanta, issue Q 2, pages 21-37.
    4. Barde, Sylvain, 2016. "Direct comparison of agent-based models of herding in financial markets," Journal of Economic Dynamics and Control, Elsevier, vol. 73(C), pages 329-353.
    5. Alessio Moneta, 2008. "Graphical causal models and VARs: an empirical assessment of the real business cycles hypothesis," Empirical Economics, Springer, vol. 35(2), pages 275-300, September.
    6. Recchioni, Maria Cristina & Tedeschi, Gabriele & Gallegati, Mauro, 2015. "A calibration procedure for analyzing stock price dynamics in an agent-based framework," Journal of Economic Dynamics and Control, Elsevier, vol. 60(C), pages 1-25.
    7. Giorgio Fagiolo & Andrea Roventini, 2017. "Macroeconomic Policy in DSGE and Agent-Based Models Redux: New Developments and Challenges Ahead," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 20(1), pages 1-1.
    8. Martin Fukac & Adrian Pagan, 2006. "Issues in Adopting DSGE Models for Use in the Policy Process," Working Papers 2006/6, Czech National Bank, Research Department.
    9. Canova, Fabio & Sala, Luca, 2009. "Back to square one: Identification issues in DSGE models," Journal of Monetary Economics, Elsevier, vol. 56(4), pages 431-449, May.
    10. Giorgio Fagiolo & Andrea Roventini, 2012. "Macroeconomic Policy in DSGE and Agent-Based Models," Revue de l'OFCE, Presses de Sciences-Po, vol. 0(5), pages 67-116.
    11. Del Negro, Marco & Schorfheide, Frank, 2008. "Forming priors for DSGE models (and how it affects the assessment of nominal rigidities)," Journal of Monetary Economics, Elsevier, vol. 55(7), pages 1191-1208, October.
    12. Finn E. Kydland & Edward C. Prescott, 1996. "The Computational Experiment: An Econometric Tool," Journal of Economic Perspectives, American Economic Association, vol. 10(1), pages 69-85, Winter.
    13. Jean-Luc Gaffard & Mauro Napoletano, 2012. "Improving the toolbox. New advances in Agent-based and Computational Models," Revue de l'OFCE, Presses de Sciences-Po, vol. 0(5), pages 7-13.
    14. Johansen, Soren & Juselius, Katarina, 1990. "Maximum Likelihood Estimation and Inference on Cointegration--With Applications to the Demand for Money," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 52(2), pages 169-210, May.
    15. Matthias Lengnick & Hans-Werner Wohltmann, 2013. "Agent-based financial markets and New Keynesian macroeconomics: a synthesis," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 8(1), pages 1-32, April.
    16. Sylvain Barde, 2015. "Direct calibration and comparison of agent-based herding models of financial markets," Studies in Economics 1507, School of Economics, University of Kent.
    17. Francesco Lamperti, 2016. "Empirical Validation of Simulated Models through the GSL-div: an Illustrative Application," LEM Papers Series 2016/18, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    18. Dosi, Giovanni & Fagiolo, Giorgio & Napoletano, Mauro & Roventini, Andrea, 2013. "Income distribution, credit and fiscal policies in an agent-based Keynesian model," Journal of Economic Dynamics and Control, Elsevier, vol. 37(8), pages 1598-1625.
    19. Kydland, Finn E & Prescott, Edward C, 1982. "Time to Build and Aggregate Fluctuations," Econometrica, Econometric Society, vol. 50(6), pages 1345-1370, November.
    20. Katarina Juselius, 2011. "On the Role of Theory and Evidence in Macroeconomics," Chapters,in: The Elgar Companion to Recent Economic Methodology, chapter 17 Edward Elgar Publishing.
    21. Blanchard, Olivier Jean, 1989. "A Traditional Interpretation of Macroeconomic Fluctuations," American Economic Review, American Economic Association, vol. 79(5), pages 1146-1164, December.
    22. Andrew G. Haldane & Vasileios Madouros, 2012. "The dog and the frisbee," Proceedings - Economic Policy Symposium - Jackson Hole, Federal Reserve Bank of Kansas City, pages 109-159.
    23. Dosi, Giovanni & Fagiolo, Giorgio & Roventini, Andrea, 2010. "Schumpeter meeting Keynes: A policy-friendly model of endogenous growth and business cycles," Journal of Economic Dynamics and Control, Elsevier, vol. 34(9), pages 1748-1767, September.
    24. Isabelle Salle & Murat Yıldızoğlu, 2014. "Efficient Sampling and Meta-Modeling for Computational Economic Models," Computational Economics, Springer;Society for Computational Economics, vol. 44(4), pages 507-536, December.
    25. Matthew Shapiro & Mark Watson, 1988. "Sources of Business Cycles Fluctuations," NBER Chapters,in: NBER Macroeconomics Annual 1988, Volume 3, pages 111-156 National Bureau of Economic Research, Inc.
    26. David A. Bessler & Seongpyo Lee, 2002. "Money and prices: U.S. Data 1869-1914 (A study with directed graphs)," Empirical Economics, Springer, vol. 27(3), pages 427-446.
    27. Dosi, Giovanni & Fagiolo, Giorgio & Napoletano, Mauro & Roventini, Andrea & Treibich, Tania, 2015. "Fiscal and monetary policies in complex evolving economies," Journal of Economic Dynamics and Control, Elsevier, vol. 52(C), pages 166-189.
    28. Sims, Christopher A, 1980. "Macroeconomics and Reality," Econometrica, Econometric Society, vol. 48(1), pages 1-48, January.
    29. Ireland, Peter N., 2004. "A method for taking models to the data," Journal of Economic Dynamics and Control, Elsevier, vol. 28(6), pages 1205-1226, March.
    30. Kirman, Alan, 1989. "The Intrinsic Limits of Modern Economic Theory: The Emperor Has No Clothes," Economic Journal, Royal Economic Society, vol. 99(395), pages 126-139, Supplemen.
    31. Kirman, Alan, 2016. "Ants And Nonoptimal Self-Organization: Lessons For Macroeconomics," Macroeconomic Dynamics, Cambridge University Press, vol. 20(02), pages 601-621, March.
    32. Lux, Thomas, 2012. "Estimation of an agent-based model of investor sentiment formation in financial markets," Journal of Economic Dynamics and Control, Elsevier, vol. 36(8), pages 1284-1302.
    33. Marco Del Negro & Frank Schorfheide, 2004. "Priors from General Equilibrium Models for VARS," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 45(2), pages 643-673, May.
    34. Guerini, Mattia & Napoletano, Mauro & Roventini, Andrea, 2018. "No man is an Island: The impact of heterogeneity and local interactions on macroeconomic dynamics," Economic Modelling, Elsevier, vol. 68(C), pages 82-95.
    35. Bernanke, Ben S., 1986. "Alternative explanations of the money-income correlation," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 25(1), pages 49-99, January.
    36. Tommaso Ciarli, 2012. "Structural Interactions and Long Run Growth. An Application of Experimental Design to Agent Based Models," Revue de l'OFCE, Presses de Sciences-Po, vol. 0(5), pages 295-345.
    37. Simone Alfarano & Thomas Lux & Friedrich Wagner, 2005. "Estimation of Agent-Based Models: The Case of an Asymmetric Herding Model," Computational Economics, Springer;Society for Computational Economics, vol. 26(1), pages 19-49, August.
    38. Alessio Moneta & Doris Entner & Patrik O. Hoyer & Alex Coad, 2013. "Causal Inference by Independent Component Analysis: Theory and Applications," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 75(5), pages 705-730, October.
    39. Gilli, M. & Winker, P., 2003. "A global optimization heuristic for estimating agent based models," Computational Statistics & Data Analysis, Elsevier, vol. 42(3), pages 299-312, March.
    40. Selva Demiralp & Kevin D. Hoover, 2003. "Searching for the Causal Structure of a Vector Autoregression," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 65(s1), pages 745-767, December.
    41. Alan P. Kirman, 1992. "Whom or What Does the Representative Individual Represent?," Journal of Economic Perspectives, American Economic Association, vol. 6(2), pages 117-136, Spring.
    42. Francesco Lamperti, 2015. "An Information Theoretic Criterion for Empirical Validation of Time Series Models," LEM Papers Series 2015/02, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    43. Alfarano, Simone & Lux, Thomas & Wagner, Friedrich, 2006. "Estimation of a simple agent-based model of financial markets: An application to Australian stock and foreign exchange data," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 370(1), pages 38-42.
    44. Bianchi, Carlo & Cirillo, Pasquale & Gallegati, Mauro & Vagliasindi, Pietro A., 2008. "Validation in agent-based models: An investigation on the CATS model," Journal of Economic Behavior & Organization, Elsevier, vol. 67(3-4), pages 947-964, September.
    45. Carlo Bianchi & Pasquale Cirillo & Mauro Gallegati & Pietro Vagliasindi, 2007. "Validating and Calibrating Agent-Based Models: A Case Study," Computational Economics, Springer;Society for Computational Economics, vol. 30(3), pages 245-264, October.
    46. Grazzini, Jakob & Richiardi, Matteo, 2015. "Estimation of ergodic agent-based models by simulated minimum distance," Journal of Economic Dynamics and Control, Elsevier, vol. 51(C), pages 148-165.
    47. Granger, C. W. J., 1980. "Testing for causality : A personal viewpoint," Journal of Economic Dynamics and Control, Elsevier, vol. 2(1), pages 329-352, May.
    48. Lucas, Robert Jr, 1976. "Econometric policy evaluation: A critique," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 1(1), pages 19-46, January.
    49. James J. Heckman, 2000. "Causal Parameters and Policy Analysis in Economics: A Twentieth Century Retrospective," The Quarterly Journal of Economics, Oxford University Press, vol. 115(1), pages 45-97.
    50. Jakob Grazzini, 2012. "Analysis of the Emergent Properties: Stationarity and Ergodicity," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 15(2), pages 1-7.
    51. Giorgio Fagiolo & Alessio Moneta & Paul Windrum, 2007. "A Critical Guide to Empirical Validation of Agent-Based Models in Economics: Methodologies, Procedures, and Open Problems," Computational Economics, Springer;Society for Computational Economics, vol. 30(3), pages 195-226, October.
    52. Hoover, Kevin D., 2004. "Lost Causes," Journal of the History of Economic Thought, Cambridge University Press, vol. 26(02), pages 149-164, June.
    53. Blake LeBaron & Leigh Tesfatsion, 2008. "Modeling Macroeconomies as Open-Ended Dynamic Systems of Interacting Agents," American Economic Review, American Economic Association, vol. 98(2), pages 246-250, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Giorgio Fagiolo & Andrea Roventini, 2017. "Macroeconomic Policy in DSGE and Agent-Based Models Redux: New Developments and Challenges Ahead," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 20(1), pages 1-1.
    2. Francesco Lamperti & Giovanni Dosi & Mauro Napoletano & Andrea Roventini & Sandro Sapio, 2017. "Faraway, so close : coupled climate and economic dynamics in an agent-based integrated assessment model," Sciences Po publications info:hdl:2441/4hs7liq1f49, Sciences Po.
    3. Francesco Lamperti & Andrea Roventini & Amir Sani, 2017. "Agent-Based Model Calibration using Machine Learning Surrogates," Sciences Po publications 2017-09, Sciences Po.
    4. Grazzini, Jakob & Richiardi, Matteo G. & Tsionas, Mike, 2017. "Bayesian estimation of agent-based models," Journal of Economic Dynamics and Control, Elsevier, vol. 77(C), pages 26-47.
    5. Balint, T. & Lamperti, F. & Mandel, A. & Napoletano, M. & Roventini, A. & Sapio, A., 2017. "Complexity and the Economics of Climate Change: A Survey and a Look Forward," Ecological Economics, Elsevier, vol. 138(C), pages 252-265.
    6. Francesco Lamperti, 2016. "Empirical Validation of Simulated Models through the GSL-div: an Illustrative Application," LEM Papers Series 2016/18, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    7. Giovanni Dosi & Marcelo C. Pereira & Andrea Roventini & Maria Enrica Virgillito, 2016. "The Effects of Labour Market Reforms upon Unemployment and Income Inequalities: an Agent Based Model," Sciences Po publications 2016-24, Sciences Po.
    8. repec:gam:jsusta:v:10:y:2018:i:4:p:998-:d:138473 is not listed on IDEAS
    9. Guerini, Mattia & Napoletano, Mauro & Roventini, Andrea, 2018. "No man is an Island: The impact of heterogeneity and local interactions on macroeconomic dynamics," Economic Modelling, Elsevier, vol. 68(C), pages 82-95.
    10. Giovanni Dosi & Andrea Roventini & Emanuele Russo, 2017. "Endogenous growth and global divergence in a multi-country agent-based model," LEM Papers Series 2017/32, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    11. Mauro Napoletano, 2017. "A Short Walk on the Wild Side: Agent-Based Models and their Implications for Macroeconomic Analysis," GREDEG Working Papers 2017-40, Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), University of Nice Sophia Antipolis.
    12. repec:eee:ecolec:v:150:y:2018:i:c:p:315-339 is not listed on IDEAS
    13. Sylvain Barde & Sander van der Hoog, 2017. "An empirical validation protocol for large-scale agent-based models," Studies in Economics 1712, School of Economics, University of Kent.
    14. Hazan, Aurélien, 2017. "Volume of the steady-state space of financial flows in a monetary stock-flow-consistent model," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 473(C), pages 589-602.
    15. Giorgio Fagiolo & Mattia Guerini & Francesco Lamperti & Alessio Moneta & Andrea Roventini, 2017. "Validation of Agent-Based Models in Economics and Finance," LEM Papers Series 2017/23, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    16. Gräbner, Claudius, 2016. "From realism to instrumentalism - and back? Methodological implications of changes in the epistemology of economics," MPRA Paper 71933, University Library of Munich, Germany.
    17. repec:eee:ecosta:v:5:y:2018:i:c:p:83-106 is not listed on IDEAS

    More about this item

    Keywords

    Models validation; Agent-Based models; Causality; Structural Vector Autoregressions;

    JEL classification:

    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • E37 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Forecasting and Simulation: Models and Applications

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:thk:wpaper:42. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Pia Malaney) or (Rebekah McClure). General contact details of provider: http://edirc.repec.org/data/inetnus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.