This file is part of IDEAS, which uses RePEc data


[ Papers | Articles | Software | Books | Chapters | Authors | Institutions | JEL Classification | NEP reports | Search | New papers by email | Author registration | Rankings | Volunteers | FAQ | Blog | Help! ]

Empirical Validation of Agent Based Models: A Critical Survey

Author info | Abstract | Publisher info | Download info | Related research | Statistics
Author Info
Giorgio Fagiolo
Paul Windrum
Alessio Moneta

Additional information is available for the following registered author(s):

Abstract

This paper addresses the problem of finding the appropriate method for conducting empirical validation in agent-based (AB) models, which is often regarded as the Achilles’ heel of the AB approach to economic modelling. The paper has two objectives. First, to identify key issues facing AB economists engaged in empirical validation. Second, to critically appraise the extent to which alternative approaches deal with these issues. We identify a first set of issues that are common to both AB and neoclassical modellers and a second set of issues which are specific to AB modellers. This second set of issues is captured in a novel taxonomy, which takes into consideration the nature of the object under study, the goal of the analysis, the nature of the modelling assumptions, and the methodology of the analysis. Having identified the nature and causes of heterogeneity in empirical validation, we examine three important approaches to validation that have been developed in AB economics: indirect calibration, the Werker-Brenner approach, and the history-friendly approach. We also discuss a set of open questions within empirical validation. These include the trade-off between empirical support and tractability of findings, the issue of over-parameterisation, unconditional objects, counterfactuals, and the non-neutrality of data.

Download Info
To download:

If you experience problems downloading a file, check if you have the proper application to view it first. Information about this may be contained in the File-Format links below. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.lem.sssup.it/WPLem/files/2006-14.pdf
File Format: application/pdf
File Function:
Download Restriction: no

Publisher Info
Paper provided by Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy in its series LEM Papers Series with number 2006/14.

Download reference. The following formats are available: HTML (with abstract), plain text (with abstract), BibTeX, RIS (EndNote, RefMan, ProCite), ReDIF
Length:
Date of creation: 15 May 2006
Date of revision:
Handle: RePEc:ssa:lemwps:2006/14

Contact details of provider:
Postal: Piazza dei Martiri della Liberta, 33, 56127 Pisa
Phone: +39-50-883343
Fax: +39-50-883344
Email:
Web page: http://www.lem.sssup.it/
More information through EDIRC

For technical questions regarding this item, or to correct its listing, contact: ().

Related research
Keywords: Empirical validation; agent-based models; calibration; history-friendly modelling;

Other versions of this item:

This paper has been announced in the following NEP Reports: References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
  1. Luigi Marengo & Marc Willinger, 1997. "Alternative methodologies for modelling evolutionary dynamics: Introduction," Journal of Evolutionary Economics, Springer, vol. 7(4), pages 331-338. [Downloadable!] (restricted)
  2. Robert E. Lucas, Jr. & Thomas J. Sargent, 1979. "After Keynesian macroeconomics," Quarterly Review, Federal Reserve Bank of Minneapolis, issue Spr. [Downloadable!]
  3. Sims, Christopher A, 1980. "Macroeconomics and Reality," Econometrica, Econometric Society, vol. 48(1), pages 1-48, January. [Downloadable!] (restricted)
  4. Paul Downward & John H. Finch & John Ramsay, 2002. "Critical realism, empirical methods and inference: a critical discussion," Cambridge Journal of Economics, Oxford University Press, vol. 26(4), pages 481-500, July.
  5. Windrum,Paul, 2004. "Neo-Schumpeterian Simulation Models," Research Memoranda 002, Maastricht : MERIT, Maastricht Economic Research Institute on Innovation and Technology. [Downloadable!]
  6. Tesfatsion, Leigh, 2002. "Agent-Based Computational Economics: Growing Economies from the Bottom Up," Staff General Research Papers 5075, Iowa State University, Department of Economics.
  7. McCloskey, Donald N, 1983. "The Rhetoric of Economics," Journal of Economic Literature, American Economic Association, vol. 21(2), pages 481-517, June. [Downloadable!] (restricted)
  8. Silverberg, Gerald & Verspagen, Bart, 1994. "Collective Learning, Innovation and Growth in a Boundedly Rational, Evolutionary World," Journal of Evolutionary Economics, Springer, vol. 4(3), pages 207-26, September.
  9. Franco Malerba & Luigi Orsenigo, 2002. "Innovation and market structure in the dynamics of the pharmaceutical industry and biotechnology: towards a history-friendly model," Industrial and Corporate Change, Oxford University Press, vol. 11(4), pages 667-703, August.
  10. Claudia Werker & Thomas Brenner, 2004. "Empirical Calibration of Simulation Models," Papers on Economics and Evolution 2004-10, Max Planck Institute of Economics, Evolutionary Economics Group. [Downloadable!]
    Other versions:
  11. Herbert Dawid & Marc Reimann, 2005. "Evaluating Market Attractiveness: Individual Incentives Versus Industry Profitability," Computational Economics, Springer, vol. 24(4), pages 321-355, June. [Downloadable!] (restricted)
  12. Hoover, Kevin D, 1995. "Facts and Artifacts: Calibration and the Empirical Assessment of Real-Business-Cycle Models," Oxford Economic Papers, Oxford University Press, vol. 47(1), pages 24-44, January. [Downloadable!] (restricted)
  13. Lucas, Robert Jr, 1976. "Econometric policy evaluation: A critique," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 1(1), pages 19-46, January. [Downloadable!] (restricted)
  14. Alessio Moneta, 2005. "Causality in macroeconometrics: some considerations about reductionism and realism," Journal of Economic Methodology, Taylor and Francis Journals, vol. 12(3), pages 433-453, September. [Downloadable!] (restricted)
  15. Lane, David A, 1993. "Artificial Worlds and Economics, Part II," Journal of Evolutionary Economics, Springer, vol. 3(3), pages 177-97, August.
  16. Dosi, Giovanni, et al, 1995. " Learning, Market Selection and the Evolution of Industrial Structures," Small Business Economics, Springer, vol. 7(6), pages 411-36, December.
  17. Silverberg, Gerald & Lehnert, Doris, 1993. "Long waves and 'evolutionary chaos' in a simple Schumpeterian model of embodied technical change," Structural Change and Economic Dynamics, Elsevier, vol. 4(1), pages 9-37, June. [Downloadable!] (restricted)
  18. Tesfatsion, Leigh S. & Judd, Kenneth L., 2003. "Handbook of Computational Economics, Vol. 2: Agent-Based Computational Economics," Staff General Research Papers 10368, Iowa State University, Department of Economics. [Downloadable!]
  19. Fagiolo, Giorgio & Dosi, Giovanni, 2003. "Exploitation, exploration and innovation in a model of endogenous growth with locally interacting agents," Structural Change and Economic Dynamics, Elsevier, vol. 14(3), pages 237-273, September. [Downloadable!] (restricted)
    Other versions:
  20. Kirman, Alan, 1989. "The Intrinsic Limits of Modern Economic Theory: The Emperor Has No Clothes," Economic Journal, Royal Economic Society, vol. 99(395), pages 126-39, Supplemen. [Downloadable!] (restricted)
  21. Koesrindartoto, Deddy & Sun, Junjie & Tesfatsion, Leigh S., 2005. "An Agent-Based Computational Laboratory for Testing the Economic Reliability of Wholesale Power Market Designs," Staff General Research Papers 12388, Iowa State University, Department of Economics. [Downloadable!]
    Other versions:
  22. Kleijnen, J.P.C., 1997. "Experimental design for sensitivity analysis, optimization, and validation of simulation models," Discussion Paper 52, Tilburg University, Center for Economic Research. [Downloadable!]
  23. Silverberg, Gerald & Dosi, Giovanni & Orsenigo, Luigi, 1988. "Innovation, Diversity and Diffusion: A Self-organisation Model," Economic Journal, Royal Economic Society, vol. 98(393), pages 1032-54, December. [Downloadable!] (restricted)
  24. Kirman, Alan P, 1992. "Whom or What Does the Representative Individual Represent?," Journal of Economic Perspectives, American Economic Association, vol. 6(2), pages 117-36, Spring. [Downloadable!] (restricted)
  25. Fagiolo, Giorgio, 2005. "Endogenous neighborhood formation in a local coordination model with negative network externalities," Journal of Economic Dynamics and Control, Elsevier, vol. 29(1-2), pages 297-319, January. [Downloadable!] (restricted)
  26. Gatti, Domenico Delli & Guilmi, Corrado Di & Gaffeo, Edoardo & Giulioni, Gianfranco & Gallegati, Mauro & Palestrini, Antonio, 2005. "A new approach to business fluctuations: heterogeneous interacting agents, scaling laws and financial fragility," Journal of Economic Behavior & Organization, Elsevier, vol. 56(4), pages 489-512, April. [Downloadable!] (restricted)
  27. Paul Windrum & Chris Birchenhall, 2005. "Structural change in the presence of network externalities: a co-evolutionary model of technological successions," Journal of Evolutionary Economics, Springer, vol. 15(2), pages 123-148, January. [Downloadable!] (restricted)
    Other versions:
  28. Dominique Foray & Robin Cowan, 2002. "Evolutionary economics and the counterfactual threat: on the nature and role of counterfactual history as an empirical tool in economics," Journal of Evolutionary Economics, Springer, vol. 12(5), pages 539-562. [Downloadable!] (restricted)
  29. Roberto Leombruni, 2002. "The Methodological Status of Agent-Based Simulations," LABORatorio R. Revelli Working Papers Series 19, LABORatorio R. Revelli, Centre for Employment Studies. [Downloadable!]
  30. Chiaromonte, Francesca & Dosi, Giovanni, 1993. "Heterogeneity, competition, and macroeconomic dynamics," Structural Change and Economic Dynamics, Elsevier, vol. 4(1), pages 39-63, June. [Downloadable!] (restricted)
  31. Christiano, Lawrence J. & Eichenbaum, Martin & Evans, Charles L., 1999. "Monetary policy shocks: What have we learned and to what end?," Handbook of Macroeconomics, in: J. B. Taylor & M. Woodford (ed.), Handbook of Macroeconomics, edition 1, volume 1, chapter 2, pages 65-148 Elsevier. [Downloadable!] (restricted)
    Other versions:
  32. Dosi, Giovanni, 1988. "Sources, Procedures, and Microeconomic Effects of Innovation," Journal of Economic Literature, American Economic Association, vol. 26(3), pages 1120-71, September. [Downloadable!] (restricted)
  33. Paul Windrum, 2004. "Heterogeneous preferences and new innovation cycles in mature industries: the camera industry 1955-1974," Papers on Economics and Evolution 2004-18, Max Planck Institute of Economics, Evolutionary Economics Group.
  34. Lane, David A, 1993. "Artificial Worlds and Economics, Part I," Journal of Evolutionary Economics, Springer, vol. 3(2), pages 89-107, May.
  35. Alan Kirman, 1997. "The economy as an evolving network," Journal of Evolutionary Economics, Springer, vol. 7(4), pages 339-353. [Downloadable!] (restricted)
  36. Christopher A. Sims, 1982. "Policy Analysis with Econometric Models," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 13(1982-1), pages 107-164. [Downloadable!]
  37. Matteo Richiardi, 2003. "The Promises and Perils of Agent-Based Computational Economics," LABORatorio R. Revelli Working Papers Series 29, LABORatorio R. Revelli, Centre for Employment Studies. [Downloadable!]
    Other versions:
  38. Bianchi, Carlo & Cirillo, Pasquale & Gallegati, Mauro & Vagliasindi, Pietro A., 2008. "Validation in agent-based models: An investigation on the CATS model," Journal of Economic Behavior & Organization, Elsevier, vol. 67(3-4), pages 947-964, September. [Downloadable!] (restricted)
  39. Leigh TESFATSION, 1995. "How Economists Can Get Alife," Economic Report 37, Iowa State University Department of Economics. [Downloadable!]
  40. Delli Gatti, Domenico & Gallegati, Mauro & Giulioni, Gianfranco & Palestrini, Antonio, 2003. "Financial fragility, patterns of firms' entry and exit and aggregate dynamics," Journal of Economic Behavior & Organization, Elsevier, vol. 51(1), pages 79-97, May. [Downloadable!] (restricted)
    Other versions:
  41. Richard R. Nelson, 1995. "Recent Evolutionary Theorizing about Economic Change," Journal of Economic Literature, American Economic Association, vol. 33(1), pages 48-90, March. [Downloadable!] (restricted)
  42. Malerba, Franco, et al, 1999. "'History-Friendly' Models of Industry Evolution: The Computer Industry," Industrial and Corporate Change, Oxford University Press, vol. 8(1), pages 3-40, March.
  43. Thomas Brenner, 2004. "Agent Learning Representation - Advice in Modelling Economic Learning," Papers on Economics and Evolution 2004-16, Max Planck Institute of Economics, Evolutionary Economics Group.
    Other versions:
  44. Klein, Lawrence R, 1969. "Estimation on Interdependent Systems in Macroeconometrics," Econometrica, Econometric Society, vol. 37(2), pages 171-92, April. [Downloadable!] (restricted)
  45. Saviotti, Pier Paolo & Pyka, Andreas, 2004. "Economic development, qualitative change and employment creation," Structural Change and Economic Dynamics, Elsevier, vol. 15(3), pages 265-287, September. [Downloadable!] (restricted)
    Other versions:
  46. Kydland, Finn E & Prescott, Edward C, 1982. "Time to Build and Aggregate Fluctuations," Econometrica, Econometric Society, vol. 50(6), pages 1345-70, November. [Downloadable!] (restricted)
    Other versions:
  47. Favero, Carlo A. & Scott, Alasdair, 2003. "Applied Macroeconometrics," Macroeconomic Dynamics, Cambridge University Press, vol. 7(02), pages 313-315, April. [Downloadable!]
  48. Brock, W A, 1999. "Scaling in Economics: A Reader's Guide," Industrial and Corporate Change, Oxford University Press, vol. 8(3), pages 409-46, September.
  49. Johann Peter Murmann & Thomas Brenner, 2003. "The Use of Simulations in Developing Robust Knowledge about Causal Processes: Methodological Considerations and an Application to Industrial Evolution," Computing in Economics and Finance 2003 66, Society for Computational Economics.
  50. Windrum, Paul & Birchenhall, Chris, 1998. "Is product life cycle theory a special case? Dominant designs and the emergence of market niches through coevolutionary-learning," Structural Change and Economic Dynamics, Elsevier, vol. 9(1), pages 109-134, March. [Downloadable!] (restricted)
  51. Silverberg, Gerald & Verspagen, Bart, 2005. "A percolation model of innovation in complex technology spaces," Journal of Economic Dynamics and Control, Elsevier, vol. 29(1-2), pages 225-244, January. [Downloadable!] (restricted)
    Other versions:
  52. Matteo Richiardi & Roberto Leombruni & Nicole J. Saam & Michele Sonnessa, 2006. "A Common Protocol for Agent-Based Social Simulation," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 9. [Downloadable!]
    Other versions:
  53. Boulding, K E, 1991. "What Is Evolutionary Economics?," Journal of Evolutionary Economics, Springer, vol. 1(1), pages 9-17, January.
  54. Windrum,Paul, 1999. "Simulation models of technological innovation: A Review," Research Memoranda 005, Maastricht : MERIT, Maastricht Economic Research Institute on Innovation and Technology. [Downloadable!]
  55. Liebowitz, S J & Margolis, Stephen E, 1990. "The Fable of the Keys," Journal of Law & Economics, University of Chicago Press, vol. 33(1), pages 1-25, April.
  56. Hoover, Kevin D, 1994. "Econometrics as Observation: The Lucas Critique and the Nature of Econometric Inference," Journal of Economic Methodology, Taylor and Francis Journals, vol. 1(1), pages 65-80, June.
  57. Hans M. Amman & David A. Kendrick, . "Computational Economics," Online economics textbooks, SUNY-Oswego, Department of Economics, number comp1, March. [Downloadable!]
  58. Andreas Pyka & Giorgio Fagiolo, 2005. "Agent-Based Modelling: A Methodology for Neo-Schumpeterian Economics," Discussion Paper Series 272, Universitaet Augsburg, Institute for Economics. [Downloadable!]
  59. Dawid, Herbert, 2006. "Agent-based Models of Innovation and Technological Change," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 25, pages 1235-1272 Elsevier. [Downloadable!] (restricted)
  60. G. Silverberg & B. Verspagen, 1995. "Evolutionary Theorizing on Economic Growth," Working Papers wp95078, International Institute for Applied Systems Analysis. [Downloadable!]
Full references

Cited by:
(explanations, Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.)

  1. Robert Marks, 2007. "Validating Simulation Models: A General Framework and Four Applied Examples," Computational Economics, Springer, vol. 30(3), pages 265-290, October. [Downloadable!] (restricted)
Statistics
Access and download statistics

Did you know? You can create a compilation of all publications of a group of people, say alumni of a program, your students or memers of an association.

This page was last updated on 2009-11-13.

This information is provided to you by IDEAS at the Department of Economics, College of Liberal Arts and Sciences, University of Connecticut using RePEc data on a server sponsored by the Society for Economic Dynamics.