IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Log in (now much improved!) to save this paper

Empirical Validation of Agent Based Models: A Critical Survey

  • Giorgio Fagiolo
  • Paul Windrum
  • Alessio Moneta

This paper addresses the problem of finding the appropriate method for conducting empirical validation in agent-based (AB) models, which is often regarded as the Achilles’ heel of the AB approach to economic modelling. The paper has two objectives. First, to identify key issues facing AB economists engaged in empirical validation. Second, to critically appraise the extent to which alternative approaches deal with these issues. We identify a first set of issues that are common to both AB and neoclassical modellers and a second set of issues which are specific to AB modellers. This second set of issues is captured in a novel taxonomy, which takes into consideration the nature of the object under study, the goal of the analysis, the nature of the modelling assumptions, and the methodology of the analysis. Having identified the nature and causes of heterogeneity in empirical validation, we examine three important approaches to validation that have been developed in AB economics: indirect calibration, the Werker-Brenner approach, and the history-friendly approach. We also discuss a set of open questions within empirical validation. These include the trade-off between empirical support and tractability of findings, the issue of over-parameterisation, unconditional objects, counterfactuals, and the non-neutrality of data.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.lem.sssup.it/WPLem/files/2006-14.pdf
Download Restriction: no

Paper provided by Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy in its series LEM Papers Series with number 2006/14.

as
in new window

Length:
Date of creation: 15 May 2006
Date of revision:
Handle: RePEc:ssa:lemwps:2006/14
Contact details of provider: Postal:
Piazza dei Martiri della Liberta, 33, 56127 Pisa

Phone: +39-50-883343
Fax: +39-50-883344
Web page: http://www.lem.sssup.it/
Email:


More information through EDIRC

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Liebowitz, S J & Margolis, Stephen E, 1990. "The Fable of the Keys," Journal of Law and Economics, University of Chicago Press, vol. 33(1), pages 1-25, April.
  2. Dosi, Giovanni, 1988. "Sources, Procedures, and Microeconomic Effects of Innovation," Journal of Economic Literature, American Economic Association, vol. 26(3), pages 1120-71, September.
  3. Joshua M. Epstein & Robert L. Axtell, 1996. "Growing Artificial Societies: Social Science from the Bottom Up," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262550253.
  4. Robert E. Lucas, Jr. & Thomas J. Sargent, 1979. "After Keynesian macroeconomics," Quarterly Review, Federal Reserve Bank of Minneapolis, issue Spr.
  5. Luigi Marengo & Marc Willinger, 1997. "Alternative methodologies for modelling evolutionary dynamics: Introduction," Journal of Evolutionary Economics, Springer, vol. 7(4), pages 331-338.
  6. Lucas, Robert Jr, 1976. "Econometric policy evaluation: A critique," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 1(1), pages 19-46, January.
  7. Hansen, Lars Peter & Sargent, Thomas J., 1980. "Formulating and estimating dynamic linear rational expectations models," Journal of Economic Dynamics and Control, Elsevier, vol. 2(1), pages 7-46, May.
  8. Malerba, Franco, et al, 1999. "'History-Friendly' Models of Industry Evolution: The Computer Industry," Industrial and Corporate Change, Oxford University Press, vol. 8(1), pages 3-40, March.
  9. Paul Windrum, 2004. "Heterogeneous preferences and new innovation cycles in mature industries: the camera industry 1955-1974," Papers on Economics and Evolution 2004-18, Philipps University Marburg, Department of Geography.
  10. Klein, Lawrence R, 1969. "Estimation on Interdependent Systems in Macroeconometrics," Econometrica, Econometric Society, vol. 37(2), pages 171-92, April.
  11. Fagiolo, Giorgio & Dosi, Giovanni, 2003. "Exploitation, exploration and innovation in a model of endogenous growth with locally interacting agents," Structural Change and Economic Dynamics, Elsevier, vol. 14(3), pages 237-273, September.
  12. G. Silverberg & B. Verspagen, 1995. "Evolutionary Theorizing on Economic Growth," Working Papers wp95078, International Institute for Applied Systems Analysis.
  13. Pier Paolo Saviotti & Andreas Pyka, 2002. "Economic Development, Qualitative Change And Employment Creation," Computing in Economics and Finance 2002 362, Society for Computational Economics.
  14. Thomas Brenner & Claudia Werker, 2004. "Empirical Calibration of Simulation Models," Computing in Economics and Finance 2004 89, Society for Computational Economics.
  15. Richard R. Nelson, 1995. "Recent Evolutionary Theorizing about Economic Change," Journal of Economic Literature, American Economic Association, vol. 33(1), pages 48-90, March.
  16. Windrum Paul, 1999. "Simulation models of technological innovation: A Review," Research Memorandum 005, Maastricht University, Maastricht Economic Research Institute on Innovation and Technology (MERIT).
  17. Chiaromonte, Francesca & Dosi, Giovanni, 1993. "Heterogeneity, competition, and macroeconomic dynamics," Structural Change and Economic Dynamics, Elsevier, vol. 4(1), pages 39-63, June.
  18. Koesrindartoto, Deddy P. & Sun, Junjie & Tesfatsion, Leigh, 2005. "An Agent-Based Computational Laboratory for Testing the Economic Reliability of Wholesale Power Market Designs," Staff General Research Papers 12388, Iowa State University, Department of Economics.
  19. Matteo Richiardi, 2003. "The Promises and Perils of Agent-Based Computational Economics," LABORatorio R. Revelli Working Papers Series 29, LABORatorio R. Revelli, Centre for Employment Studies.
  20. Gatti, Domenico Delli & Guilmi, Corrado Di & Gaffeo, Edoardo & Giulioni, Gianfranco & Gallegati, Mauro & Palestrini, Antonio, 2005. "A new approach to business fluctuations: heterogeneous interacting agents, scaling laws and financial fragility," Journal of Economic Behavior & Organization, Elsevier, vol. 56(4), pages 489-512, April.
  21. Brock, W A, 1999. "Scaling in Economics: A Reader's Guide," Industrial and Corporate Change, Oxford University Press, vol. 8(3), pages 409-46, September.
  22. Fagiolo, Giorgio, 2005. "Endogenous neighborhood formation in a local coordination model with negative network externalities," Journal of Economic Dynamics and Control, Elsevier, vol. 29(1-2), pages 297-319, January.
  23. Domenico Delli Gatti, Mauro Gallegati, Gianfranco Giulioni, Antonio Palestrini, -DISCUSSANT: Thomas Brenner, 2000. "Financial Fragility, Patterns Of Firms' Entry And Exit And Aggregate Dynamics," Computing in Economics and Finance 2000 282, Society for Computational Economics.
  24. Kydland, Finn E & Prescott, Edward C, 1982. "Time to Build and Aggregate Fluctuations," Econometrica, Econometric Society, vol. 50(6), pages 1345-70, November.
  25. Leigh TESFATSION, 1995. "How Economists Can Get Alife," Economic Report 37, Iowa State University Department of Economics.
  26. Kevin Hoover, 1994. "Econometrics as observation: the Lucas critique and the nature of econometric inference," Journal of Economic Methodology, Taylor & Francis Journals, vol. 1(1), pages 65-80.
  27. Alessio Moneta, 2005. "Causality in macroeconometrics: some considerations about reductionism and realism," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(3), pages 433-453.
  28. Alan P. Kirman, 1992. "Whom or What Does the Representative Individual Represent?," Journal of Economic Perspectives, American Economic Association, vol. 6(2), pages 117-136, Spring.
  29. Lane, David A, 1993. "Artificial Worlds and Economics, Part I," Journal of Evolutionary Economics, Springer, vol. 3(2), pages 89-107, May.
  30. Bianchi, Carlo & Cirillo, Pasquale & Gallegati, Mauro & Vagliasindi, Pietro A., 2008. "Validation in agent-based models: An investigation on the CATS model," Journal of Economic Behavior & Organization, Elsevier, vol. 67(3-4), pages 947-964, September.
  31. Christiano, Lawrence J. & Eichenbaum, Martin & Evans, Charles L., 1999. "Monetary policy shocks: What have we learned and to what end?," Handbook of Macroeconomics, in: J. B. Taylor & M. Woodford (ed.), Handbook of Macroeconomics, edition 1, volume 1, chapter 2, pages 65-148 Elsevier.
  32. Dominique Foray & Robin Cowan, 2002. "Evolutionary economics and the counterfactual threat: on the nature and role of counterfactual history as an empirical tool in economics," Journal of Evolutionary Economics, Springer, vol. 12(5), pages 539-562.
  33. Sims, Christopher A, 1980. "Macroeconomics and Reality," Econometrica, Econometric Society, vol. 48(1), pages 1-48, January.
  34. Franco Malerba & Luigi Orsenigo, 2002. "Innovation and market structure in the dynamics of the pharmaceutical industry and biotechnology: towards a history-friendly model," Industrial and Corporate Change, Oxford University Press, vol. 11(4), pages 667-703, August.
  35. Gerald Silverberg & Bart Verspagen, 2002. "A Percolation Model of Innovation in Complex Technology Spaces," Computing in Economics and Finance 2002 24, Society for Computational Economics.
  36. Christopher A. Sims, 1982. "Policy Analysis with Econometric Models," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 13(1), pages 107-164.
  37. Alan Kirman, 1997. "The economy as an evolving network," Journal of Evolutionary Economics, Springer, vol. 7(4), pages 339-353.
  38. Uskali Maki, 2005. "Models are experiments, experiments are models," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(2), pages 303-315.
  39. Paul Downward & John H. Finch & John Ramsay, 2002. "Critical realism, empirical methods and inference: a critical discussion," Cambridge Journal of Economics, Oxford University Press, vol. 26(4), pages 481-500, July.
  40. Hicks, J. R., 1979. "Critical Essays in Monetary Theory," OUP Catalogue, Oxford University Press, number 9780198284239, June.
  41. Roberto Leombruni, 2002. "The Methodological Status of Agent-Based Simulations," LABORatorio R. Revelli Working Papers Series 19, LABORatorio R. Revelli, Centre for Employment Studies.
  42. Roberto Leombruni & Matteo Richiardi & Nicole J. Saam & Michele Sonnessa, 2005. "A Common Protocol for Agent-Based Social Simulation," LABORatorio R. Revelli Working Papers Series 47, LABORatorio R. Revelli, Centre for Employment Studies.
  43. Favero, Carlo A., 2001. "Applied Macroeconometrics," OUP Catalogue, Oxford University Press, number 9780198296850, June.
  44. Tesfatsion, Leigh & Judd, Kenneth L., 2006. "Handbook of Computational Economics, Vol. 2: Agent-Based Computational Economics," Staff General Research Papers 10368, Iowa State University, Department of Economics.
  45. Thomas Brenner, 2004. "Agent Learning Representation - Advice in Modelling Economic Learning," Papers on Economics and Evolution 2004-16, Philipps University Marburg, Department of Geography.
  46. Hoover, Kevin D, 1995. "Facts and Artifacts: Calibration and the Empirical Assessment of Real-Business-Cycle Models," Oxford Economic Papers, Oxford University Press, vol. 47(1), pages 24-44, January.
  47. Paul Windrum & Chris Birchenhall, 2005. "Structural change in the presence of network externalities: a co-evolutionary model of technological successions," Journal of Evolutionary Economics, Springer, vol. 15(2), pages 123-148, January.
  48. Andreas Pyka & Giorgio Fagiolo, 2005. "Agent-Based Modelling: A Methodology for Neo-Schumpeterian Economics," Discussion Paper Series 272, Universitaet Augsburg, Institute for Economics.
  49. Forni, Mario & Lippi, Marco, 1997. "Aggregation and the Microfoundations of Dynamic Macroeconomics," OUP Catalogue, Oxford University Press, number 9780198288008, June.
  50. Herbert Dawid & Marc Reimann, 2005. "Evaluating Market Attractiveness: Individual Incentives Versus Industry Profitability," Computational Economics, Society for Computational Economics, vol. 24(4), pages 321-355, June.
  51. Johann Peter Murmann & Thomas Brenner, 2003. "The Use of Simulations in Developing Robust Knowledge about Causal Processes: Methodological Considerations and an Application to Industrial Evolution," Computing in Economics and Finance 2003 66, Society for Computational Economics.
  52. Keuzenkamp,Hugo A., 2000. "Probability, Econometrics and Truth," Cambridge Books, Cambridge University Press, number 9780521553599, January.
  53. Hendry,David F. & Morgan,Mary S., 1997. "The Foundations of Econometric Analysis," Cambridge Books, Cambridge University Press, number 9780521588706, January.
  54. Silverberg, Gerald & Verspagen, Bart, 1994. "Collective Learning, Innovation and Growth in a Boundedly Rational, Evolutionary World," Journal of Evolutionary Economics, Springer, vol. 4(3), pages 207-26, September.
  55. Dawid, Herbert, 2006. "Agent-based Models of Innovation and Technological Change," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 25, pages 1235-1272 Elsevier.
  56. Silverberg, Gerald & Lehnert, Doris, 1993. "Long waves and 'evolutionary chaos' in a simple Schumpeterian model of embodied technical change," Structural Change and Economic Dynamics, Elsevier, vol. 4(1), pages 9-37, June.
  57. Boulding, K E, 1991. "What Is Evolutionary Economics?," Journal of Evolutionary Economics, Springer, vol. 1(1), pages 9-17, January.
  58. Windrum, Paul & Birchenhall, Chris, 1998. "Is product life cycle theory a special case? Dominant designs and the emergence of market niches through coevolutionary-learning," Structural Change and Economic Dynamics, Elsevier, vol. 9(1), pages 109-134, March.
  59. Silverberg, Gerald & Dosi, Giovanni & Orsenigo, Luigi, 1988. "Innovation, Diversity and Diffusion: A Self-organisation Model," Economic Journal, Royal Economic Society, vol. 98(393), pages 1032-54, December.
  60. Lane, David A, 1993. "Artificial Worlds and Economics, Part II," Journal of Evolutionary Economics, Springer, vol. 3(3), pages 177-97, August.
  61. Tesfatsion, Leigh S., 2002. "Agent-Based Computational Economics: Growing Economies from the Bottom Up," Staff General Research Papers 5075, Iowa State University, Department of Economics.
  62. McCloskey, Donald N, 1983. "The Rhetoric of Economics," Journal of Economic Literature, American Economic Association, vol. 21(2), pages 481-517, June.
  63. Hans M. Amman & David A. Kendrick, . "Computational Economics," Online economics textbooks, SUNY-Oswego, Department of Economics, number comp1.
  64. Kirman, Alan, 1989. "The Intrinsic Limits of Modern Economic Theory: The Emperor Has No Clothes," Economic Journal, Royal Economic Society, vol. 99(395), pages 126-39, Supplemen.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:ssa:lemwps:2006/14. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ()

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.