IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Log in (now much improved!) to save this paper

Empirical Validation of Agent Based Models: A Critical Survey

Listed author(s):
  • Giorgio Fagiolo
  • Paul Windrum
  • Alessio Moneta

This paper addresses the problem of finding the appropriate method for conducting empirical validation in agent-based (AB) models, which is often regarded as the Achilles’ heel of the AB approach to economic modelling. The paper has two objectives. First, to identify key issues facing AB economists engaged in empirical validation. Second, to critically appraise the extent to which alternative approaches deal with these issues. We identify a first set of issues that are common to both AB and neoclassical modellers and a second set of issues which are specific to AB modellers. This second set of issues is captured in a novel taxonomy, which takes into consideration the nature of the object under study, the goal of the analysis, the nature of the modelling assumptions, and the methodology of the analysis. Having identified the nature and causes of heterogeneity in empirical validation, we examine three important approaches to validation that have been developed in AB economics: indirect calibration, the Werker-Brenner approach, and the history-friendly approach. We also discuss a set of open questions within empirical validation. These include the trade-off between empirical support and tractability of findings, the issue of over-parameterisation, unconditional objects, counterfactuals, and the non-neutrality of data.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.lem.sssup.it/WPLem/files/2006-14.pdf
Download Restriction: no

Paper provided by Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy in its series LEM Papers Series with number 2006/14.

as
in new window

Length:
Date of creation: 15 May 2006
Handle: RePEc:ssa:lemwps:2006/14
Contact details of provider: Postal:
Piazza dei Martiri della Liberta, 33, 56127 Pisa

Phone: +39-50-883343
Fax: +39-50-883344
Web page: http://www.lem.sssup.it/
Email:


More information through EDIRC

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as
in new window


  1. Gerald Silverberg & Bart Verspagen, 2002. "A Percolation Model of Innovation in Complex Technology Spaces," Computing in Economics and Finance 2002 24, Society for Computational Economics.
  2. Lawrence J. Christiano & Martin Eichenbaum & Charles L. Evans, 1997. "Monetary policy shocks: what have we learned and to what end?," Working Paper Series, Macroeconomic Issues WP-97-18, Federal Reserve Bank of Chicago.
  3. Dosi, Giovanni, 1988. "Sources, Procedures, and Microeconomic Effects of Innovation," Journal of Economic Literature, American Economic Association, vol. 26(3), pages 1120-1171, September.
  4. A. Pyka & G. Fagiolo, 2007. "Agent-based Modelling: A Methodology for Neo-Schumpetarian Economics," Chapters, in: Elgar Companion to Neo-Schumpeterian Economics, chapter 29 Edward Elgar Publishing.
  5. Lane, David A, 1993. "Artificial Worlds and Economics, Part II," Journal of Evolutionary Economics, Springer, vol. 3(3), pages 177-197, August.
  6. Herbert Dawid & Marc Reimann, 2005. "Evaluating Market Attractiveness: Individual Incentives Versus Industry Profitability," Computational Economics, Springer;Society for Computational Economics, vol. 24(4), pages 321-355, June.
  7. Silverberg, Gerald & Dosi, Giovanni & Orsenigo, Luigi, 1988. "Innovation, Diversity and Diffusion: A Self-organisation Model," Economic Journal, Royal Economic Society, vol. 98(393), pages 1032-1054, December.
  8. Kydland, Finn E & Prescott, Edward C, 1982. "Time to Build and Aggregate Fluctuations," Econometrica, Econometric Society, vol. 50(6), pages 1345-1370, November.
  9. Alan P. Kirman, 1992. "Whom or What Does the Representative Individual Represent?," Journal of Economic Perspectives, American Economic Association, vol. 6(2), pages 117-136, Spring.
  10. Robert E. Lucas, Jr. & Thomas J. Sargent, 1979. "After Keynesian macroeconomics," Quarterly Review, Federal Reserve Bank of Minneapolis, issue Spr.
  11. Delli Gatti, Domenico & Gallegati, Mauro & Giulioni, Gianfranco & Palestrini, Antonio, 2003. "Financial fragility, patterns of firms' entry and exit and aggregate dynamics," Journal of Economic Behavior & Organization, Elsevier, vol. 51(1), pages 79-97, May.
  12. Lane, David A, 1993. "Artificial Worlds and Economics, Part I," Journal of Evolutionary Economics, Springer, vol. 3(2), pages 89-107, May.
  13. G. Silverberg & B. Verspagen, 1995. "Evolutionary Theorizing on Economic Growth," Working Papers wp95078, International Institute for Applied Systems Analysis.
  14. Tesfatsion, Leigh, 1997. "How Economists Can Get Alife," Staff General Research Papers Archive 1685, Iowa State University, Department of Economics.
  15. Pier Paolo Saviotti & Andreas Pyka, 2002. "Economic Development, Qualitative Change And Employment Creation," Computing in Economics and Finance 2002 362, Society for Computational Economics.
  16. Alessio Moneta, 2005. "Causality in macroeconometrics: some considerations about reductionism and realism," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(3), pages 433-453.
  17. Giorgio Fagiolo & Giovanni Dosi, 2002. "Exploitation, Exploration and Innovation in a Model of Endogenous Growth with Locally interacting Agents," LEM Papers Series 2002/25, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
  18. Roberto Leombruni, 2002. "The Methodological Status of Agent-Based Simulations," LABORatorio R. Revelli Working Papers Series 19, LABORatorio R. Revelli, Centre for Employment Studies.
  19. Chiaromonte, Francesca & Dosi, Giovanni, 1993. "Heterogeneity, competition, and macroeconomic dynamics," Structural Change and Economic Dynamics, Elsevier, vol. 4(1), pages 39-63, June.
  20. Tesfatsion, Leigh S., 2002. "Agent-Based Computational Economics: Growing Economies from the Bottom Up," Staff General Research Papers Archive 5075, Iowa State University, Department of Economics.
  21. Hendry,David F. & Morgan,Mary S., 1997. "The Foundations of Econometric Analysis," Cambridge Books, Cambridge University Press, number 9780521588706, December.
  22. Klein, Lawrence R, 1969. "Estimation on Interdependent Systems in Macroeconometrics," Econometrica, Econometric Society, vol. 37(2), pages 171-192, April.
  23. Hans M. Amman & David A. Kendrick, . "Computational Economics," Online economics textbooks, SUNY-Oswego, Department of Economics, number comp1.
  24. Johann Peter Murmann & Thomas Brenner, 2003. "The Use of Simulations in Developing Robust Knowledge about Causal Processes: Methodological Considerations and an Application to Industrial Evolution," Computing in Economics and Finance 2003 66, Society for Computational Economics.
  25. Alan Kirman, 1997. "The economy as an evolving network," Journal of Evolutionary Economics, Springer, vol. 7(4), pages 339-353.
  26. Dawid, Herbert, 2006. "Agent-based Models of Innovation and Technological Change," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 25, pages 1235-1272 Elsevier.
  27. Hoover, Kevin D, 1995. "Facts and Artifacts: Calibration and the Empirical Assessment of Real-Business-Cycle Models," Oxford Economic Papers, Oxford University Press, vol. 47(1), pages 24-44, January.
  28. Luigi Marengo & Marc Willinger, 1997. "Alternative methodologies for modelling evolutionary dynamics: Introduction," Journal of Evolutionary Economics, Springer, vol. 7(4), pages 331-338.
  29. McCloskey, Donald N, 1983. "The Rhetoric of Economics," Journal of Economic Literature, American Economic Association, vol. 21(2), pages 481-517, June.
  30. Kirman, Alan, 1989. "The Intrinsic Limits of Modern Economic Theory: The Emperor Has No Clothes," Economic Journal, Royal Economic Society, vol. 99(395), pages 126-139, Supplemen.
  31. Silverberg, Gerald & Lehnert, Doris, 1993. "Long waves and 'evolutionary chaos' in a simple Schumpeterian model of embodied technical change," Structural Change and Economic Dynamics, Elsevier, vol. 4(1), pages 9-37, June.
  32. Werker, C. & Brenner, T., 2004. "Empirical calibration of simulation models," Working Papers 04.13, Eindhoven Center for Innovation Studies.
  33. Joshua M. Epstein & Robert L. Axtell, 1996. "Growing Artificial Societies: Social Science from the Bottom Up," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262550253, September.
  34. Paul Windrum, 2004. "Heterogeneous preferences and new innovation cycles in mature industries: the camera industry 1955-1974," Papers on Economics and Evolution 2004-18, Philipps University Marburg, Department of Geography.
  35. Koesrindartoto, Deddy P. & Sun, Junjie & Tesfatsion, Leigh, 2005. "An Agent-Based Computational Laboratory for Testing the Economic Reliability of Wholesale Power Market Designs," Staff General Research Papers Archive 12388, Iowa State University, Department of Economics.
  36. Windrum Paul & Birchenhall Chris, 2004. "Structural change in the presence of network externalities: a co-evolutionary model of technological successions," Research Memorandum 012, Maastricht University, Maastricht Economic Research Institute on Innovation and Technology (MERIT).
  37. Roberto Leombruni & Matteo Richiardi & Nicole J. Saam & Michele Sonnessa, 2005. "A Common Protocol for Agent-Based Social Simulation," LABORatorio R. Revelli Working Papers Series 47, LABORatorio R. Revelli, Centre for Employment Studies.
  38. Silverberg, Gerald & Verspagen, Bart, 1994. "Collective Learning, Innovation and Growth in a Boundedly Rational, Evolutionary World," Journal of Evolutionary Economics, Springer, vol. 4(3), pages 207-226, September.
  39. Dominique Foray & Robin Cowan, 2002. "Evolutionary economics and the counterfactual threat: on the nature and role of counterfactual history as an empirical tool in economics," Journal of Evolutionary Economics, Springer, vol. 12(5), pages 539-562.
  40. Richard R. Nelson, 1995. "Recent Evolutionary Theorizing about Economic Change," Journal of Economic Literature, American Economic Association, vol. 33(1), pages 48-90, March.
  41. Matteo Richiardi, 2004. "The Promises and Perils of Agent-Based Computational Economics," Computational Economics 0401001, EconWPA.
  42. Liebowitz, S J & Margolis, Stephen E, 1990. "The Fable of the Keys," Journal of Law and Economics, University of Chicago Press, vol. 33(1), pages 1-25, April.
  43. Favero, Carlo A., 2001. "Applied Macroeconometrics," OUP Catalogue, Oxford University Press, number 9780198296850.
  44. Lucas, Robert Jr, 1976. "Econometric policy evaluation: A critique," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 1(1), pages 19-46, January.
  45. Windrum, Paul & Birchenhall, Chris, 1998. "Is product life cycle theory a special case? Dominant designs and the emergence of market niches through coevolutionary-learning," Structural Change and Economic Dynamics, Elsevier, vol. 9(1), pages 109-134, March.
  46. Kevin Hoover, 1994. "Econometrics as observation: the Lucas critique and the nature of econometric inference," Journal of Economic Methodology, Taylor & Francis Journals, vol. 1(1), pages 65-80.
  47. Thomas Brenner, 2004. "Agent Learning Representation - Advice in Modelling Economic Learning," Papers on Economics and Evolution 2004-16, Philipps University Marburg, Department of Geography.
  48. Paul Downward & John H. Finch & John Ramsay, 2002. "Critical realism, empirical methods and inference: a critical discussion," Cambridge Journal of Economics, Oxford University Press, vol. 26(4), pages 481-500, July.
  49. Forni, Mario & Lippi, Marco, 1997. "Aggregation and the Microfoundations of Dynamic Macroeconomics," OUP Catalogue, Oxford University Press, number 9780198288008.
  50. Hansen, Lars Peter & Sargent, Thomas J., 1980. "Formulating and estimating dynamic linear rational expectations models," Journal of Economic Dynamics and Control, Elsevier, vol. 2(1), pages 7-46, May.
  51. Hicks, J. R., 1979. "Critical Essays in Monetary Theory," OUP Catalogue, Oxford University Press, number 9780198284239.
  52. Gatti, Domenico Delli & Guilmi, Corrado Di & Gaffeo, Edoardo & Giulioni, Gianfranco & Gallegati, Mauro & Palestrini, Antonio, 2005. "A new approach to business fluctuations: heterogeneous interacting agents, scaling laws and financial fragility," Journal of Economic Behavior & Organization, Elsevier, vol. 56(4), pages 489-512, April.
  53. Franco Malerba & Luigi Orsenigo, 2002. "Innovation and market structure in the dynamics of the pharmaceutical industry and biotechnology: towards a history-friendly model," Industrial and Corporate Change, Oxford University Press, vol. 11(4), pages 667-703, August.
  54. Fagiolo, Giorgio, 2005. "Endogenous neighborhood formation in a local coordination model with negative network externalities," Journal of Economic Dynamics and Control, Elsevier, vol. 29(1-2), pages 297-319, January.
  55. Christopher A. Sims, 1982. "Policy Analysis with Econometric Models," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 13(1), pages 107-164.
  56. Sims, Christopher A, 1980. "Macroeconomics and Reality," Econometrica, Econometric Society, vol. 48(1), pages 1-48, January.
  57. Brock, W A, 1999. "Scaling in Economics: A Reader's Guide," Industrial and Corporate Change, Oxford University Press, vol. 8(3), pages 409-446, September.
  58. Windrum Paul, 1999. "Simulation models of technological innovation: A Review," Research Memorandum 005, Maastricht University, Maastricht Economic Research Institute on Innovation and Technology (MERIT).
  59. Malerba, Franco, et al, 1999. "'History-Friendly' Models of Industry Evolution: The Computer Industry," Industrial and Corporate Change, Oxford University Press, vol. 8(1), pages 3-40, March.
  60. Tesfatsion, Leigh & Judd, Kenneth L., 2006. "Handbook of Computational Economics, Vol. 2: Agent-Based Computational Economics," Staff General Research Papers Archive 10368, Iowa State University, Department of Economics.
  61. Bianchi, Carlo & Cirillo, Pasquale & Gallegati, Mauro & Vagliasindi, Pietro A., 2008. "Validation in agent-based models: An investigation on the CATS model," Journal of Economic Behavior & Organization, Elsevier, vol. 67(3-4), pages 947-964, September.
  62. Uskali Maki, 2005. "Models are experiments, experiments are models," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(2), pages 303-315.
  63. Boulding, K E, 1991. "What Is Evolutionary Economics?," Journal of Evolutionary Economics, Springer, vol. 1(1), pages 9-17, January.
  64. Keuzenkamp,Hugo A., 2000. "Probability, Econometrics and Truth," Cambridge Books, Cambridge University Press, number 9780521553599, December.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:ssa:lemwps:2006/14. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ()

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.