IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Login to save this paper or follow this series

Empirical Validation of Agent Based Models: A Critical Survey

  • Giorgio Fagiolo
  • Paul Windrum
  • Alessio Moneta

This paper addresses the problem of finding the appropriate method for conducting empirical validation in agent-based (AB) models, which is often regarded as the Achilles’ heel of the AB approach to economic modelling. The paper has two objectives. First, to identify key issues facing AB economists engaged in empirical validation. Second, to critically appraise the extent to which alternative approaches deal with these issues. We identify a first set of issues that are common to both AB and neoclassical modellers and a second set of issues which are specific to AB modellers. This second set of issues is captured in a novel taxonomy, which takes into consideration the nature of the object under study, the goal of the analysis, the nature of the modelling assumptions, and the methodology of the analysis. Having identified the nature and causes of heterogeneity in empirical validation, we examine three important approaches to validation that have been developed in AB economics: indirect calibration, the Werker-Brenner approach, and the history-friendly approach. We also discuss a set of open questions within empirical validation. These include the trade-off between empirical support and tractability of findings, the issue of over-parameterisation, unconditional objects, counterfactuals, and the non-neutrality of data.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.lem.sssup.it/WPLem/files/2006-14.pdf
Download Restriction: no

Paper provided by Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy in its series LEM Papers Series with number 2006/14.

as
in new window

Length:
Date of creation: 15 May 2006
Date of revision:
Handle: RePEc:ssa:lemwps:2006/14
Contact details of provider: Postal: Piazza dei Martiri della Liberta, 33, 56127 Pisa
Phone: +39-50-883343
Fax: +39-50-883344
Web page: http://www.lem.sssup.it/
Email:


More information through EDIRC

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Roberto Leombruni & Matteo Richiardi & Nicole J. Saam & Michele Sonnessa, 2005. "A Common Protocol for Agent-Based Social Simulation," LABORatorio R. Revelli Working Papers Series 47, LABORatorio R. Revelli, Centre for Employment Studies.
  2. McCloskey, Donald N, 1983. "The Rhetoric of Economics," Journal of Economic Literature, American Economic Association, vol. 21(2), pages 481-517, June.
  3. Forni, Mario & Lippi, Marco, 1997. "Aggregation and the Microfoundations of Dynamic Macroeconomics," OUP Catalogue, Oxford University Press, number 9780198288008, March.
  4. Johann Peter Murmann & Thomas Brenner, 2003. "The Use of Simulations in Developing Robust Knowledge about Causal Processes: Methodological Considerations and an Application to Industrial Evolution," Computing in Economics and Finance 2003 66, Society for Computational Economics.
  5. Windrum Paul & Birchenhall Chris, 2004. "Structural change in the presence of network externalities: a co-evolutionary model of technological successions," Research Memorandum 012, Maastricht University, Maastricht Economic Research Institute on Innovation and Technology (MERIT).
  6. Giorgio Fagiolo & Giovanni Dosi, 2002. "Exploitation, Exploration and Innovation in a Model of Endogenous Growth with Locally interacting Agents," LEM Papers Series 2002/25, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
  7. Joshua M. Epstein & Robert L. Axtell, 1996. "Growing Artificial Societies: Social Science from the Bottom Up," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262550253, June.
  8. Robert E. Lucas, Jr. & Thomas J. Sargent, 1979. "After Keynesian macroeconomics," Quarterly Review, Federal Reserve Bank of Minneapolis, issue Spr.
  9. Richard R. Nelson, 1995. "Recent Evolutionary Theorizing about Economic Change," Journal of Economic Literature, American Economic Association, vol. 33(1), pages 48-90, March.
  10. Delli Gatti, Domenico & Gallegati, Mauro & Giulioni, Gianfranco & Palestrini, Antonio, 2003. "Financial fragility, patterns of firms' entry and exit and aggregate dynamics," Journal of Economic Behavior & Organization, Elsevier, vol. 51(1), pages 79-97, May.
  11. Brock, W A, 1999. "Scaling in Economics: A Reader's Guide," Industrial and Corporate Change, Oxford University Press, vol. 8(3), pages 409-46, September.
  12. Thomas Brenner, 2004. "Agent Learning Representation - Advice in Modelling Economic Learning," Papers on Economics and Evolution 2004-16, Philipps University Marburg, Department of Geography.
  13. Thomas Brenner & Claudia Werker, 2004. "Empirical Calibration of Simulation Models," Computing in Economics and Finance 2004 89, Society for Computational Economics.
  14. Paul Windrum, 2004. "Heterogeneous preferences and new innovation cycles in mature industries: the camera industry 1955-1974," Papers on Economics and Evolution 2004-18, Philipps University Marburg, Department of Geography.
  15. Bianchi, Carlo & Cirillo, Pasquale & Gallegati, Mauro & Vagliasindi, Pietro A., 2008. "Validation in agent-based models: An investigation on the CATS model," Journal of Economic Behavior & Organization, Elsevier, vol. 67(3-4), pages 947-964, September.
  16. Alessio Moneta, 2005. "Causality in macroeconometrics: some considerations about reductionism and realism," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(3), pages 433-453.
  17. Hansen, Lars Peter & Sargent, Thomas J., 1980. "Formulating and estimating dynamic linear rational expectations models," Journal of Economic Dynamics and Control, Elsevier, vol. 2(1), pages 7-46, May.
  18. Tesfatsion, Leigh, 1997. "How Economists Can Get Alife," Staff General Research Papers 1685, Iowa State University, Department of Economics.
  19. Klein, Lawrence R, 1969. "Estimation on Interdependent Systems in Macroeconometrics," Econometrica, Econometric Society, vol. 37(2), pages 171-92, April.
  20. Kirman, Alan, 1989. "The Intrinsic Limits of Modern Economic Theory: The Emperor Has No Clothes," Economic Journal, Royal Economic Society, vol. 99(395), pages 126-39, Supplemen.
  21. Lane, David A, 1993. "Artificial Worlds and Economics, Part I," Journal of Evolutionary Economics, Springer, vol. 3(2), pages 89-107, May.
  22. A. Pyka & G. Fagiolo, 2007. "Agent-based Modelling: A Methodology for Neo-Schumpetarian Economics," Chapters, in: Elgar Companion to Neo-Schumpeterian Economics, chapter 29 Edward Elgar.
  23. Deddy Koesrindartoto & Junjie Sun, 2005. "An Agent-Based Computational Laboratory for Testing the Economic Reliability of Wholesale Power Market Designs," Computing in Economics and Finance 2005 50, Society for Computational Economics.
  24. Paul Downward & John H. Finch & John Ramsay, 2002. "Critical realism, empirical methods and inference: a critical discussion," Cambridge Journal of Economics, Oxford University Press, vol. 26(4), pages 481-500, July.
  25. Malerba, Franco, et al, 1999. "'History-Friendly' Models of Industry Evolution: The Computer Industry," Industrial and Corporate Change, Oxford University Press, vol. 8(1), pages 3-40, March.
  26. Silverberg, Gerald & Verspagen, Bart, 2005. "A percolation model of innovation in complex technology spaces," Journal of Economic Dynamics and Control, Elsevier, vol. 29(1-2), pages 225-244, January.
  27. Herbert Dawid & Marc Reimann, 2005. "Evaluating Market Attractiveness: Individual Incentives Versus Industry Profitability," Computational Economics, Society for Computational Economics, vol. 24(4), pages 321-355, June.
  28. Lawrence J. Christiano & Martin Eichenbaum & Charles L. Evans, 1997. "Monetary policy shocks: what have we learned and to what end?," Working Paper Series, Macroeconomic Issues WP-97-18, Federal Reserve Bank of Chicago.
  29. Saviotti, Pier Paolo & Pyka, Andreas, 2004. "Economic development, qualitative change and employment creation," Structural Change and Economic Dynamics, Elsevier, vol. 15(3), pages 265-287, September.
  30. Matteo Richiardi, 2003. "The Promises and Perils of Agent-Based Computational Economics," LABORatorio R. Revelli Working Papers Series 29, LABORatorio R. Revelli, Centre for Employment Studies.
  31. Silverberg, Gerald & Verspagen, Bart, 1994. "Collective Learning, Innovation and Growth in a Boundedly Rational, Evolutionary World," Journal of Evolutionary Economics, Springer, vol. 4(3), pages 207-26, September.
  32. Hoover, Kevin D, 1995. "Facts and Artifacts: Calibration and the Empirical Assessment of Real-Business-Cycle Models," Oxford Economic Papers, Oxford University Press, vol. 47(1), pages 24-44, January.
  33. Alan P. Kirman, 1992. "Whom or What Does the Representative Individual Represent?," Journal of Economic Perspectives, American Economic Association, vol. 6(2), pages 117-136, Spring.
  34. repec:dgr:kubcen:199752 is not listed on IDEAS
  35. Hicks, J. R., 1979. "Critical Essays in Monetary Theory," OUP Catalogue, Oxford University Press, number 9780198284239, March.
  36. Franco Malerba & Luigi Orsenigo, 2002. "Innovation and market structure in the dynamics of the pharmaceutical industry and biotechnology: towards a history-friendly model," Industrial and Corporate Change, Oxford University Press, vol. 11(4), pages 667-703, August.
  37. Boulding, K E, 1991. "What Is Evolutionary Economics?," Journal of Evolutionary Economics, Springer, vol. 1(1), pages 9-17, January.
  38. Lane, David A, 1993. "Artificial Worlds and Economics, Part II," Journal of Evolutionary Economics, Springer, vol. 3(3), pages 177-97, August.
  39. Tesfatsion, Leigh S., 2002. "Agent-Based Computational Economics: Growing Economies from the Bottom Up," Staff General Research Papers 5075, Iowa State University, Department of Economics.
  40. Uskali Maki, 2005. "Models are experiments, experiments are models," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(2), pages 303-315.
  41. Fagiolo, Giorgio, 2005. "Endogenous neighborhood formation in a local coordination model with negative network externalities," Journal of Economic Dynamics and Control, Elsevier, vol. 29(1-2), pages 297-319, January.
  42. Dawid, Herbert, 2006. "Agent-based Models of Innovation and Technological Change," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 25, pages 1235-1272 Elsevier.
  43. Liebowitz, S J & Margolis, Stephen E, 1990. "The Fable of the Keys," Journal of Law and Economics, University of Chicago Press, vol. 33(1), pages 1-25, April.
  44. Dosi, Giovanni, 1988. "Sources, Procedures, and Microeconomic Effects of Innovation," Journal of Economic Literature, American Economic Association, vol. 26(3), pages 1120-71, September.
  45. Dominique Foray & Robin Cowan, 2002. "Evolutionary economics and the counterfactual threat: on the nature and role of counterfactual history as an empirical tool in economics," Journal of Evolutionary Economics, Springer, vol. 12(5), pages 539-562.
  46. Windrum Paul, 1999. "Simulation models of technological innovation: A Review," Research Memorandum 005, Maastricht University, Maastricht Economic Research Institute on Innovation and Technology (MERIT).
  47. Sims, Christopher A, 1980. "Macroeconomics and Reality," Econometrica, Econometric Society, vol. 48(1), pages 1-48, January.
  48. Kevin Hoover, 1994. "Econometrics as observation: the Lucas critique and the nature of econometric inference," Journal of Economic Methodology, Taylor & Francis Journals, vol. 1(1), pages 65-80.
  49. Roberto Leombruni, 2002. "The Methodological Status of Agent-Based Simulations," LABORatorio R. Revelli Working Papers Series 19, LABORatorio R. Revelli, Centre for Employment Studies.
  50. Alan Kirman, 1997. "The economy as an evolving network," Journal of Evolutionary Economics, Springer, vol. 7(4), pages 339-353.
  51. Chiaromonte, Francesca & Dosi, Giovanni, 1993. "Heterogeneity, competition, and macroeconomic dynamics," Structural Change and Economic Dynamics, Elsevier, vol. 4(1), pages 39-63, June.
  52. Favero, Carlo A., 2001. "Applied Macroeconometrics," OUP Catalogue, Oxford University Press, number 9780198296850, March.
  53. Silverberg, Gerald & Lehnert, Doris, 1993. "Long waves and 'evolutionary chaos' in a simple Schumpeterian model of embodied technical change," Structural Change and Economic Dynamics, Elsevier, vol. 4(1), pages 9-37, June.
  54. Lucas, Robert Jr, 1976. "Econometric policy evaluation: A critique," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 1(1), pages 19-46, January.
  55. Hans M. Amman & David A. Kendrick, . "Computational Economics," Online economics textbooks, SUNY-Oswego, Department of Economics, number comp1.
  56. G. Silverberg & B. Verspagen, 1995. "Evolutionary Theorizing on Economic Growth," Working Papers wp95078, International Institute for Applied Systems Analysis.
  57. Luigi Marengo & Marc Willinger, 1997. "Alternative methodologies for modelling evolutionary dynamics: Introduction," Journal of Evolutionary Economics, Springer, vol. 7(4), pages 331-338.
  58. Gatti, Domenico Delli & Guilmi, Corrado Di & Gaffeo, Edoardo & Giulioni, Gianfranco & Gallegati, Mauro & Palestrini, Antonio, 2005. "A new approach to business fluctuations: heterogeneous interacting agents, scaling laws and financial fragility," Journal of Economic Behavior & Organization, Elsevier, vol. 56(4), pages 489-512, April.
  59. Christopher A. Sims, 1982. "Policy Analysis with Econometric Models," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 13(1), pages 107-164.
  60. Windrum, Paul & Birchenhall, Chris, 1998. "Is product life cycle theory a special case? Dominant designs and the emergence of market niches through coevolutionary-learning," Structural Change and Economic Dynamics, Elsevier, vol. 9(1), pages 109-134, March.
  61. Kydland, Finn E & Prescott, Edward C, 1982. "Time to Build and Aggregate Fluctuations," Econometrica, Econometric Society, vol. 50(6), pages 1345-70, November.
  62. Silverberg, Gerald & Dosi, Giovanni & Orsenigo, Luigi, 1988. "Innovation, Diversity and Diffusion: A Self-organisation Model," Economic Journal, Royal Economic Society, vol. 98(393), pages 1032-54, December.
  63. Tesfatsion, Leigh & Judd, Kenneth L., 2006. "Handbook of Computational Economics, Vol. 2: Agent-Based Computational Economics," Staff General Research Papers 10368, Iowa State University, Department of Economics.
  64. repec:dgr:tuecis:0413 is not listed on IDEAS
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:ssa:lemwps:2006/14. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ()

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.