IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Log in (now much improved!) to save this article

Field experiments in economics: The past, the present, and the future

  • Levitt, Steven D.
  • List, John A.

This study presents an overview of modern field experiments and their usage in economics. Our discussion focuses on three distinct periods of field experimentation that have influenced the economics literature. The first might well be thought of as the dawn of "field" experimentation: the work of Neyman and Fisher, who laid the experimental foundation in the 1920s and 1930s by conceptualizing randomization as an instrument to achieve identification via experimentation with agricultural plots. The second, the large-scale social experiments conducted by government agencies in the mid-twentieth century, moved the exploration from plots of land to groups of individuals. More recently, the nature and range of field experiments has expanded, with a diverse set of controlled experiments being completed outside of the typical laboratory environment. With this growth, the number and types of questions that can be explored using field experiments has grown tremendously. After discussing these three distinct phases, we speculate on the future of field experimental methods, a future that we envision including a strong collaborative effort with outside parties, most importantly private entities.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.sciencedirect.com/science/article/pii/S0014-2921(08)00115-3
Download Restriction: Full text for ScienceDirect subscribers only

As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

Article provided by Elsevier in its journal European Economic Review.

Volume (Year): 53 (2009)
Issue (Month): 1 (January)
Pages: 1-18

as
in new window

Handle: RePEc:eee:eecrev:v:53:y:2009:i:1:p:1-18
Contact details of provider: Web page: http://www.elsevier.com/locate/eer

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Catherine Eckel, 2005. "Subsidizing Charitable Contributions: A Field Test Comparing Matching and Rebate Subsidies," Working Papers 2098, The Field Experiments Website.
  2. Armin Falk, 2007. "Gift Exchange in the Field," Econometrica, Econometric Society, vol. 75(5), pages 1501-1511, 09.
  3. Joshua Angrist & Eric Bettinger & Michael Kremer, 2004. "Long-Term Consequences of Secondary School Vouchers: Evidence from Administrative Records in Colombia," NBER Working Papers 10713, National Bureau of Economic Research, Inc.
  4. Oriana Bandiera & Iwan Barankay & Imran Rasul, 2005. "Social Preferences and the Response to Incentives: Evidence from Personnel Data," The Quarterly Journal of Economics, Oxford University Press, vol. 120(3), pages 917-962.
  5. Anne Rozan & Anne Stenger-Letheux & Marc Willinger, 2004. "Willingness-to-pay for food safety : an experimental investigation of quality certification on bidding behavior," Post-Print hal-01072925, HAL.
  6. Gine, Xavier & Karlan, Dean S., 2006. "Group versus individual liability : a field experiment in the Philippines," Policy Research Working Paper Series 4008, The World Bank.
  7. Orana Bandiera & Iwan Barankay & Imran Rasul, 2006. "The evolution of cooperative norms: Evidence from a natural field experiment," Natural Field Experiments 00210, The Field Experiments Website.
  8. John List, 2008. "Informed consent in social science," Artefactual Field Experiments 00086, The Field Experiments Website.
  9. John A. List, 2002. "Preference Reversals of a Different Kind: The "More Is Less" Phenomenon," American Economic Review, American Economic Association, vol. 92(5), pages 1636-1643, December.
  10. Ernst Fehr & Joseph Henrich & Robert Boyd, 2003. "In Search of Homo Economicus: Behavioral Experiments in 15 Small- Scale Societies," Microeconomics 0305009, EconWPA.
  11. Peter Bohm, 1972. "Estimating the demand for public goods: An experiment," Framed Field Experiments 00126, The Field Experiments Website.
  12. John A. List, 2004. "Young, Selfish and Male: Field evidence of social preferences," Economic Journal, Royal Economic Society, vol. 114(492), pages 121-149, 01.
  13. John A. List, 2007. "Field Experiments: A Bridge Between Lab and Naturally-Occurring Data," NBER Working Papers 12992, National Bureau of Economic Research, Inc.
  14. Craig E. Landry & Andreas Lange & John A. List & Michael K. Price & Nicholas G. Rupp, 2006. "Toward an Understanding of the Economics of Charity: Evidence from a Field Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 121(2), pages 747-782.
  15. P. A. Riach & J. Rich, 2002. "Field Experiments of Discrimination in the Market Place," Economic Journal, Royal Economic Society, vol. 112(483), pages 480-518, November.
  16. Matthias Benz & Stephan Meier, 2008. "Do people behave in experiments as in the field?—evidence from donations," Experimental Economics, Springer;Economic Science Association, vol. 11(3), pages 268-281, September.
  17. Ginger Jin & Andrew Kato, 2004. "Dividing online and offline: A case study," Natural Field Experiments 00276, The Field Experiments Website.
  18. List, John A. & Shogren, Jason F., 1998. "Calibration of the difference between actual and hypothetical valuations in a field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 37(2), pages 193-205, October.
  19. Lawrence F. Katz & Jeffrey R. Kling & Jeffrey B. Liebman, 2001. "Moving to Opportunity in Boston: Early Results of a Randomized Mobility Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 116(2), pages 607-654.
  20. Ferber,Robert & Hirsch,Werner Z., 1982. "Social Experimentation and Economic Policy," Cambridge Books, Cambridge University Press, number 9780521285070, October.
  21. Joseph Henrich & Robert Boyd & Samuel Bowles & Colin Camerer & Ernst Fehr & Herbert Gintis & Richard McElreath & Michael Alvard & Abigail Barr & Jean Ensminger & Kim Hill & Francisco Gil-White & Micha, 2001. "Economic Man in Cross-Cultural Perspective: Behavioral Experiments in Fifteen Small-Scale Societies," Working Papers 01-11-063, Santa Fe Institute.
  22. John A. List, 2003. "Neoclassical Theory Versus Prospect Theory: Evidence from the Marketplace," NBER Working Papers 9736, National Bureau of Economic Research, Inc.
  23. Glenn Harrison & Morten Lau & Elisabet Rutstrom, 2005. "Risk attitudes, randomization to treatment, and self-selection into experiments," Artefactual Field Experiments 00061, The Field Experiments Website.
  24. Jerry A. Hausman & David A. Wise, 1985. "Introduction to "Social Experimentation"," NBER Chapters, in: Social Experimentation, pages 1-10 National Bureau of Economic Research, Inc.
  25. Min Ding & Rajdeep Grewal & John Liechty, 2005. "Incentive-aligned conjoint analysis," Framed Field Experiments 00139, The Field Experiments Website.
  26. Kremer, Michael Robert & Miguel, Edward A. & Thorton, Rebecca L, 2004. "Incentives to Learn," Center for International and Development Economics Research, Working Paper Series qt9kc4p47q, Center for International and Development Economics Research, Institute for Business and Economic Research, UC Berkeley.
  27. Steven Levitt & John List, 2007. "What do Laboratory Experiments Measuring Social Preferences Reveal About the Real World," Artefactual Field Experiments 00480, The Field Experiments Website.
  28. Camerer, Colin & Babcock, Linda & Loewenstein, George & Thaler, Richard, 1996. "Labor Supply of New York City Cab Drivers: One Day At A time," Working Papers 960, California Institute of Technology, Division of the Humanities and Social Sciences.
  29. Richard Blundell & Monica Costa Dias, 2008. "Alternative approaches to evaluation in empirical microeconomics," CeMMAP working papers CWP26/08, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  30. repec:fth:prinin:396 is not listed on IDEAS
  31. List, John A, 2003. "Using Random nth Price Auctions to Value Non-market Goods and Services," Journal of Regulatory Economics, Springer, vol. 23(2), pages 193-205, March.
  32. Abhijit V. Banerjee & Shawn Cole & Esther Duflo & Leigh Linden, 2007. "Remedying Education: Evidence from Two Randomized Experiments in India," The Quarterly Journal of Economics, Oxford University Press, vol. 122(3), pages 1235-1264.
  33. Marianne Bertrand & Dean Karlan & Sendhil Mullainathan & Eldar Shafir & Jonathan Zinman, 2006. "What's psychology worth? A field experiment in the consumer credit market," Natural Field Experiments 00217, The Field Experiments Website.
  34. Ginger Zhe Jin & Andrew Kato & John A. List, 2006. "That's News to Me! Information Revelation in Professional Certification Markets," NBER Working Papers 12390, National Bureau of Economic Research, Inc.
  35. John List & Craig Gallet, 2001. "What Experimental Protocol Influence Disparities Between Actual and Hypothetical Stated Values?," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 20(3), pages 241-254, November.
  36. Joshua Angrist & Alan B. Krueger, 2001. "Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments," Working Papers 834, Princeton University, Department of Economics, Industrial Relations Section..
  37. Jerry A. Hausman & David A. Wise, 1985. "Social Experimentation," NBER Books, National Bureau of Economic Research, Inc, number haus85-1, Enero.
  38. Karlan, Dean S. & Zinman, Jonathan, 2007. "Credit Elasticities in Less-Developed Economies: Implications for Microfinance," CEPR Discussion Papers 6071, C.E.P.R. Discussion Papers.
  39. Omar Azfar & Clifford Zinnes, 2006. "Which incentives work? An experimental analysis of incentives for trainers," Natural Field Experiments 00209, The Field Experiments Website.
  40. James J. Heckman, 1989. "Choosing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs: The Case of Manpower Training," NBER Working Papers 2861, National Bureau of Economic Research, Inc.
  41. Stephen T. Ziliak, 2008. "Retrospectives: Guinnessometrics: The Economic Foundation of "Student's" t," Journal of Economic Perspectives, American Economic Association, vol. 22(4), pages 199-216, Fall.
  42. Richard Engelbrecht-Wiggans & John A. List & David H. Reiley, 2006. "Demand Reduction In Multi-Unit Auctions With Varying Numbers Of Bidders: Theory And Evidence From A Field Experiment ," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 47(1), pages 203-231, 02.
  43. Paul Schultz, T., 2004. "School subsidies for the poor: evaluating the Mexican Progresa poverty program," Journal of Development Economics, Elsevier, vol. 74(1), pages 199-250, June.
  44. John A. List, 2001. "Do Explicit Warnings Eliminate the Hypothetical Bias in Elicitation Procedures? Evidence from Field Auctions for Sportscards," American Economic Review, American Economic Association, vol. 91(5), pages 1498-1507, December.
  45. Xavier Gine & Pamela Jakiela & Dean Karlan & Jonathan Morduch, 2006. "Microfinance games," Framed Field Experiments 00150, The Field Experiments Website.
  46. Dean Karlan & Jonathan Zinman, 2010. "Expanding Credit Access: Using Randomized Supply Decisions to Estimate the Impacts," Review of Financial Studies, Society for Financial Studies, vol. 23(1), pages 433-464, January.
  47. David Lucking-Reiley, 1999. "Using Field Experiments to Test Equivalence between Auction Formats: Magic on the Internet," American Economic Review, American Economic Association, vol. 89(5), pages 1063-1080, December.
  48. Ackerberg, Daniel & Lanier Benkard, C. & Berry, Steven & Pakes, Ariel, 2007. "Econometric Tools for Analyzing Market Outcomes," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 63 Elsevier.
  49. Edward P. Lazear, 1999. "Personnel Economics: Past Lessons and Future Directions," NBER Working Papers 6957, National Bureau of Economic Research, Inc.
  50. John List, 2002. "Testing neoclassical competitive market theory in the field," Artefactual Field Experiments 00082, The Field Experiments Website.
  51. Uri Gneezy & John List, 2006. "Putting behavioral economics to work: Testing for gift exchange in labor markets using field experiments," Natural Field Experiments 00259, The Field Experiments Website.
  52. James J. Heckman & Vytlacil, Edward J., 2007. "Econometric Evaluation of Social Programs, Part I: Causal Models, Structural Models and Econometric Policy Evaluation," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 70 Elsevier.
  53. Costas Meghir & Mårten Palme, 1999. "Assessing the effect of schooling on earnings using a social experiment," IFS Working Papers W99/10, Institute for Fiscal Studies.
  54. Juan-Camilo Cardenas, 2002. "Real wealth and experimental cooperation: Evidence from field experiments," Artefactual Field Experiments 00019, The Field Experiments Website.
  55. Jeffrey Carpenter & Erika Seki, 2005. "Competitive Work Environments and Social Preferences: Field experimental evidence from a japanese fishing community," Middlebury College Working Paper Series 0513, Middlebury College, Department of Economics.
  56. Ernst Fehr & Lorenz Goette, 2007. "Do Workers Work More if Wages Are High? Evidence from a Randomized Field Experiment," American Economic Review, American Economic Association, vol. 97(1), pages 298-317, March.
  57. John Fox & Jayson Lusk, 2003. "Value elicitation in laboratory and retail environments," Framed Field Experiments 00185, The Field Experiments Website.
  58. Eric Anderson & Duncan Simester, 2003. "Effects of $9 Price Endings on Retail Sales: Evidence from Field Experiments," Quantitative Marketing and Economics (QME), Springer, vol. 1(1), pages 93-110, March.
  59. Nava Ashraf & James Berry & Jesse M. Shapiro, 2007. "Can Higher Prices Stimulate Product Use? Evidence from a Field Experiment in Zambia," NBER Working Papers 13247, National Bureau of Economic Research, Inc.
  60. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
  61. Dean Karlan & John A. List, 2007. "Does Price Matter in Charitable Giving? Evidence from a Large-Scale Natural Field Experiment," American Economic Review, American Economic Association, vol. 97(5), pages 1774-1793, December.
  62. John List & David Reiley, 2008. "Field experiments," Artefactual Field Experiments 00091, The Field Experiments Website.
  63. John A. List & David Lucking-Reiley, 2000. "The Effects of Seed Money and Refunds on Charitable Giving: Experimental Evidence from a University Capital Campaign," Vanderbilt University Department of Economics Working Papers 0008, Vanderbilt University Department of Economics.
  64. repec:feb:artefa:0106 is not listed on IDEAS
  65. Yan Chen & Xin Li & Jeffrey MacKie-Mason, 2006. "Online fund-raising mechanisms: A field experiment," Natural Field Experiments 00225, The Field Experiments Website.
  66. Karlan, Dean S. & Zinman, Jonathan, 2007. "Observing Unobservables: Identifying Information Asymmetries with a Consumer Credit Field Experiment," CEPR Discussion Papers 6182, C.E.P.R. Discussion Papers.
  67. Abbring, Jaap H. & Heckman, James J., 2007. "Econometric Evaluation of Social Programs, Part III: Distributional Treatment Effects, Dynamic Treatment Effects, Dynamic Discrete Choice, and General Equilibrium Policy Evaluation," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 72 Elsevier.
  68. Hossain Tanjim & Morgan John, 2006. "...Plus Shipping and Handling: Revenue (Non) Equivalence in Field Experiments on eBay," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 5(2), pages 1-30, January.
  69. David Greenberg & Mark Shroder & Matthew Onstott, 1999. "The Social Experiment Market," Journal of Economic Perspectives, American Economic Association, vol. 13(3), pages 157-172, Summer.
  70. Uri Gneezy & Aldo Rustichini, 2000. "Pay Enough or Don't Pay at All," The Quarterly Journal of Economics, Oxford University Press, vol. 115(3), pages 791-810.
  71. Kenneth I. Wolpin & Mark R. Rosenzweig, 2000. "Natural "Natural Experiments" in Economics," Journal of Economic Literature, American Economic Association, vol. 38(4), pages 827-874, December.
  72. Nava Ashraf & Dean Karlan & Wesley Yin, 2006. "Tying Odysseus to the Mast: Evidence From a Commitment Savings Product in the Philippines," The Quarterly Journal of Economics, Oxford University Press, vol. 121(2), pages 635-672.
  73. John A. List & Michael K. Price, 2005. "Conspiraces and Secret Price Discounts in the Marketplace: Evidence from Field Experiments," RAND Journal of Economics, The RAND Corporation, vol. 36(3), pages 700-717, Autumn.
  74. Bohm, Peter, 1972. "Estimating demand for public goods: An experiment," European Economic Review, Elsevier, vol. 3(2), pages 111-130.
  75. John List & David Lucking-Reiley, 2000. "Demand Reduction in Multiunit Auctions: Evidence from a Sportscard Field Experiment," Natural Field Experiments 00520, The Field Experiments Website.
  76. Dean S. Karlan & Jonathan Zinman, 2005. "Observing Unobservables: Identifying Information Asymmetries with a Consumer Credit Field Experiment," Working Papers 911, Economic Growth Center, Yale University.
  77. Jonathan E. Alevy & Michael S. Haigh & John A. List, 2007. "Information Cascades: Evidence from a Field Experiment with Financial Market Professionals," Journal of Finance, American Finance Association, vol. 62(1), pages 151-180, 02.
  78. Esther Duflo & Pascaline Dupas & Michael Kremer & Samuel Sinei, 2006. "Education and hiv/aids prevention: Evidence from a randomized evaluation in western kenya," Framed Field Experiments 00143, The Field Experiments Website.
  79. Jeffrey Carpenter & Amrita Daniere & Lois Takahashi, 2003. "Cooperation, Trust, and Social Capital in Southeast Asian Urban Slums," Middlebury College Working Paper Series 0309, Middlebury College, Department of Economics.
  80. Rachel Croson & Jen Shang, 2008. "The impact of downward social information on contribution decisions," Natural Field Experiments 00322, The Field Experiments Website.
  81. Henry S. Farber, 2005. "Is Tomorrow Another Day? The Labor Supply of New York City Cabdrivers," Journal of Political Economy, University of Chicago Press, vol. 113(1), pages 46-82, February.
  82. Steven D. Levitt & John A. List, 2007. "Viewpoint: On the generalizability of lab behaviour to the field," Canadian Journal of Economics, Canadian Economics Association, vol. 40(2), pages 347-370, May.
  83. Duflo, Esther & Dupas, Pascaline & Kremer, Michael & Sinei, Samuel, 2006. "Education and HIV/AIDS prevention : evidence from a randomized evaluation in Western Kenya," Policy Research Working Paper Series 4024, The World Bank.
  84. John List, 2004. "The nature and extent of discrimination in the marketplace: Evidence from the field," Natural Field Experiments 00299, The Field Experiments Website.
  85. John List, 2004. "Testing Neoclassical Competitive Theory in Multilateral Decentralized Markets," Natural Field Experiments 00490, The Field Experiments Website.
  86. Colin F. Camerer, 1998. "Can Asset Markets Be Manipulated? A Field Experiment with Racetrack Betting," Journal of Political Economy, University of Chicago Press, vol. 106(3), pages 457-482, June.
  87. Abigail Barr & Pieter Serneels, 2004. "Wages and Reciprocity in the Workplace," CSAE Working Paper Series 2004-18, Centre for the Study of African Economies, University of Oxford.
  88. Juan-Camilo Cardenas, 2004. "Norms from outside and from inside: An experimental analysis on the governance of local ecosystems," Artefactual Field Experiments 00025, The Field Experiments Website.
  89. repec:feb:artefa:0102 is not listed on IDEAS
  90. repec:feb:framed:0081 is not listed on IDEAS
  91. Dirk Engelmann & Veronika Grimm, 2009. "Bidding Behaviour in Multi-Unit Auctions - An Experimental Investigation," Economic Journal, Royal Economic Society, vol. 119(537), pages 855-882, 04.
  92. Bruno S. Frey & Stephan Meier, 2004. "Social Comparisons and Pro-social Behavior: Testing "Conditional Cooperation" in a Field Experiment," American Economic Review, American Economic Association, vol. 94(5), pages 1717-1722, December.
  93. C. F. Manski, "undated". "Learning about social programs from experiments with random assignment of treatments," Institute for Research on Poverty Discussion Papers 1061-95, University of Wisconsin Institute for Research on Poverty.
  94. Lars Hultkrantz & Gunnar Lindberg & Jan-Eric Nilsson & Fridtjof Thomas, 2005. "Pay-as-you-speed: Two field experiments on controlling adverse selection and moral hazard in traffic insurance," Framed Field Experiments 00170, The Field Experiments Website.
  95. Jonathan E. Alevy & Michael S. Haigh & John List, 2006. "Information Cascades: Evidence from An Experiment with Financial Market Professionals," NBER Working Papers 12767, National Bureau of Economic Research, Inc.
  96. Abigail Barr & Pieter Serneels, 2009. "Reciprocity in the workplace," Experimental Economics, Springer;Economic Science Association, vol. 12(1), pages 99-112, March.
  97. Meyer, Bruce D, 1995. "Natural and Quasi-experiments in Economics," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(2), pages 151-161, April.
  98. Hausman, Jerry A & Wise, David A, 1979. "Attrition Bias in Experimental and Panel Data: The Gary Income Maintenance Experiment," Econometrica, Econometric Society, vol. 47(2), pages 455-473, March.
  99. James J. Heckman & Vytlacil, Edward J., 2007. "Econometric Evaluation of Social Programs, Part II: Using the Marginal Treatment Effect to Organize Alternative Econometric Estimators to Evaluate Social Programs, and to Forecast their Effects in New," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 71 Elsevier.
  100. Peter Bohm, 1984. "Revealing demand for an actual public good," Framed Field Experiments 00129, The Field Experiments Website.
  101. Bohm, Peter, 1984. "Revealing demand for an actual public good," Journal of Public Economics, Elsevier, vol. 24(2), pages 135-151, July.
  102. Julie Berry Cullen & Brian A Jacob & Steven Levitt, 2006. "The Effect of School Choice on Participants: Evidence from Randomized Lotteries," Econometrica, Econometric Society, vol. 74(5), pages 1191-1230, 09.
  103. Marianne Bertrand & Sendhil Mullainathan & Dean Karlan & Eldar Shafir & Jonathan Zinman, 2009. "What's Advertising Content Worth? Evidence from a Consumer Credit Marketing Field Experiment," Working Papers 968, Economic Growth Center, Yale University.
  104. repec:feb:artefa:0090 is not listed on IDEAS
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:eee:eecrev:v:53:y:2009:i:1:p:1-18. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Shamier, Wendy)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.