IDEAS home Printed from https://ideas.repec.org/a/aea/jecper/v25y2011i3p39-62.html
   My bibliography  Save this article

The Role of Theory in Field Experiments

Author

Listed:
  • David Card
  • Stefano DellaVigna
  • Ulrike Malmendier

Abstract

We classify all published field experiments in five top economics journals from 1975 to 2010 according to how closely the experimental design and analysis are linked to economic theory. We find that the vast majority of field experiments (68 percent) are Descriptive studies that lack any explicit model; 18 percent are Single Model studies that test a single model-based hypothesis; 6 percent are Competing Models studies that test competing model-based hypotheses; and 8 percent are Parameter Estimation studies that estimate structural parameters in a completely specified model. We also classify laboratory experiments published in these journals over the same period and find that economic theory has played a more central role in the laboratory than in the field. Finally, we discuss in detail three sets of field experiments—on gift exchange, on charitable giving, and on negative income tax—that illustrate both the benefits and the potential costs of a tighter link between experimental design and theoretical underpinnings.

Suggested Citation

  • David Card & Stefano DellaVigna & Ulrike Malmendier, 2011. "The Role of Theory in Field Experiments," Journal of Economic Perspectives, American Economic Association, vol. 25(3), pages 39-62, Summer.
  • Handle: RePEc:aea:jecper:v:25:y:2011:i:3:p:39-62
    Note: DOI: 10.1257/jep.25.3.39
    as

    Download full text from publisher

    File URL: http://www.aeaweb.org/articles.php?doi=10.1257/jep.25.3.39
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Uri Gneezy & John A List, 2006. "Putting Behavioral Economics to Work: Testing for Gift Exchange in Labor Markets Using Field Experiments," Econometrica, Econometric Society, vol. 74(5), pages 1365-1384, September.
    2. repec:pri:rpdevs:deaton_understanding_mechanisms_of_economic_development_with_abstract_apr is not listed on IDEAS
    3. Oriana Bandiera & Iwan Barankay & Imran Rasul, 2009. "Social Connections and Incentives in the Workplace: Evidence From Personnel Data," Econometrica, Econometric Society, vol. 77(4), pages 1047-1094, July.
    4. Nicholas Barberis & Wei Xiong, 2009. "What Drives the Disposition Effect? An Analysis of a Long-Standing Preference-Based Explanation," Journal of Finance, American Finance Association, vol. 64(2), pages 751-784, April.
    5. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    6. Bruce Sacerdote, 2001. "Peer Effects with Random Assignment: Results for Dartmouth Roommates," The Quarterly Journal of Economics, Oxford University Press, vol. 116(2), pages 681-704.
    7. Terrance Odean, 1998. "Are Investors Reluctant to Realize Their Losses?," Journal of Finance, American Finance Association, vol. 53(5), pages 1775-1798, October.
    8. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    9. Ernst Fehr & Georg Kirchsteiger & Arno Riedl, 1993. "Does Fairness Prevent Market Clearing? An Experimental Investigation," The Quarterly Journal of Economics, Oxford University Press, vol. 108(2), pages 437-459.
    10. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records," American Economic Review, American Economic Association, vol. 80(3), pages 313-336, June.
    11. Stefano DellaVigna & John A. List & Ulrike Malmendier, 2012. "Testing for Altruism and Social Pressure in Charitable Giving," The Quarterly Journal of Economics, Oxford University Press, vol. 127(1), pages 1-56.
    12. Michael C. Keeley & Philip K. Robins, 1980. "Experimental Design, the Conlisk-Watts Assignment Model, and the Proper Estimation of Behavioral Response," Journal of Human Resources, University of Wisconsin Press, vol. 15(4), pages 480-498.
    13. Angus Deaton, 2010. "Understanding the Mechanisms of Economic Development," Journal of Economic Perspectives, American Economic Association, vol. 24(3), pages 3-16, Summer.
    14. George A. Akerlof, 1982. "Labor Contracts as Partial Gift Exchange," The Quarterly Journal of Economics, Oxford University Press, vol. 97(4), pages 543-569.
    15. Oriana Bandiera & Iwan Barankay & Imran Rasul, 2007. "Incentives for Managers and Inequality among Workers: Evidence from a Firm-Level Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 122(2), pages 729-773.
    16. David K. Levine, 1998. "Modeling Altruism and Spitefulness in Experiment," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 1(3), pages 593-622, July.
    17. Terry R. Johnson & John H. Pencavel, 1982. "Forecasting the Effects of a Negative Income Tax Program," ILR Review, Cornell University, ILR School, vol. 35(2), pages 221-234, January.
    18. Ernst Fehr & Lorenz Goette, 2007. "Do Workers Work More if Wages Are High? Evidence from a Randomized Field Experiment," American Economic Review, American Economic Association, vol. 97(1), pages 298-317, March.
    19. Edward Miguel & Michael Kremer, 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities," Econometrica, Econometric Society, vol. 72(1), pages 159-217, January.
    20. Daniel S. Nagin & James B. Rebitzer & Seth Sanders & Lowell J. Taylor, 2002. "Monitoring, Motivation, and Management: The Determinants of Opportunistic Behavior in a Field Experiment," American Economic Review, American Economic Association, vol. 92(4), pages 850-873, September.
    21. repec:pri:rpdevs:deaton_understanding_mechanisms_of_economic_development_with_abstract_apr_2010 is not listed on IDEAS
    22. Axel Ockenfels & Gary E. Bolton, 2000. "ERC: A Theory of Equity, Reciprocity, and Competition," American Economic Review, American Economic Association, vol. 90(1), pages 166-193, March.
    23. Rabin, Matthew, 1993. "Incorporating Fairness into Game Theory and Economics," American Economic Review, American Economic Association, vol. 83(5), pages 1281-1302, December.
    24. Tuma, Nancy Brandon & Robins, Philip K, 1980. "A Dynamic Model of Employment Behavior: An Application to the Seattle and Denver Income Maintenance Experiments," Econometrica, Econometric Society, vol. 48(4), pages 1031-1052, May.
    25. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records: Errata," American Economic Review, American Economic Association, vol. 80(5), pages 1284-1286, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. An incrementalist view of Impact Evaluation and knowledge
      by Jed Friedman in Development Impact on 2012-05-02 18:27:23

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Trier Damgaard, Mette & Gravert, Christina, 2016. "The hidden costs of nudging: Experimental evidence from reminders in fundraising," Working Papers in Economics 650, University of Gothenburg, Department of Economics.
    2. Stefano DellaVigna & John List & Ulrike Malmendier & Gautam Rao, 2016. "Estimating Social Preferences and Gift Exchange at Work," Natural Field Experiments 00586, The Field Experiments Website.
    3. Belot, Michèle & James, Jonathan, 2016. "Partner selection into policy relevant field experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 123(C), pages 31-56.
    4. David Hirshleife, 2015. "Behavioral Finance," Annual Review of Financial Economics, Annual Reviews, vol. 7(1), pages 133-159, December.
    5. Günther Fink & Margaret McConnell & Sebastian Vollmer, 2014. "Testing for heterogeneous treatment effects in experimental data: false discovery risks and correction procedures," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(1), pages 44-57, January.
    6. Altmann, Steffen & Traxler, Christian, 2014. "Nudges at the dentist," European Economic Review, Elsevier, vol. 72(C), pages 19-38.
    7. Bonev, Petyo & Glachant, Matthieu & Söderberg, Magnus, 2018. "A Mechanism for Institutionalised Threat of Regulation: Evidence from the Swedish District Heating Market," Economics Working Paper Series 1805, University of St. Gallen, School of Economics and Political Science.
    8. Madsen Paul E., 2013. "Evaluating Accounting Standards: A Comment on Ramanna’s ‘The International Politics of IFRS Harmonization’," Accounting, Economics, and Law: A Convivium, De Gruyter, vol. 3(2), pages 1-15, January.
    9. Michael Chirico & Robert Inman & Charles Loeffler & John MacDonald & Holger Sieg, 2017. "Deterring Tax Delinquency in Philadelphia," NBER Working Papers 23243, National Bureau of Economic Research, Inc.
    10. Vojtech Bartos & Michal Bauer & Julie Chytilova & Filip Matejka, 2013. "Attention Discrimination: Theory and Field Experiments," CERGE-EI Working Papers wp499, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    11. Breitmoser, Yves & Vorjohann, Pauline, 2018. "Welfare-Based Altruism," Rationality and Competition Discussion Paper Series 89, CRC TRR 190 Rationality and Competition.
    12. Michael L. Anderson & Jeremy Magruder, 2017. "Split-Sample Strategies for Avoiding False Discoveries," NBER Working Papers 23544, National Bureau of Economic Research, Inc.
    13. van der Klaauw, Bas, 2014. "From micro data to causality: Forty years of empirical labor economics," Labour Economics, Elsevier, vol. 30(C), pages 88-97.
    14. Axel Franzen & Sonja Pointner, 2013. "The external validity of giving in the dictator game," Experimental Economics, Springer;Economic Science Association, vol. 16(2), pages 155-169, June.
    15. Rodney J. Andrews & Trevon D. Logan & Michael J. Sinkey, 2012. "Identifying Confirmatory Bias in the Field: Evidence from a Poll of Experts," NBER Working Papers 18064, National Bureau of Economic Research, Inc.
    16. Stefano DellaVigna & Devin Pope, 2016. "What Motivates Effort? Evidence and Expert Forecasts," NBER Working Papers 22193, National Bureau of Economic Research, Inc.
    17. Stefano Dellavigna & John A. List & Ulrike Malmendier & Gautam Rao, 2017. "Voting to Tell Others," Review of Economic Studies, Oxford University Press, vol. 84(1), pages 143-181.

    More about this item

    JEL classification:

    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:jecper:v:25:y:2011:i:3:p:39-62. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Michael P. Albert). General contact details of provider: http://edirc.repec.org/data/aeaaaea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.