IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/17047.html
   My bibliography  Save this paper

The Role of Theory in Field Experiments

Author

Listed:
  • David Card
  • Stefano DellaVigna
  • Ulrike Malmendier

Abstract

We propose a new classification of experiments that captures the extent to which the experimental design and analysis are linked to economic theory. We then use this system to classify all published field experiments in the five top economics journals from 1975 to 2010. We find that the vast majority of field experiments (68%) are Descriptive studies that lack any explicit model; 18% are Single Model studies that test a single model-based hypothesis; 6% are Competing Models studies that test competing model-based hypotheses; and 8% are Parameter Estimation studies that estimate structural parameters in a completely specified model. Using the same system to classify laboratory experiments published over the same period, we find that economic theory has played a more central role in the laboratory than in the field. Finally, we discuss in detail three sets of field experiments, on gift exchange, on charitable giving, and on negative income tax, that illustrate both the benefits and the potential costs of a tighter link between experimental design and theoretical underpinnings.

Suggested Citation

  • David Card & Stefano DellaVigna & Ulrike Malmendier, 2011. "The Role of Theory in Field Experiments," NBER Working Papers 17047, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:17047
    Note: CH ED EEE HC HE IO LE LS PE POL
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w17047.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    2. Ernst Fehr & Georg Kirchsteiger & Arno Riedl, 1993. "Does Fairness Prevent Market Clearing? An Experimental Investigation," The Quarterly Journal of Economics, Oxford University Press, vol. 108(2), pages 437-459.
    3. Stefano DellaVigna & John A. List & Ulrike Malmendier, 2012. "Testing for Altruism and Social Pressure in Charitable Giving," The Quarterly Journal of Economics, Oxford University Press, vol. 127(1), pages 1-56.
    4. David K. Levine, 1998. "Modeling Altruism and Spitefulness in Experiment," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 1(3), pages 593-622, July.
    5. Tuma, Nancy Brandon & Robins, Philip K, 1980. "A Dynamic Model of Employment Behavior: An Application to the Seattle and Denver Income Maintenance Experiments," Econometrica, Econometric Society, vol. 48(4), pages 1031-1052, May.
    6. Oriana Bandiera & Iwan Barankay & Imran Rasul, 2009. "Social Connections and Incentives in the Workplace: Evidence From Personnel Data," Econometrica, Econometric Society, vol. 77(4), pages 1047-1094, July.
    7. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    8. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records," American Economic Review, American Economic Association, vol. 80(3), pages 313-336, June.
    9. Angus Deaton, 2010. "Understanding the Mechanisms of Economic Development," Journal of Economic Perspectives, American Economic Association, vol. 24(3), pages 3-16, Summer.
    10. Ernst Fehr & Lorenz Goette, 2007. "Do Workers Work More if Wages Are High? Evidence from a Randomized Field Experiment," American Economic Review, American Economic Association, vol. 97(1), pages 298-317, March.
    11. George A. Akerlof, 1982. "Labor Contracts as Partial Gift Exchange," The Quarterly Journal of Economics, Oxford University Press, vol. 97(4), pages 543-569.
    12. Axel Ockenfels & Gary E. Bolton, 2000. "ERC: A Theory of Equity, Reciprocity, and Competition," American Economic Review, American Economic Association, vol. 90(1), pages 166-193, March.
    13. Oriana Bandiera & Iwan Barankay & Imran Rasul, 2007. "Incentives for Managers and Inequality among Workers: Evidence from a Firm-Level Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 122(2), pages 729-773.
    14. Terrance Odean, 1998. "Are Investors Reluctant to Realize Their Losses?," Journal of Finance, American Finance Association, vol. 53(5), pages 1775-1798, October.
    15. repec:pri:rpdevs:deaton_understanding_mechanisms_of_economic_development_with_abstract_apr is not listed on IDEAS
    16. Nicholas Barberis & Wei Xiong, 2009. "What Drives the Disposition Effect? An Analysis of a Long-Standing Preference-Based Explanation," Journal of Finance, American Finance Association, vol. 64(2), pages 751-784, April.
    17. Michael C. Keeley & Philip K. Robins, 1980. "Experimental Design, the Conlisk-Watts Assignment Model, and the Proper Estimation of Behavioral Response," Journal of Human Resources, University of Wisconsin Press, vol. 15(4), pages 480-498.
    18. Terry R. Johnson & John H. Pencavel, 1982. "Forecasting the Effects of a Negative Income Tax Program," ILR Review, Cornell University, ILR School, vol. 35(2), pages 221-234, January.
    19. repec:pri:rpdevs:deaton_understanding_mechanisms_of_economic_development_with_abstract_apr_2010 is not listed on IDEAS
    20. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records: Errata," American Economic Review, American Economic Association, vol. 80(5), pages 1284-1286, December.
    21. Bruce Sacerdote, 2001. "Peer Effects with Random Assignment: Results for Dartmouth Roommates," The Quarterly Journal of Economics, Oxford University Press, vol. 116(2), pages 681-704.
    22. Uri Gneezy & John A List, 2006. "Putting Behavioral Economics to Work: Testing for Gift Exchange in Labor Markets Using Field Experiments," Econometrica, Econometric Society, vol. 74(5), pages 1365-1384, September.
    23. Edward Miguel & Michael Kremer, 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities," Econometrica, Econometric Society, vol. 72(1), pages 159-217, January.
    24. Daniel S. Nagin & James B. Rebitzer & Seth Sanders & Lowell J. Taylor, 2002. "Monitoring, Motivation, and Management: The Determinants of Opportunistic Behavior in a Field Experiment," American Economic Review, American Economic Association, vol. 92(4), pages 850-873, September.
    25. Rabin, Matthew, 1993. "Incorporating Fairness into Game Theory and Economics," American Economic Review, American Economic Association, vol. 83(5), pages 1281-1302, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Stefano DellaVigna & John List & Ulrike Malmendier & Gautam Rao, 2016. "Estimating Social Preferences and Gift Exchange at Work," Natural Field Experiments 00586, The Field Experiments Website.
    2. Michael L. Anderson & Jeremy Magruder, 2017. "Split-Sample Strategies for Avoiding False Discoveries," NBER Working Papers 23544, National Bureau of Economic Research, Inc.
    3. repec:oup:restud:v:84:y:2017:i:1:p:143-181. is not listed on IDEAS
    4. Belot, Michèle & James, Jonathan, 2016. "Partner selection into policy relevant field experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 123(C), pages 31-56.
    5. Altmann, Steffen & Traxler, Christian, 2014. "Nudges at the dentist," European Economic Review, Elsevier, vol. 72(C), pages 19-38.
    6. Vojtech Bartos & Michal Bauer & Julie Chytilova & Filip Matejka, 2013. "Attention Discrimination: Theory and Field Experiments," CERGE-EI Working Papers wp499, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    7. van der Klaauw, Bas, 2014. "From micro data to causality: Forty years of empirical labor economics," Labour Economics, Elsevier, vol. 30(C), pages 88-97.
    8. Axel Franzen & Sonja Pointner, 2013. "The external validity of giving in the dictator game," Experimental Economics, Springer;Economic Science Association, vol. 16(2), pages 155-169, June.
    9. Stefano Dellavigna & John A. List & Ulrike Malmendier & Gautam Rao, 2017. "Voting to Tell Others," Review of Economic Studies, Oxford University Press, vol. 84(1), pages 143-181.
    10. Trier Damgaard, Mette & Gravert, Christina, 2016. "The hidden costs of nudging: Experimental evidence from reminders in fundraising," Working Papers in Economics 650, University of Gothenburg, Department of Economics.
    11. Günther Fink & Margaret McConnell & Sebastian Vollmer, 2014. "Testing for heterogeneous treatment effects in experimental data: false discovery risks and correction procedures," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(1), pages 44-57, January.
    12. repec:eee:ecolet:v:168:y:2018:i:c:p:56-60 is not listed on IDEAS
    13. Bonev, Petyo & Glachant, Matthieu & Söderberg, Magnus, 2018. "A Mechanism for Institutionalised Threat of Regulation: Evidence from the Swedish District Heating Market," Economics Working Paper Series 1805, University of St. Gallen, School of Economics and Political Science.
    14. Michael Chirico & Robert Inman & Charles Loeffler & John MacDonald & Holger Sieg, 2017. "Deterring Tax Delinquency in Philadelphia," NBER Working Papers 23243, National Bureau of Economic Research, Inc.
    15. Breitmoser, Yves & Vorjohann, Pauline, 2018. "Welfare-Based Altruism," Rationality and Competition Discussion Paper Series 89, CRC TRR 190 Rationality and Competition.
    16. Stefano DellaVigna & Devin Pope, 2018. "What Motivates Effort? Evidence and Expert Forecasts," Review of Economic Studies, Oxford University Press, vol. 85(2), pages 1029-1069.
    17. David Hirshleife, 2015. "Behavioral Finance," Annual Review of Financial Economics, Annual Reviews, vol. 7(1), pages 133-159, December.
    18. Madsen Paul E., 2013. "Evaluating Accounting Standards: A Comment on Ramanna’s ‘The International Politics of IFRS Harmonization’," Accounting, Economics, and Law: A Convivium, De Gruyter, vol. 3(2), pages 1-15, January.
    19. Rodney J. Andrews & Trevon D. Logan & Michael J. Sinkey, 2012. "Identifying Confirmatory Bias in the Field: Evidence from a Poll of Experts," NBER Working Papers 18064, National Bureau of Economic Research, Inc.

    More about this item

    JEL classification:

    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:17047. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (). General contact details of provider: http://edirc.repec.org/data/nberrus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.