Advanced Search
MyIDEAS: Login to save this paper or follow this series

The Credibility Revolution in Empirical Economics: How Better Research Design is Taking the Con out of Econometrics

Contents:

Author Info

  • Joshua D. Angrist
  • Jörn-Steffen Pischke

Abstract

This essay reviews progress in empirical economics since Leamer’s (1983) critique. Leamer highlighted the benefits of sensitivity analysis, a procedure in which researchers show how their results change with changes in specification or functional form. Sensitivity analysis has had a salutary but not a revolutionary effect on econometric practice. As we see it, the credibility revolution in empirical work can be traced to the rise of a design-based approach that emphasizes the identification of causal effects. Design-based studies typically feature either real or natural experiments and are distinguished by their prima facie credibility and by the attention investigators devote to making the case for a causal interpretation of the findings their designs generate. Design-based studies are most often found in the microeconomic fields of Development, Education, Environment, Labor, Health, and Public Finance, but are still rare in Industrial Organization and Macroeconomics. We explain why IO and Macro would do well to embrace a design-based approach. Finally, we respond to the charge that the design-based revolution has overreached.

(This abstract was borrowed from another version of this item.)

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://www.ratswd.de/download/RatSWD_WP_2010/RatSWD_WP_142.pdf
Download Restriction: no

Bibliographic Info

Paper provided by German Council for Social and Economic Data (RatSWD) in its series Working Paper Series of the German Council for Social and Economic Data with number 142.

as in new window
Length: 40
Date of creation: 2010
Date of revision:
Handle: RePEc:rsw:rswwps:rswwps142

Contact details of provider:
Email:
Web page: http://www.ratswd.de/eng/index.html
More information through EDIRC

Related research

Keywords:

Other versions of this item:

Find related papers by JEL classification:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Angus Deaton, 2009. "Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development," Working Papers 1122, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
  2. Finn E. Kydland & Edward C. Prescott, 1994. "The computational experiment: an econometric tool," Staff Report 178, Federal Reserve Bank of Minneapolis.
  3. Ashenfelter, Orley, 1987. "The case for evaluating training programs with randomized trials," Economics of Education Review, Elsevier, vol. 6(4), pages 333-338, August.
  4. Urquiola, Miguel & Verhoogen, Eric, 2007. "Class Size and Sorting in Market Equilibrium: Theory and Evidence," IZA Discussion Papers 2963, Institute for the Study of Labor (IZA).
  5. Karlan, Dean S. & Zinman, Jonathan, 2007. "Observing Unobservables: Identifying Information Asymmetries with a Consumer Credit Field Experiment," CEPR Discussion Papers 6182, C.E.P.R. Discussion Papers.
  6. James Heckman & Sergio Urzua, 2010. "Comparing IV with structural models: what simple IV can and cannot identify," CeMMAP working papers CWP08/10, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  7. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records," American Economic Review, American Economic Association, vol. 80(3), pages 313-36, June.
  8. Hanushek, Eric A, 1986. "The Economics of Schooling: Production and Efficiency in Public Schools," Journal of Economic Literature, American Economic Association, vol. 24(3), pages 1141-77, September.
  9. Passell, Peter & Taylor, John B, 1977. "The Deterrent Effect of Capital Punishment: Another View," American Economic Review, American Economic Association, vol. 67(3), pages 445-51, June.
  10. Meyer, Bruce D, 1995. "Natural and Quasi-experiments in Economics," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(2), pages 151-61, April.
  11. John J. Donohue III & Justin Wolfers, 2006. "Uses and Abuses of Empirical Evidence in the Death Penalty Debate," NBER Working Papers 11982, National Bureau of Economic Research, Inc.
  12. Feldstein, Martin & Horioka, Charles, 1980. "Domestic Saving and International Capital Flows," Economic Journal, Royal Economic Society, vol. 90(358), pages 314-29, June.
  13. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-38, May.
  14. Card, David & Krueger, Alan B, 1992. "School Quality and Black-White Relative Earnings: A Direct Assessment," The Quarterly Journal of Economics, MIT Press, vol. 107(1), pages 151-200, February.
  15. Jeffrey R Kling & Jeffrey B Liebman & Lawrence F Katz, 2007. "Experimental Analysis of Neighborhood Effects," Econometrica, Econometric Society, vol. 75(1), pages 83-119, 01.
  16. N. Gregory Mankiw, 2006. "The Macroeconomist as Scientist and Engineer," Journal of Economic Perspectives, American Economic Association, vol. 20(4), pages 29-46, Fall.
  17. Orley Ashenfelter, 1990. "Non-Parametric Estimates of the Labor Supply Effects of Negative Income Tax Programs," Working Papers 639, Princeton University, Department of Economics, Industrial Relations Section..
  18. Stefano DellaVigna & Ulrike Malmendier, 2006. "Paying Not to Go to the Gym," American Economic Review, American Economic Association, vol. 96(3), pages 694-719, June.
  19. Isaac Ehrlich, 1973. "The Deterrent Effect of Capital Punishment: A Question of Life and Death," NBER Working Papers 0018, National Bureau of Economic Research, Inc.
  20. Joshua D. Angrist & Victor Lavy, 1999. "Using Maimonides' Rule To Estimate The Effect Of Class Size On Scholastic Achievement," The Quarterly Journal of Economics, MIT Press, vol. 114(2), pages 533-575, May.
  21. Nathan Nunn, 2008. "The Long-Term Effects of Africa's Slave Trades," The Quarterly Journal of Economics, MIT Press, vol. 123(1), pages 139-176, 02.
  22. Stefano DellaVigna & M. Daniele Paserman, 2004. "Job Search and Impatience," NBER Working Papers 10837, National Bureau of Economic Research, Inc.
  23. Brian A. Jacob, 2004. "Public Housing, Housing Vouchers, and Student Achievement: Evidence from Public Housing Demolitions in Chicago," American Economic Review, American Economic Association, vol. 94(1), pages 233-258, March.
  24. Summers, Anita A & Wolfe, Barbara L, 1977. "Do Schools Make a Difference?," American Economic Review, American Economic Association, vol. 67(4), pages 639-52, September.
  25. Eric A. Hanushek & John F. Kain & Steven G. Rivkin, 1998. "Teachers, Schools, and Academic Achievement," NBER Working Papers 6691, National Bureau of Economic Research, Inc.
  26. McCrary, Justin, 2008. "Manipulation of the running variable in the regression discontinuity design: A density test," Journal of Econometrics, Elsevier, vol. 142(2), pages 698-714, February.
  27. Angrist, Joshua & Chen, Stacey, 2008. "Long-Term Economic Consequences of Vietnam-Era Conscription: Schooling, Experience and Earnings," IZA Discussion Papers 3628, Institute for the Study of Labor (IZA).
  28. Christina D. Romer & David H. Romer, 2010. "The Macroeconomic Effects of Tax Changes: Estimates Based on a New Measure of Fiscal Shocks," American Economic Review, American Economic Association, vol. 100(3), pages 763-801, June.
  29. Aviv Nevo, 2000. "Mergers with Differentiated Products: The Case of the Ready-to-Eat Cereal Industry," RAND Journal of Economics, The RAND Corporation, vol. 31(3), pages 395-421, Autumn.
  30. Daron Acemoglu & Simon Johnson & James A. Robinson, 2001. "The Colonial Origins of Comparative Development: An Empirical Investigation," American Economic Review, American Economic Association, vol. 91(5), pages 1369-1401, December.
  31. Ehrlich, Isaac, 1977. "Capital Punishment and Deterrence: Some Further Thoughts and Additional Evidence," Journal of Political Economy, University of Chicago Press, vol. 85(4), pages 741-88, August.
  32. François Velde, 2006. "Chronicles of a deflation unforetold," Working Paper Series WP-06-12, Federal Reserve Bank of Chicago.
  33. Raghuram G. Rajan, 2005. "Aid and Growth: What Does The Cross-Country Evidence Really Show?," Working Papers id:54, eSocialSciences.
  34. Benjamin J. Keys & Tanmoy Mukherjee & Amit Seru & Vikrant Vig, 2010. "Did Securitization Lead to Lax Screening? Evidence from Subprime Loans," The Quarterly Journal of Economics, MIT Press, vol. 125(1), pages 307-362, February.
  35. Manning, Willard G, et al, 1987. "Health Insurance and the Demand for Medical Care: Evidence from a Randomized Experiment," American Economic Review, American Economic Association, vol. 77(3), pages 251-77, June.
  36. Valerie A. Ramey & Matthew D. Shapiro, 1999. "Costly Capital Reallocation and the Effects of Government Spending," NBER Working Papers 6283, National Bureau of Economic Research, Inc.
  37. Paul Schultz, T., 2004. "School subsidies for the poor: evaluating the Mexican Progresa poverty program," Journal of Development Economics, Elsevier, vol. 74(1), pages 199-250, June.
  38. Gary Richardson & William Troost, 2009. "Monetary Intervention Mitigated Banking Panics during the Great Depression: Quasi-Experimental Evidence from a Federal Reserve District Border, 1929-1933," Journal of Political Economy, University of Chicago Press, vol. 117(6), pages 1031-1073, December.
  39. Orley C. Ashenfelter & Daniel Hosken & Matthew Weinberg, 2009. "Generating Evidence to Guide Merger Enforcement," Working Papers 1137, Princeton University, Department of Economics, Center for Economic Policy Studies..
  40. Sims, Christopher A, 1988. "Uncertainty across Models," American Economic Review, American Economic Association, vol. 78(2), pages 163-67, May.
  41. Kydland, Finn E & Prescott, Edward C, 1977. "Rules Rather Than Discretion: The Inconsistency of Optimal Plans," Journal of Political Economy, University of Chicago Press, vol. 85(3), pages 473-91, June.
  42. Ernst Fehr & Lorenz Götte, 2005. "Do Workers Work More if Wages are High? Evidence from a Randomized Field Experiment," IEW - Working Papers 125, Institute for Empirical Research in Economics - University of Zurich.
  43. Orley Ashenfelter & Daniel Hosken, 2008. "The Effect of Mergers on Consumer Prices: Evidence from Five Selected Case Studies," Working Papers 1037, Princeton University, Department of Economics, Center for Economic Policy Studies..
  44. Hoyt Bleakley, 2007. "Disease and Development: Evidence from Hookworm Eradication in the American South," The Quarterly Journal of Economics, MIT Press, vol. 122(1), pages 73-117, 02.
  45. Paul Gertler, 2004. "Do Conditional Cash Transfers Improve Child Health? Evidence from PROGRESA's Control Randomized Experiment," American Economic Review, American Economic Association, vol. 94(2), pages 336-341, May.
  46. David Card, 1988. "Unexpected Inflation, Real Wages, and Employment Determination in Union Contracts," Working Papers 612, Princeton University, Department of Economics, Industrial Relations Section..
  47. Gary Solon, 1982. "Work Incentive Effects of Taxing Employment Benefits," Working Papers 529, Princeton University, Department of Economics, Industrial Relations Section..
  48. Guido Imbens & Thomas Lemieux, 2007. "Regression Discontinuity Designs: A Guide to Practice," NBER Technical Working Papers 0337, National Bureau of Economic Research, Inc.
  49. Jonathan B. Baker, 2003. "The Case for Antitrust Enforcement," Journal of Economic Perspectives, American Economic Association, vol. 17(4), pages 27-50, Fall.
  50. Borenstein, Severin, 1990. "Airline Mergers, Airport Dominance, and Market Power," American Economic Review, American Economic Association, vol. 80(2), pages 400-404, May.
  51. Hausman, Jerry A & Leonard, Gregory K, 2002. "The Competitive Effects of a New Product Introduction: A Case Study," Journal of Industrial Economics, Wiley Blackwell, vol. 50(3), pages 237-63, September.
  52. Olivier Deschênes & Michael Greenstone, 2007. "Climate Change, Mortality, and Adaptation: Evidence from Annual Fluctuations in Weather in the US," Working Papers 0707, Massachusetts Institute of Technology, Center for Energy and Environmental Policy Research.
  53. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
  54. Crandall, Robert W. & Winston, Clifford, 2004. "Does Antitrust Policy Improve Consumer Welfare? Assessing the Evidence," Working paper 263, Regulation2point0.
  55. Dani Rodrik & Romain Wacziarg, 2005. "Do Democratic Transitions Produce Bad Economic Outcomes?," American Economic Review, American Economic Association, vol. 95(2), pages 50-55, May.
  56. David Y. Albouy, 2008. "The Colonial Origins of Comparative Development: An Investigation of the Settler Mortality Data," NBER Working Papers 14130, National Bureau of Economic Research, Inc.
  57. Ehrlich, Isaac & Liu, Zhiqiang, 1999. "Sensitivity Analyses of the Deterrence Hypothesis: Let's Keep the Econ in Econometrics," Journal of Law and Economics, University of Chicago Press, vol. 42(1), pages 455-87, April.
  58. McAleer, Michael & Pagan, Adrian R & Volker, Paul A, 1985. "What Will Take the Con out of Econometrics?," American Economic Review, American Economic Association, vol. 75(3), pages 293-307, June.
  59. Michael Conlin & Ted O'Donoghue & Timothy J. Vogelsang, 2007. "Projection Bias in Catalog Orders," American Economic Review, American Economic Association, vol. 97(4), pages 1217-1249, September.
  60. Robert J. Barro & Charles J. Redlick, 2009. "Macroeconomic Effects from Government Purchases and Taxes," NBER Working Papers 15369, National Bureau of Economic Research, Inc.
  61. Cecilia Elena Rouse, 1998. "Private School Vouchers And Student Achievement: An Evaluation Of The Milwaukee Parental Choice Program," The Quarterly Journal of Economics, MIT Press, vol. 113(2), pages 553-602, May.
  62. James Heckman, 1997. "Instrumental Variables: A Study of Implicit Behavioral Assumptions Used in Making Program Evaluations," Journal of Human Resources, University of Wisconsin Press, vol. 32(3), pages 441-462.
  63. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records: Errata," American Economic Review, American Economic Association, vol. 80(5), pages 1284-86, December.
  64. Ehrlich, Isaac & Brower, George D, 1987. "On the Issue of Causality in the Economic Model of Crime and Law Enforcement: Some Theoretical Considerations and Experimental Evidence," American Economic Review, American Economic Association, vol. 77(2), pages 99-106, May.
  65. Greenberg, David & Halsey, Harlan, 1983. "Systematic Misreporting and Effects of Income Maintenance Experiments on Work Effort: Evidence from the Seattle-Denver Experiment," Journal of Labor Economics, University of Chicago Press, vol. 1(4), pages 380-407, October.
  66. White, Halbert, 1980. "Using Least Squares to Approximate Unknown Regression Functions," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 21(1), pages 149-70, February.
  67. Leamer, Edward E, 1985. "Sensitivity Analyses Would Help," American Economic Review, American Economic Association, vol. 75(3), pages 308-13, June.
  68. Guido W. Imbens, 2009. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," NBER Working Papers 14896, National Bureau of Economic Research, Inc.
  69. Joshua D. Angrist & Guido M. Kuersteiner, 2004. "Semiparametric Causality Tests Using the Policy Propensity Score," NBER Working Papers 10975, National Bureau of Economic Research, Inc.
  70. Caroline M. Hoxby, 2000. "The Effects Of Class Size On Student Achievement: New Evidence From Population Variation," The Quarterly Journal of Economics, MIT Press, vol. 115(4), pages 1239-1285, November.
  71. Isaac Ehrlich, 1996. "Crime, Punishment, and the Market for Offenses," Journal of Economic Perspectives, American Economic Association, vol. 10(1), pages 43-67, Winter.
  72. Ehrlich, Isaac, 1977. "The Deterrent Effect of Capital Punishment: Reply," American Economic Review, American Economic Association, vol. 67(3), pages 452-58, June.
  73. Justine S. Hastings, 2004. "Vertical Relationships and Competition in Retail Gasoline Markets: Empirical Evidence from Contract Changes in Southern California," American Economic Review, American Economic Association, vol. 94(1), pages 317-328, March.
  74. Gerald S. Oettinger, 1999. "An Empirical Analysis of the Daily Labor Supply of Stadium Vendors," Journal of Political Economy, University of Chicago Press, vol. 107(2), pages 360-392, April.
  75. Milton Friedman & Anna J. Schwartz, 1963. "A Monetary History of the United States, 1867-1960," NBER Books, National Bureau of Economic Research, Inc, number frie63-1, July.
  76. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-20, September.
Full references (including those not matched with items on IDEAS)

Citations

Blog mentions

As found by EconAcademics.org, the blog aggregator for Economics research:
  1. O “núcleo” da inflação no Brasil.
    by ? in Análise Real on 2012-04-12 12:00:00
  2. Notes on books for Macroeconometrics
    by Gray in Pseudo-true News on 2013-04-08 15:49:43
  3. An incrementalist view of Impact Evaluation and knowledge
    by Jed Friedman in Development Impact on 2012-05-02 13:27:23
Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
This item has more than 25 citations. To prevent cluttering this page, these citations are listed on a separate page.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:rsw:rswwps:rswwps142. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (RatSWD) The email address of this maintainer does not seem to be valid anymore. Please ask RatSWD to update the entry or send us the correct address.

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.