IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/17062.html
   My bibliography  Save this paper

Mechanism Experiments and Policy Evaluations

Author

Listed:
  • Jens Ludwig
  • Jeffrey R. Kling
  • Sendhil Mullainathan

Abstract

Randomized controlled trials are increasingly used to evaluate policies. How can we make these experiments as useful as possible for policy purposes? We argue greater use should be made of experiments that identify behavioral mechanisms that are central to clearly specified policy questions, what we call "mechanism experiments." These types of experiments can be of great policy value even if the intervention that is tested (or its setting) does not correspond exactly to any realistic policy option.

Suggested Citation

  • Jens Ludwig & Jeffrey R. Kling & Sendhil Mullainathan, 2011. "Mechanism Experiments and Policy Evaluations," NBER Working Papers 17062, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:17062
    Note: POL
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w17062.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    2. Bisakha Sen & Stephen Mennemeyer & Lisa C. Gary, 2009. "The Relationship Between Neighborhood Quality and Obesity Among Children," NBER Working Papers 14985, National Bureau of Economic Research, Inc.
    3. James J. Heckman, 2010. "Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 356-398, June.
    4. Glenn W. Harrison & John A. List, 2004. "Field Experiments," Journal of Economic Literature, American Economic Association, vol. 42(4), pages 1009-1055, December.
    5. Lisa Sanbonmatsu & Jeffrey R. Kling & Greg J. Duncan & Jeanne Brooks-Gunn, 2006. "Neighborhoods and Academic Achievement: Results from the Moving to Opportunity Experiment," Journal of Human Resources, University of Wisconsin Press, vol. 41(4).
    6. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records," American Economic Review, American Economic Association, vol. 80(3), pages 313-336, June.
    7. Jane G. Fortson & Lisa Sanbonmatsu, 2010. "Child Health and Neighborhood Conditions: Results from a Randomized Housing Voucher Experiment," Journal of Human Resources, University of Wisconsin Press, vol. 45(4), pages 840-864.
    8. DiNardo, John & Lee, David S., 2011. "Program Evaluation and Research Designs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 5, pages 463-536, Elsevier.
    9. Jeffrey R. Kling & Jens Ludwig & Lawrence F. Katz, 2005. "Neighborhood Effects on Crime for Female and Male Youth: Evidence from a Randomized Housing Voucher Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 120(1), pages 87-130.
    10. Esther Duflo & Emmanuel Saez, 2003. "The Role of Information and Social Interactions in Retirement Plan Decisions: Evidence from a Randomized Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 118(3), pages 815-842.
    11. Katz, Lawrence & Duncan, Greg J. & Kling, Jeffrey R. & Kessler, Ronald C. & Ludwig, Jens & Sanbonmatsu, Lisa & Liebman, Jeffrey B., 2008. "What Can We Learn about Neighborhood Effects from the Moving to Opportunity Experiment?," Scholarly Articles 2766959, Harvard University Department of Economics.
    12. Edward P. Lazear, 2001. "Educational Production," The Quarterly Journal of Economics, Oxford University Press, vol. 116(3), pages 777-803.
    13. Christopher Jepsen & Steven Rivkin, 2009. "Class Size Reduction and Student Achievement: The Potential Tradeoff between Teacher Quality and Class Size," Journal of Human Resources, University of Wisconsin Press, vol. 44(1).
    14. Diane Whitmore Schanzenbach, 2006. "What Have Researchers Learned from Project STAR?," Working Papers 0606, Harris School of Public Policy Studies, University of Chicago.
    15. Abhijit V. Banerjee & Esther Duflo, 2009. "The Experimental Approach to Development Economics," Annual Review of Economics, Annual Reviews, vol. 1(1), pages 151-178, May.
    16. Meyer, Bruce D, 1995. "Natural and Quasi-experiments in Economics," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(2), pages 151-161, April.
    17. Justine S. Hastings & Lydia Tejeda-Ashton, 2008. "Financial Literacy, Information, and Demand Elasticity: Survey and Experimental Evidence from Mexico," NBER Working Papers 14538, National Bureau of Economic Research, Inc.
    18. Elizabeth A. Stuart & Stephen R. Cole & Catherine P. Bradshaw & Philip J. Leaf, 2011. "The use of propensity scores to assess the generalizability of results from randomized trials," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(2), pages 369-386, April.
    19. Joseph Hotz, V. & Imbens, Guido W. & Mortimer, Julie H., 2005. "Predicting the efficacy of future training programs using past experiences at other locations," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 241-270.
    20. Kenneth I. Wolpin, 2007. "Ex Ante Policy Evaluation, Structural Estimation and Model Selection," American Economic Review, American Economic Association, vol. 97(2), pages 48-52, May.
    21. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records: Errata," American Economic Review, American Economic Association, vol. 80(5), pages 1284-1286, December.
    22. Philip Oreopoulos & Kjell G. Salvanes, 2009. "How large are returns to schooling? Hint: Money isn't everything," NBER Working Papers 15339, National Bureau of Economic Research, Inc.
    23. Anup Malani, 2006. "Identifying Placebo Effects with Data from Clinical Trials," Journal of Political Economy, University of Chicago Press, vol. 114(2), pages 236-256, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dionissi Aliprantis, 2011. "Assessing the evidence on neighborhood effects from moving to opportunity," Working Papers (Old Series) 1101, Federal Reserve Bank of Cleveland.
    2. Ludwig, Jens & Duncan, Greg J. & Katz, Lawrence F. & Kessler, Ronald & Kling, Jeffrey R. & Gennetian, Lisa A. & Sanbonmatsu, Lisa, 2012. "Neighborhood Effects on the Long-Term Well-Being of Low-Income Adults," Scholarly Articles 11870359, Harvard University Department of Economics.
    3. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    4. Duo Qin & Yanqun Zhang, 2013. "A History of Polyvalent Structural Parameters: the Case of Instrument Variable Estimators," Working Papers 183, Department of Economics, SOAS, University of London, UK.
    5. Guido W. Imbens, 2010. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 399-423, June.
    6. Jennifer Gippel & Tom Smith & Yushu Zhu, 2015. "Endogeneity in Accounting and Finance Research: Natural Experiments as a State-of-the-Art Solution," Abacus, Accounting Foundation, University of Sydney, vol. 51(2), pages 143-168, June.
    7. Angus S. Deaton, 2009. "Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development," NBER Working Papers 14690, National Bureau of Economic Research, Inc.
    8. Caterina Gennaioli & Ralf Martin & Mirabelle Muûls, 2013. "Using micro data to examine causal effects of climate policy," Chapters, in: Roger Fouquet (ed.), Handbook on Energy and Climate Change, chapter 20, pages 453-470, Edward Elgar Publishing.
    9. Dionissi Aliprantis, 2017. "Assessing the evidence on neighborhood effects from Moving to Opportunity," Empirical Economics, Springer, vol. 52(3), pages 925-954, May.
    10. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    11. Steve Gibbons & Max Nathan & Henry G. Overman, 2014. "Evaluating Spatial Policies," SERC Policy Papers 012, Spatial Economics Research Centre, LSE.
    12. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    13. Kling, Jeffrey R., 2007. "Methodological Frontiers of Public Finance Field Experiments," National Tax Journal, National Tax Association;National Tax Journal, vol. 60(1), pages 109-127, March.
    14. Seals, Richard Alan & Stern, Liliana V., 2013. "Cognitive ability and the division of labor in urban ghettos: Evidence from gang activity in U.S. data," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 44(C), pages 140-149.
    15. Sebastian Galiani & Martín A. Rossi & Ernesto Schargrodsky, 2011. "Conscription and Crime: Evidence from the Argentine Draft Lottery," American Economic Journal: Applied Economics, American Economic Association, vol. 3(2), pages 119-136, April.
    16. Guido Imbens, 2019. "Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics," NBER Working Papers 26104, National Bureau of Economic Research, Inc.
    17. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    18. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    19. Joshua D. Angrist & Alan B. Krueger, 2001. "Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments," Journal of Economic Perspectives, American Economic Association, vol. 15(4), pages 69-85, Fall.
    20. Bogliacino, Francesco & Codagnone, Cristiano, 2021. "Microfoundations, behaviour, and evolution: Evidence from experiments," Structural Change and Economic Dynamics, Elsevier, vol. 56(C), pages 372-385.

    More about this item

    JEL classification:

    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:17062. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.