IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/20773.html
   My bibliography  Save this paper

External Validity in Fuzzy Regression Discontinuity Designs

Author

Listed:
  • Marinho Bertanha
  • Guido W. Imbens

Abstract

Many empirical studies use Fuzzy Regression Discontinuity (FRD) designs to identify treatment effects when the receipt of treatment is potentially correlated to outcomes. Existing FRD methods identify the local average treatment effect (LATE) on the subpopulation of compliers with values of the forcing variable that are equal to the threshold. We develop methods that assess the plausibility of generalizing LATE to subpopulations other than compliers, and to subpopulations other than those with forcing variable equal to the threshold. Specifically, we focus on testing the equality of the distributions of potential outcomes for treated compliers and always-takers, and for non-treated compliers and never-takers. We show that equality of these pairs of distributions implies that the expected outcome conditional on the forcing variable and the treatment status is continuous in the forcing variable at the threshold, for each of the two treatment regimes. As a matter of routine, we recommend that researchers present graphs with estimates of these two conditional expectations in addition to graphs with estimates of the expected outcome conditional on the forcing variable alone. We illustrate our methods using data on the academic performance of students attending the summer school program in two large school districts in the US.

Suggested Citation

  • Marinho Bertanha & Guido W. Imbens, 2014. "External Validity in Fuzzy Regression Discontinuity Designs," NBER Working Papers 20773, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:20773
    Note: CH DEV ED HC IO LS PE
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w20773.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Lalive, Rafael, 2008. "How do extended benefits affect unemployment duration A regression discontinuity approach," Journal of Econometrics, Elsevier, vol. 142(2), pages 785-806, February.
    2. Sandra E. Black, 1999. "Do Better Schools Matter? Parental Valuation of Elementary Education," The Quarterly Journal of Economics, Oxford University Press, vol. 114(2), pages 577-599.
    3. Sebastian Calonico & Matias D. Cattaneo & Rocio Titiunik, 2014. "Robust data-driven inference in the regression-discontinuity design," Stata Journal, StataCorp LP, vol. 14(4), pages 909-946, December.
    4. Sebastian Calonico & Matias D. Cattaneo & Rocio Titiunik, 2014. "Robust Nonparametric Confidence Intervals for Regression‐Discontinuity Designs," Econometrica, Econometric Society, vol. 82, pages 2295-2326, November.
    5. Brian A. Jacob & Lars Lefgren, 2004. "Remedial Education and Student Achievement: A Regression-Discontinuity Analysis," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 226-244, February.
    6. Jens Ludwig & Douglas L. Miller, 2007. "Does Head Start Improve Children's Life Chances? Evidence from a Regression Discontinuity Design," The Quarterly Journal of Economics, Oxford University Press, vol. 122(1), pages 159-208.
    7. Joshua Angrist & Miikka Rokkanen, 2012. "Wanna Get Away? RD Identification Away from the Cutoff," NBER Working Papers 18662, National Bureau of Economic Research, Inc.
    8. Yingying Dong & Arthur Lewbel, 2015. "Identifying the Effect of Changing the Policy Threshold in Regression Discontinuity Models," The Review of Economics and Statistics, MIT Press, vol. 97(5), pages 1081-1092, December.
    9. Wilbert van der Klaauw, 2002. "Estimating the Effect of Financial Aid Offers on College Enrollment: A Regression-Discontinuity Approach," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 43(4), pages 1249-1287, November.
    10. repec:ags:stataj:267111 is not listed on IDEAS
    11. BERTANHA, Marinho, 2016. "Regression Discontinuity Design with Many Thresholds," CORE Discussion Papers 2016026, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    12. Andrew Gelman & Guido Imbens, 2014. "Why High-order Polynomials Should not be Used in Regression Discontinuity Designs," NBER Working Papers 20405, National Bureau of Economic Research, Inc.
    13. Doug Miller & Jens Ludwig, 2005. "Does Head Start Improve Children?s Life Chances? Evidence from a Regression Discontinuity Design," Working Papers 534, University of California, Davis, Department of Economics.
    14. Joshua D. Angrist & Alan B. Keueger, 1991. "Does Compulsory School Attendance Affect Schooling and Earnings?," The Quarterly Journal of Economics, Oxford University Press, vol. 106(4), pages 979-1014.
    15. Hahn, Jinyong & Todd, Petra & Van der Klaauw, Wilbert, 2001. "Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design," Econometrica, Econometric Society, vol. 69(1), pages 201-209, January.
    16. Matsudaira, Jordan D., 2008. "Mandatory summer school and student achievement," Journal of Econometrics, Elsevier, vol. 142(2), pages 829-850, February.
    17. Joshua D. Angrist & Victor Lavy, 1999. "Using Maimonides' Rule to Estimate the Effect of Class Size on Scholastic Achievement," The Quarterly Journal of Economics, Oxford University Press, vol. 114(2), pages 533-575.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Thomas Dee & Emily Penner, 2016. "The Causal Effects of Cultural Relevance: Evidence from an Ethnic Studies Curriculum," NBER Working Papers 21865, National Bureau of Economic Research, Inc.
    2. Dan A. Black & Joonhwi Joo & Robert LaLonde & Jeffrey Andrew Smith & Evan J. Taylor, 2017. "Simple Tests for Selection: Learning More from Instrumental Variables," CESifo Working Paper Series 6392, CESifo Group Munich.
    3. Patrick Kline & Christopher R. Walters, 2017. "On Heckits, LATE, and Numerical Equivalence," Papers 1706.05982, arXiv.org, revised Oct 2018.
    4. Christopher Jepsen & Peter Mueser & Kenneth Troske, 2017. "Second Chance for High School Dropouts? A Regression Discontinuity Analysis of Postsecondary Educational Returns to the GED," Journal of Labor Economics, University of Chicago Press, vol. 35(S1), pages 273-304.
    5. Yoichi Arai & Yu-Chin Hsu & Toru Kitagawa & Ismael Mourifie & Yuanyuan Wan, 2018. "Testing Identifying Assumptions In Fuzzy Regression Discontinuity Designs," Working Papers tecipa-623, University of Toronto, Department of Economics.
    6. Amanda E. Kowalski, 2018. "Extrapolation using Selection and Moral Hazard Heterogeneity from within the Oregon Health Insurance Experiment," Cowles Foundation Discussion Papers 2135, Cowles Foundation for Research in Economics, Yale University.
    7. Bertanha, Marinho Angelo & Moreira, Marcelo J., 2017. "Impossible inference in econometrics: theory and applications to regression discontinuity, bunching, and exogeneity tests," FGV/EPGE Economics Working Papers (Ensaios Economicos da EPGE) 787, FGV/EPGE - Escola Brasileira de Economia e Finanças, Getulio Vargas Foundation (Brazil).
    8. Thomas S. Dee & Hans Henrik Sievertsen, 2018. "The gift of time? School starting age and mental health," Health Economics, John Wiley & Sons, Ltd., vol. 27(5), pages 781-802, May.
    9. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2016. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," NBER Working Papers 22923, National Bureau of Economic Research, Inc.
    10. Amanda Kowalski, 2016. "Doing more when you're running LATE: Applying marginal treatment effect methods to examine treatment effect heterogeneity in experiments," Artefactual Field Experiments 00560, The Field Experiments Website.
    11. Gabriel Ehrlich & Jeffrey Perry, 2015. "Do Large-Scale Refinancing Programs Reduce Mortgage Defaults? Evidence From a Regression Discontinuity Design: Working Paper 2015-06," Working Papers 50871, Congressional Budget Office.
    12. Denni Tommasi & Arthur Lewbel & Rossella Calvi, 2017. "LATE with Mismeasured or Misspecified Treatment: An application to Women's Empowerment in India," Working Papers ECARES ECARES 2017-27, ULB -- Universite Libre de Bruxelles.
    13. Gerard, Francois & Rokkanen, Miikka & Rothe, Christoph, 2016. "Bounds on Treatment Effects in Regression Discontinuity Designs under Manipulation of the Running Variable, with an Application to Unemployment Insurance in Brazil," CEPR Discussion Papers 11668, C.E.P.R. Discussion Papers.
    14. Amanda E. Kowalski, 2018. "Behavior within a Clinical Trial and Implications for Mammography Guidelines," NBER Working Papers 25049, National Bureau of Economic Research, Inc.
    15. Amanda E. Kowalski, 2016. "Doing More When You're Running LATE: Applying Marginal Treatment Effect Methods to Examine Treatment Effect Heterogeneity in Experiments for the Young and Privately Insured?," Cowles Foundation Discussion Papers 2045, Cowles Foundation for Research in Economics, Yale University.
    16. Marinho Bertanha & Marcelo J. Moreira, 2016. "Impossible Inference in Econometrics: Theory and Applications," Papers 1612.02024, arXiv.org, revised Nov 2018.
    17. Susan Athey & Guido Imbens, 2016. "The Econometrics of Randomized Experiments," Papers 1607.00698, arXiv.org.
    18. Isaiah Andrews & Emily Oster, 2017. "Weighting for External Validity," NBER Working Papers 23826, National Bureau of Economic Research, Inc.
    19. Schleicher, Michael & Klonner, Stefan & Sauerborn, Rainer & Sié, Alie & Souares, Aurélia, 2018. "The Demand for Health Insurance in a Poor Economy: Evidence from Burkina Faso," Working Papers 0648, University of Heidelberg, Department of Economics.
    20. Black, Dan A. & Joo, Joonhwi & LaLonde, Robert J. & Smith, Jeffrey A. & Taylor, Evan J., 2015. "Simple Tests for Selection Bias: Learning More from Instrumental Variables," IZA Discussion Papers 9346, Institute for the Study of Labor (IZA).
    21. repec:spr:stpapr:v:58:y:2017:i:4:d:10.1007_s00362-016-0745-z is not listed on IDEAS
    22. Huber, Martin & Wüthrich, Kaspar, 2017. "Evaluating local average and quantile treatment effects under endogeneity based on instruments: a review," FSES Working Papers 479, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    23. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    24. Matias D. Cattaneo & Luke Keele & Rocio Titiunik & Gonzalo Vazquez-Bare, 2018. "Extrapolating Treatment Effects in Multi-Cutoff Regression Discontinuity Designs," Papers 1808.04416, arXiv.org.
    25. François Gerard & Miikka Rokkanen & Christoph Rothe, 2016. "Bounds on Treatment Effects in Regression Discontinuity Designs with a Manipulated Running Variable," NBER Working Papers 22892, National Bureau of Economic Research, Inc.

    More about this item

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:20773. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (). General contact details of provider: http://edirc.repec.org/data/nberrus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.