IDEAS home Printed from https://ideas.repec.org/p/azt/cemmap/21-17.html
   My bibliography  Save this paper

Approximate permutation tests and induced order statistics in the regression discontinuity design

Author

Listed:
  • Ivan A. Canay
  • Vishal Kamat

Abstract

In the regression discontinuity design (RDD), it is common practice to asses the credibility of the design by testing whether the means of baseline covariates do not change at the cutoff (or threshold) of the running variable. This practice is partly motivated by the stronger im-plication derived by Lee (2008), who showed that under certain conditions the distribution of baseline covariates in the RDD must be continuous at the cutoff. We propose a permutation test based on the so-called induced ordered statistics for the null hypothesis of continuity of the distribution of baseline covariates at the cutoff; and introduce a novel asymptotic framework to analyze its properties. The asymptotic framework is intended to approximate a small sample phenomenon: even though the total number n of observations may be large, the number of effective observations local to the cutoff is often small. Thus, while traditional asymptotics in RDD require a growing number of observations local to the cutoff as n → ∞, our framework keeps the number q of observations local to the cutoff fixed as n → ∞. The new test is easy to implement, asymptotically valid under weak conditions, exhibits finite sample validity under stronger conditions than those needed for its asymptotic validity, and has favorable power properties relative to tests based on means. In a simulation study, we find that the new test controls size remarkably well across designs. We then use our test to evaluate the plausibility of the design in Lee (2008), a well-known application of the RDD to study incumbency advantage.

Suggested Citation

  • Ivan A. Canay & Vishal Kamat, 2017. "Approximate permutation tests and induced order statistics in the regression discontinuity design," CeMMAP working papers 21/17, Institute for Fiscal Studies.
  • Handle: RePEc:azt:cemmap:21/17
    DOI: 10.1920/wp.cem.2017.2117
    as

    Download full text from publisher

    File URL: https://www.cemmap.ac.uk/wp-content/uploads/2020/08/CWP2117.pdf
    Download Restriction: no

    File URL: https://libkey.io/10.1920/wp.cem.2017.2117?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Douglas Almond & Joseph J. Doyle & Amanda E. Kowalski & Heidi Williams, 2010. "Estimating Marginal Returns to Medical Care: Evidence from At-risk Newborns," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(2), pages 591-634.
    2. Sebastian Calonico & Matias D. Cattaneo & Rocio Titiunik, 2014. "Robust Nonparametric Confidence Intervals for Regression‐Discontinuity Designs," Econometrica, Econometric Society, vol. 82, pages 2295-2326, November.
    3. Shu Shen & Xiaohan Zhang, 2016. "Distributional Tests for Regression Discontinuity: Theory and Empirical Examples," The Review of Economics and Statistics, MIT Press, vol. 98(4), pages 685-700, October.
    4. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    5. Miriam Bruhn & David McKenzie, 2009. "In Pursuit of Balance: Randomization in Practice in Development Field Experiments," American Economic Journal: Applied Economics, American Economic Association, vol. 1(4), pages 200-232, October.
    6. Peter Ganong & Simon Jäger, 2018. "A Permutation Test for the Regression Kink Design," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 494-504, April.
    7. Hahn, Jinyong & Todd, Petra & Van der Klaauw, Wilbert, 2001. "Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design," Econometrica, Econometric Society, vol. 69(1), pages 201-209, January.
    8. Guido Imbens & Karthik Kalyanaraman, 2012. "Optimal Bandwidth Choice for the Regression Discontinuity Estimator," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 79(3), pages 933-959.
    9. David S. Lee & Thomas Lemieux, 2010. "Regression Discontinuity Designs in Economics," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 281-355, June.
    10. François Gerard & Miikka Rokkanen & Christoph Rothe, 2020. "Bounds on treatment effects in regression discontinuity designs with a manipulated running variable," Quantitative Economics, Econometric Society, vol. 11(3), pages 839-870, July.
    11. Ivan A. Canay & Joseph P. Romano & Azeem M. Shaikh, 2017. "Randomization Tests Under an Approximate Symmetry Assumption," Econometrica, Econometric Society, vol. 85, pages 1013-1030, May.
    12. Gerard, François & Rothe, Christoph & Rokkanen, Miikka, 2016. "Bounds on Treatment Effects in Regression Discontinuity Designs under Manipulation of the Running Variable, with an Application," CEPR Discussion Papers 11668, C.E.P.R. Discussion Papers.
    13. Lee, David S., 2008. "Randomized experiments from non-random selection in U.S. House elections," Journal of Econometrics, Elsevier, vol. 142(2), pages 675-697, February.
    14. McCrary, Justin, 2008. "Manipulation of the running variable in the regression discontinuity design: A density test," Journal of Econometrics, Elsevier, vol. 142(2), pages 698-714, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Federico A. Bugni & Jia Li & Qiyuan Li, 2023. "Permutation‐based tests for discontinuities in event studies," Quantitative Economics, Econometric Society, vol. 14(1), pages 37-70, January.
    2. James J. Heckman & Rodrigo Pinto & Azeem Shaikh, 2023. "Dealing with Imperfect Randomization: Inference for the HighScope Perry Preschool Program," NBER Working Papers 31982, National Bureau of Economic Research, Inc.
    3. Zuleika Ferre & Patricia Triunfo & José‐Ignacio Antón, 2023. "Subdermal contraceptive implants and repeat teenage motherhood: Evidence from a major maternity hospital‐based program in Uruguay," Health Economics, John Wiley & Sons, Ltd., vol. 32(12), pages 2679-2693, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ivan A Canay & Vishal Kamat, 2018. "Approximate Permutation Tests and Induced Order Statistics in the Regression Discontinuity Design," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 85(3), pages 1577-1608.
    2. Ivan A. Canay & Vishal Kamat, 2016. "Approximate permutation tests and induced order statistics in the regression discontinuity design," CeMMAP working papers 33/16, Institute for Fiscal Studies.
    3. Blaise Melly & Rafael Lalive, 2020. "Estimation, Inference, and Interpretation in the Regression Discontinuity Design," Diskussionsschriften dp2016, Universitaet Bern, Departement Volkswirtschaft.
    4. Bugni, Federico A. & Canay, Ivan A., 2021. "Testing continuity of a density via g-order statistics in the regression discontinuity design," Journal of Econometrics, Elsevier, vol. 221(1), pages 138-159.
    5. Matias D. Cattaneo & Rocío Titiunik, 2022. "Regression Discontinuity Designs," Annual Review of Economics, Annual Reviews, vol. 14(1), pages 821-851, August.
    6. Mauricio Villamizar‐Villegas & Freddy A. Pinzon‐Puerto & Maria Alejandra Ruiz‐Sanchez, 2022. "A comprehensive history of regression discontinuity designs: An empirical survey of the last 60 years," Journal of Economic Surveys, Wiley Blackwell, vol. 36(4), pages 1130-1178, September.
    7. Yoichi Arai & Yu‐Chin Hsu & Toru Kitagawa & Ismael Mourifié & Yuanyuan Wan, 2022. "Testing identifying assumptions in fuzzy regression discontinuity designs," Quantitative Economics, Econometric Society, vol. 13(1), pages 1-28, January.
    8. Prakash, Nishith & Rockmore, Marc & Uppal, Yogesh, 2019. "Do criminally accused politicians affect economic outcomes? Evidence from India," Journal of Development Economics, Elsevier, vol. 141(C).
    9. Christina Korting & Carl Lieberman & Jordan Matsudaira & Zhuan Pei & Yi Shen, 2023. "Visual Inference and Graphical Representation in Regression Discontinuity Designs," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 138(3), pages 1977-2019.
    10. Joaquín Artés & Ignacio Jurado, 2018. "Government fragmentation and fiscal deficits: a regression discontinuity approach," Public Choice, Springer, vol. 175(3), pages 367-391, June.
    11. Crespo Cristian, 2020. "Beyond Manipulation: Administrative Sorting in Regression Discontinuity Designs," Journal of Causal Inference, De Gruyter, vol. 8(1), pages 164-181, January.
    12. Montoya, Ana Maria & Noton, Carlos & Solis, Alex, 2018. "The Returns to College Choice: Loans, Scholarships and Labor Outcomes," Working Paper Series 2018:12, Uppsala University, Department of Economics.
    13. Bartalotti Otávio, 2019. "Regression Discontinuity and Heteroskedasticity Robust Standard Errors: Evidence from a Fixed-Bandwidth Approximation," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-26, January.
    14. Ivan A. Canay & Vishal Kamat, 2015. "Approximate permutation tests and induced order statistics in the regression discontinuity design," CeMMAP working papers 27/15, Institute for Fiscal Studies.
    15. Jin-young Choi & Myoung-jae Lee, 2017. "Regression discontinuity: review with extensions," Statistical Papers, Springer, vol. 58(4), pages 1217-1246, December.
    16. Angelo D'Andrea, 2019. "Mayor’s wage and Public procurement," BAFFI CAREFIN Working Papers 19125, BAFFI CAREFIN, Centre for Applied Research on International Markets Banking Finance and Regulation, Universita' Bocconi, Milano, Italy.
    17. Bertanha, Marinho & Moreira, Marcelo J., 2020. "Impossible inference in econometrics: Theory and applications," Journal of Econometrics, Elsevier, vol. 218(2), pages 247-270.
    18. Crespo Cristian, 2020. "Beyond Manipulation: Administrative Sorting in Regression Discontinuity Designs," Journal of Causal Inference, De Gruyter, vol. 8(1), pages 164-181, January.
    19. Deng, Taotao & Hu, Yukun & Ma, Mulan, 2019. "Regional policy and tourism: A quasi-natural experiment," Annals of Tourism Research, Elsevier, vol. 74(C), pages 1-16.
    20. Yiqi Liu & Yuan Qi, 2023. "Using Forests in Multivariate Regression Discontinuity Designs," Papers 2303.11721, arXiv.org, revised Jul 2024.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:azt:cemmap:21/17. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Dermot Watson (email available below). General contact details of provider: https://edirc.repec.org/data/ifsssuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.