IDEAS home Printed from https://ideas.repec.org/a/oup/restud/v85y2018i3p1577-1608..html
   My bibliography  Save this article

Approximate Permutation Tests and Induced Order Statistics in the Regression Discontinuity Design*

* This paper is a replication of an original study

Author

Listed:
  • Ivan A Canay
  • Vishal Kamat

Abstract

In the regression discontinuity design (RDD), it is common practice to assess the credibility of the design by testing whether the means of baseline covariates do not change at the cut-off (or threshold) of the running variable. This practice is partly motivated by the stronger implication derived by Lee (2008), who showed that under certain conditions the distribution of baseline covariates in the RDD must be continuous at the cut-off. We propose a permutation test based on the so-called induced ordered statistics for the null hypothesis of continuity of the distribution of baseline covariates at the cut-off; and introduce a novel asymptotic framework to analyse its properties. The asymptotic framework is intended to approximate a small sample phenomenon: even though the total number $n$ of observations may be large, the number of effective observations local to the cut-off is often small. Thus, while traditional asymptotics in RDD require a growing number of observations local to the cut-off as $n\to \infty$, our framework keeps the number $q$ of observations local to the cut-off fixed as $n\to \infty$. The new test is easy to implement, asymptotically valid under weak conditions, exhibits finite sample validity under stronger conditions than those needed for its asymptotic validity, and has favourable power properties relative to tests based on means. In a simulation study, we find that the new test controls size remarkably well across designs. We then use our test to evaluate the plausibility of the design in Lee (2008), a well-known application of the RDD to study incumbency advantage.

Suggested Citation

  • Ivan A Canay & Vishal Kamat, 2018. "Approximate Permutation Tests and Induced Order Statistics in the Regression Discontinuity Design," Review of Economic Studies, Oxford University Press, vol. 85(3), pages 1577-1608.
  • Handle: RePEc:oup:restud:v:85:y:2018:i:3:p:1577-1608.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/restud/rdx062
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Frandsen, Brigham R. & Frölich, Markus & Melly, Blaise, 2012. "Quantile treatment effects in the regression discontinuity design," Journal of Econometrics, Elsevier, vol. 168(2), pages 382-395.
    2. Miguel Urquiola & Eric Verhoogen, 2009. "Class-Size Caps, Sorting, and the Regression-Discontinuity Design," American Economic Review, American Economic Association, vol. 99(1), pages 179-215, March.
    3. David S. Lee & Thomas Lemieux, 2010. "Regression Discontinuity Designs in Economics," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 281-355, June.
    4. Federico A. Bugni & Ivan A. Canay & Azeem M. Shaikh, 2018. "Inference Under Covariate-Adaptive Randomization," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(524), pages 1784-1796, October.
    5. Douglas Almond & Joseph J. Doyle & Amanda E. Kowalski & Heidi Williams, 2010. "Estimating Marginal Returns to Medical Care: Evidence from At-risk Newborns," The Quarterly Journal of Economics, Oxford University Press, vol. 125(2), pages 591-634.
    6. François Gerard & Miikka Rokkanen & Christoph Rothe, 2020. "Bounds on treatment effects in regression discontinuity designs with a manipulated running variable," Quantitative Economics, Econometric Society, vol. 11(3), pages 839-870, July.
    7. Sebastian Calonico & Matias D. Cattaneo & Rocio Titiunik, 2014. "Robust Nonparametric Confidence Intervals for Regression‐Discontinuity Designs," Econometrica, Econometric Society, vol. 82, pages 2295-2326, November.
    8. Shu Shen & Xiaohan Zhang, 2016. "Distributional Tests for Regression Discontinuity: Theory and Empirical Examples," The Review of Economics and Statistics, MIT Press, vol. 98(4), pages 685-700, October.
    9. Ivan A. Canay & Joseph P. Romano & Azeem M. Shaikh, 2017. "Randomization Tests Under an Approximate Symmetry Assumption," Econometrica, Econometric Society, vol. 85, pages 1013-1030, May.
    10. Gerard, Francois & Rokkanen, Miikka & Rothe, Christoph, 2016. "Bounds on Treatment Effects in Regression Discontinuity Designs under Manipulation of the Running Variable, with an Application to Unemployment Insurance in Brazil," CEPR Discussion Papers 11668, C.E.P.R. Discussion Papers.
    11. David Card & Carlos Dobkin & Nicole Maestas, 2008. "The Impact of Nearly Universal Insurance Coverage on Health Care Utilization: Evidence from Medicare," American Economic Review, American Economic Association, vol. 98(5), pages 2242-2258, December.
    12. Melissa Dell, 2010. "The Persistent Effects of Peru's Mining Mita," Econometrica, Econometric Society, vol. 78(6), pages 1863-1903, November.
    13. Guido Imbens & Karthik Kalyanaraman, 2012. "Optimal Bandwidth Choice for the Regression Discontinuity Estimator," Review of Economic Studies, Oxford University Press, vol. 79(3), pages 933-959.
    14. Lee, David S., 2008. "Randomized experiments from non-random selection in U.S. House elections," Journal of Econometrics, Elsevier, vol. 142(2), pages 675-697, February.
    15. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    16. McCrary, Justin, 2008. "Manipulation of the running variable in the regression discontinuity design: A density test," Journal of Econometrics, Elsevier, vol. 142(2), pages 698-714, February.
    17. Peter Ganong & Simon Jäger, 2018. "A Permutation Test for the Regression Kink Design," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 494-504, April.
    18. Hahn, Jinyong & Todd, Petra & Van der Klaauw, Wilbert, 2001. "Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design," Econometrica, Econometric Society, vol. 69(1), pages 201-209, January.
    19. Seth D. Zimmerman, 2014. "The Returns to College Admission for Academically Marginal Students," Journal of Labor Economics, University of Chicago Press, vol. 32(4), pages 711-754.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Goldman, Matt & Kaplan, David M., 2018. "Comparing distributions by multiple testing across quantiles or CDF values," Journal of Econometrics, Elsevier, vol. 206(1), pages 143-166.
    2. Yoichi Arai & Yu-Chin Hsu & Toru Kitagawa & Ismael Mourifié & Yuanyuan Wan, 2018. "Testing identifying assumptions in fuzzy regression discontinuity designs," CeMMAP working papers CWP50/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    3. Yingying DONG & Ying-Ying LEE & Michael GOU, 2019. "Regression Discontinuity Designs with a Continuous Treatment," Discussion papers 19058, Research Institute of Economy, Trade and Industry (RIETI).
    4. Matias D. Cattaneo & Rocio Titiunik, 2021. "Regression Discontinuity Designs," Papers 2108.09400, arXiv.org.
    5. Giuntella, Osea & Mazzonna, Fabrizio, 2019. "Sunset time and the economic effects of social jetlag: evidence from US time zone borders," Journal of Health Economics, Elsevier, vol. 65(C), pages 210-226.
    6. Iizuka, Toshiaki & Nishiyama, Katsuhiko & Chen, Brian & Eggleston, Karen, 2021. "False alarm? Estimating the marginal value of health signals," Journal of Public Economics, Elsevier, vol. 195(C).
    7. Chen, Heng & Huynh, Kim P. & Shy, Oz, 2019. "Cash versus card: Payment discontinuities and the burden of holding coins," Journal of Banking & Finance, Elsevier, vol. 99(C), pages 192-201.
    8. Bugni, Federico A. & Canay, Ivan A., 2021. "Testing continuity of a density via g-order statistics in the regression discontinuity design," Journal of Econometrics, Elsevier, vol. 221(1), pages 138-159.
    9. Zhang, Yali & Li, Wenqi & Wu, Feng, 2020. "Does energy transition improve air quality? Evidence derived from China’s Winter Clean Heating Pilot (WCHP) project," Energy, Elsevier, vol. 206(C).
    10. Atila Abdulkadiroglu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak, 2019. "Breaking Ties: Regression Discontinuity Design Meets Market Design," Cowles Foundation Discussion Papers 2170R, Cowles Foundation for Research in Economics, Yale University, revised Dec 2020.
    11. Fernando Aragón & Ricardo Pique, 2020. "Better the devil you know? Reelected politicians and policy outcomes under no term limits," Public Choice, Springer, vol. 182(1), pages 1-16, January.
    12. Adam C. Sales & Ben B. Hansen, 2020. "Limitless Regression Discontinuity," Journal of Educational and Behavioral Statistics, , vol. 45(2), pages 143-174, April.
    13. Yang Lixiong, 2019. "Regression discontinuity designs with unknown state-dependent discontinuity points: estimation and testing," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 23(2), pages 1-18, April.
    14. Chen, Heng & Fan, Yanqin, 2019. "Identification and wavelet estimation of weighted ATE under discontinuous and kink incentive assignment mechanisms," Journal of Econometrics, Elsevier, vol. 212(2), pages 476-502.
    15. Harding, Robin & Prem, Mounu & Ruiz, Nelson A. & Vargas, David L., 2021. "Buying a Blind Eye: Campaign Donations, Forbearance, and Deforestation in Colombia," Working papers 84, Red Investigadores de Economía.
    16. Makarin, Alexey & Piqué, Ricardo & Aragón, Fernando, 2020. "National or sub-national parties: Does party geographic scope matter?," Journal of Development Economics, Elsevier, vol. 146(C).
    17. Blaise Melly & Rafael Lalive, 2020. "Estimation, Inference, and Interpretation in the Regression Discontinuity Design," Diskussionsschriften dp2016, Universitaet Bern, Departement Volkswirtschaft.
    18. Bonilla-Mejía, Leonardo & Higuera-Mendieta, Iván, 2019. "Protected Areas under Weak Institutions: Evidence from Colombia," World Development, Elsevier, vol. 122(C), pages 585-596.
    19. Yong Cai & Ivan A. Canay & Deborah Kim & Azeem M. Shaikh, 2021. "On the implementation of Approximate Randomization Tests in Linear Models with a Small Number of Clusters," Papers 2102.09058, arXiv.org, revised Oct 2021.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Blaise Melly & Rafael Lalive, 2020. "Estimation, Inference, and Interpretation in the Regression Discontinuity Design," Diskussionsschriften dp2016, Universitaet Bern, Departement Volkswirtschaft.
    2. Bugni, Federico A. & Canay, Ivan A., 2021. "Testing continuity of a density via g-order statistics in the regression discontinuity design," Journal of Econometrics, Elsevier, vol. 221(1), pages 138-159.
    3. Prakash, Nishith & Rockmore, Marc & Uppal, Yogesh, 2019. "Do criminally accused politicians affect economic outcomes? Evidence from India," Journal of Development Economics, Elsevier, vol. 141(C).
    4. Montoya, Ana Maria & Noton, Carlos & Solis, Alex, 2018. "The Returns to College Choice: Loans, Scholarships and Labor Outcomes," Working Paper Series 2018:12, Uppsala University, Department of Economics.
    5. Deng, Taotao & Hu, Yukun & Ma, Mulan, 2019. "Regional policy and tourism: A quasi-natural experiment," Annals of Tourism Research, Elsevier, vol. 74(C), pages 1-16.
    6. Xu, Ke-Li, 2017. "Regression discontinuity with categorical outcomes," Journal of Econometrics, Elsevier, vol. 201(1), pages 1-18.
    7. Dong, Yingying, 2010. "Jumpy or Kinky? Regression Discontinuity without the Discontinuity," MPRA Paper 25461, University Library of Munich, Germany.
    8. Christelis, Dimitris & Georgarakos, Dimitris & Sanz-de-Galdeano, Anna, 2020. "The impact of health insurance on stockholding: A regression discontinuity approach," Journal of Health Economics, Elsevier, vol. 69(C).
    9. Joaquín Artés & Ignacio Jurado, 2018. "Government fragmentation and fiscal deficits: a regression discontinuity approach," Public Choice, Springer, vol. 175(3), pages 367-391, June.
    10. Crespo Cristian, 2020. "Beyond Manipulation: Administrative Sorting in Regression Discontinuity Designs," Journal of Causal Inference, De Gruyter, vol. 8(1), pages 164-181, January.
    11. Takahide Yanagi, 2014. "The Effect of Measurement Error in the Sharp Regression Discontinuity Design," KIER Working Papers 910, Kyoto University, Institute of Economic Research.
    12. Steven F. Koch & Jeffrey S. Racine, 2016. "Healthcare facility choice and user fee abolition: regression discontinuity in a multinomial choice setting," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(4), pages 927-950, October.
    13. Bartalotti Otávio, 2019. "Regression Discontinuity and Heteroskedasticity Robust Standard Errors: Evidence from a Fixed-Bandwidth Approximation," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-26, January.
    14. Mauricio Villamizar-Villegas & Freddy A. Pinzón-Puerto & María Alejandra Ruiz-Sánchez, 2020. "A Comprehensive History of Regression Discontinuity Designs: An Empirical Survey of the last 60 Years," Borradores de Economia 1112, Banco de la Republica de Colombia.
    15. Jin-young Choi & Myoung-jae Lee, 2017. "Regression discontinuity: review with extensions," Statistical Papers, Springer, vol. 58(4), pages 1217-1246, December.
    16. Angelo D'Andrea, 2019. "Mayor’s wage and Public procurement," BAFFI CAREFIN Working Papers 19125, BAFFI CAREFIN, Centre for Applied Research on International Markets Banking Finance and Regulation, Universita' Bocconi, Milano, Italy.
    17. Bertanha, Marinho & Moreira, Marcelo J., 2020. "Impossible inference in econometrics: Theory and applications," Journal of Econometrics, Elsevier, vol. 218(2), pages 247-270.
    18. Crespo Cristian, 2020. "Beyond Manipulation: Administrative Sorting in Regression Discontinuity Designs," Journal of Causal Inference, De Gruyter, vol. 8(1), pages 164-181, January.
    19. Cappelletti, Giuseppe & Ponte Marques, Aurea & Varraso, Paolo & Budrys, Žymantas & Peeters, Jonas, 2019. "Impact of higher capital buffers on banks’ lending and risk-taking: evidence from the euro area experiments," Working Paper Series 2292, European Central Bank.
    20. Guido Imbens & Stefan Wager, 2019. "Optimized Regression Discontinuity Designs," The Review of Economics and Statistics, MIT Press, vol. 101(2), pages 264-278, May.

    Replication

    This item is a replication of:
  • Lee, David S., 2008. "Randomized experiments from non-random selection in U.S. House elections," Journal of Econometrics, Elsevier, vol. 142(2), pages 675-697, February.
  • More about this item

    Keywords

    Regression discontinuity design; Permutation tests; Randomization tests; Induced ordered statistics; Rank tests;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General

    Lists

    This item is featured on the following reading lists, Wikipedia, or ReplicationWiki pages:
    1. Approximate Permutation Tests and Induced Order Statistics in the Regression Discontinuity Design (REStud 2018) in ReplicationWiki

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:restud:v:85:y:2018:i:3:p:1577-1608.. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press or Christopher F. Baum (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.