IDEAS home Printed from https://ideas.repec.org/p/qsh/wpaper/174531.html
   My bibliography  Save this paper

A Permutation Test for the Regression Kink Design

Author

Listed:
  • Peter Ganong
  • Simon Jäger

Abstract

The Regression Kink (RK) design is an increasingly popular empirical method for causal inference. Analogous to the Regression Discontinuity design, which evaluates discontinuous changes in the level of an outcome variable with respect to the running variable at a point at which the level of a policy changes, the RK design evaluates discontinuous changes in the slope of an outcome variable with respect to the running variable at a kink point at which the slope of a policy with respect to the running variable changes. We document empirically that RK estimates are highly sensitive to nonlinearity in the underlying relationship between the outcome and the assignment variable. As an alternative to standard inference, we propose that researchers construct a distribution of placebo estimates in regions with and without a policy kink and use this distribution to gauge statistical significance. Under the assumption that the location of the kink point is random, this permutation test has exact size in finite samples for testing a sharp null hypothesis of no effect of the policy on the outcome. In simulation studies with policy kinks, we find that statistical significance based on conventional standard errors may be spurious. In contrast, our permutation test has exact size even in the presence of non-linearity.

Suggested Citation

  • Peter Ganong & Simon Jäger, 2015. "A Permutation Test for the Regression Kink Design," Working Paper 174531, Harvard University OpenScholar.
  • Handle: RePEc:qsh:wpaper:174531
    as

    Download full text from publisher

    File URL: http://scholar.harvard.edu/jaeger/node/174531
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. J. P. Florens & J. J. Heckman & C. Meghir & E. Vytlacil, 2008. "Identification of Treatment Effects Using Control Functions in Models With Continuous, Endogenous Treatment and Heterogeneous Effects," Econometrica, Econometric Society, vol. 76(5), pages 1191-1206, September.
    2. Camille Landais, 2015. "Assessing the Welfare Effects of Unemployment Benefits Using the Regression Kink Design," American Economic Journal: Economic Policy, American Economic Association, vol. 7(4), pages 243-278, November.
    3. David Card & David S. Lee & Zhuan Pei & Andrea Weber, 2015. "Inference on Causal Effects in a Generalized Regression Kink Design," Econometrica, Econometric Society, vol. 83, pages 2453-2483, November.
    4. Helena Skyt Nielsen & Torben Sørensen & Christopher Taber, 2010. "Estimating the Effect of Student Aid on College Enrollment: Evidence from a Government Grant Policy Reform," NBER Chapters,in: Income Taxation, Trans-Atlantic Public Economics Seminar (TAPES), pages 185-215 National Bureau of Economic Research, Inc.
    5. Calonico, Sebastian & Cattaneo, Matias D. & Titiunik, Rocío, 2014. "Robust data-driven inference in the regression-discontinuity design," Stata Journal, StataCorp LP, vol. 14(4).
    6. Raj Chetty & Adam Looney & Kory Kroft, 2009. "Salience and Taxation: Theory and Evidence," American Economic Review, American Economic Association, vol. 99(4), pages 1145-1177, September.
    7. Per Engström & Katarina Nordblom & Henry Ohlsson & Annika Persson, 2015. "Tax Compliance and Loss Aversion," American Economic Journal: Economic Policy, American Economic Association, vol. 7(4), pages 132-164, November.
    8. Sebastian Calonico & Matias D. Cattaneo & Rocio Titiunik, 2014. "Robust Nonparametric Confidence Intervals for Regression‐Discontinuity Designs," Econometrica, Econometric Society, vol. 82, pages 2295-2326, November.
    9. David Card & Andrew Johnston & Pauline Leung & Alexandre Mas & Zhuan Pei, 2015. "The Effect of Unemployment Benefits on the Duration of Unemployment Insurance Receipt: New Evidence from a Regression Kink Design in Missouri, 2003-2013," American Economic Review, American Economic Association, vol. 105(5), pages 126-130, May.
    10. Michihito Ando, 2017. "How much should we trust regression-kink-design estimates?," Empirical Economics, Springer, vol. 53(3), pages 1287-1322, November.
    11. Jens Ludwig & Douglas L. Miller, 2007. "Does Head Start Improve Children's Life Chances? Evidence from a Regression Discontinuity Design," The Quarterly Journal of Economics, Oxford University Press, vol. 122(1), pages 159-208.
    12. Ho, Daniel E. & Imai, Kosuke, 2006. "Randomization Inference With Natural Experiments: An Analysis of Ballot Effects in the 2003 California Recall Election," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 888-900, September.
    13. Sebastian Calonico & Matias D. Cattaneo & Rocío Titiunik, 2015. "Optimal Data-Driven Regression Discontinuity Plots," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1753-1769, December.
    14. Jonas Kolsrud & Camille Landais & Peter Nilsson & Johannes Spinnewijn, 2018. "The Optimal Timing of Unemployment Benefits: Theory and Evidence from Sweden," American Economic Review, American Economic Association, vol. 108(4-5), pages 985-1033, April.
    15. Guido Imbens & Karthik Kalyanaraman, 2012. "Optimal Bandwidth Choice for the Regression Discontinuity Estimator," Review of Economic Studies, Oxford University Press, vol. 79(3), pages 933-959.
    16. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, Oxford University Press, vol. 119(1), pages 249-275.
    17. Lee, David S., 2008. "Randomized experiments from non-random selection in U.S. House elections," Journal of Econometrics, Elsevier, vol. 142(2), pages 675-697, February.
    18. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    19. Andrew Gelman & Guido Imbens, 2014. "Why High-order Polynomials Should not be Used in Regression Discontinuity Designs," NBER Working Papers 20405, National Bureau of Economic Research, Inc.
    20. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2014. "Finite Population Causal Standard Errors," NBER Working Papers 20325, National Bureau of Economic Research, Inc.
    21. Guido W. Imbens & Paul R. Rosenbaum, 2005. "Robust, accurate confidence intervals with a weak instrument: quarter of birth and education," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(1), pages 109-126.
    22. D. G. C. Britto, 2016. "Unemployment Insurance and the Duration of Employment: Evidence from a Regression Kink Design," Working Papers wp1058, Dipartimento Scienze Economiche, Universita' di Bologna.
    23. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    24. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. repec:spr:empeco:v:53:y:2017:i:3:d:10.1007_s00181-016-1155-8 is not listed on IDEAS
    2. Ivan A. Canay & Vishal Kamat, 2015. "Approximate permutation tests and induced order statistics in the regression discontinuity design," CeMMAP working papers CWP27/15, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    3. David Card & David S. Lee & Zhuan Pei & Andrea Weber, 2015. "Inference on Causal Effects in a Generalized Regression Kink Design," Econometrica, Econometric Society, vol. 83, pages 2453-2483, November.
    4. Yoici Arai & Taisuke Otsu & Myung Hwan Seo, 2019. "Causal inference on regression discontinuity designs by high-dimensional methods," STICERD - Econometrics Paper Series 601, Suntory and Toyota International Centres for Economics and Related Disciplines, LSE.
    5. David Card & Andrew Johnston & Pauline Leung & Alexandre Mas & Zhuan Pei, 2015. "The Effect of Unemployment Benefits on the Duration of Unemployment Insurance Receipt: New Evidence from a Regression Kink Design in Missouri, 2003-2013," American Economic Review, American Economic Association, vol. 105(5), pages 126-130, May.
    6. Michihito Ando, 2017. "How much should we trust regression-kink-design estimates?," Empirical Economics, Springer, vol. 53(3), pages 1287-1322, November.
    7. Böckerman, Petri & Kanninen, Ohto & Suoniemi, Ilpo, 2014. "A Kink that Makes You Sick: the Effect of Sick Pay on Absence in a Social Insurance System," MPRA Paper 61010, University Library of Munich, Germany.
    8. George B. Bulman & Caroline M. Hoxby, 2015. "The Returns to the Federal Tax Credits for Higher Education," Tax Policy and the Economy, University of Chicago Press, vol. 29(1), pages 13-88.
    9. repec:tsj:stataj:y:17:y:2017:i:3:p:630-651 is not listed on IDEAS

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:qsh:wpaper:174531. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Richard Brandon). General contact details of provider: http://edirc.repec.org/data/cbrssus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.