IDEAS home Printed from https://ideas.repec.org/p/qsh/wpaper/174531.html
   My bibliography  Save this paper

A Permutation Test for the Regression Kink Design

Author

Listed:
  • Peter Ganong
  • Simon Jäger

Abstract

The Regression Kink (RK) design is an increasingly popular empirical method for causal inference. Analogous to the Regression Discontinuity design, which evaluates discontinuous changes in the level of an outcome variable with respect to the running variable at a point at which the level of a policy changes, the RK design evaluates discontinuous changes in the slope of an outcome variable with respect to the running variable at a kink point at which the slope of a policy with respect to the running variable changes. We document empirically that RK estimates are highly sensitive to nonlinearity in the underlying relationship between the outcome and the assignment variable. As an alternative to standard inference, we propose that researchers construct a distribution of placebo estimates in regions with and without a policy kink and use this distribution to gauge statistical significance. Under the assumption that the location of the kink point is random, this permutation test has exact size in finite samples for testing a sharp null hypothesis of no effect of the policy on the outcome. In simulation studies with policy kinks, we find that statistical significance based on conventional standard errors may be spurious. In contrast, our permutation test has exact size even in the presence of non-linearity.

Suggested Citation

  • Peter Ganong & Simon Jäger, 2015. "A Permutation Test for the Regression Kink Design," Working Paper 174531, Harvard University OpenScholar.
  • Handle: RePEc:qsh:wpaper:174531
    as

    Download full text from publisher

    File URL: http://scholar.harvard.edu/jaeger/node/174531
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. J. P. Florens & J. J. Heckman & C. Meghir & E. Vytlacil, 2008. "Identification of Treatment Effects Using Control Functions in Models With Continuous, Endogenous Treatment and Heterogeneous Effects," Econometrica, Econometric Society, vol. 76(5), pages 1191-1206, September.
    2. Camille Landais, 2015. "Assessing the Welfare Effects of Unemployment Benefits Using the Regression Kink Design," American Economic Journal: Economic Policy, American Economic Association, vol. 7(4), pages 243-278, November.
    3. David Card & David S. Lee & Zhuan Pei & Andrea Weber, 2015. "Inference on Causal Effects in a Generalized Regression Kink Design," Econometrica, Econometric Society, vol. 83, pages 2453-2483, November.
    4. Helena Skyt Nielsen & Torben Sørensen & Christopher Taber, 2010. "Estimating the Effect of Student Aid on College Enrollment: Evidence from a Government Grant Policy Reform," NBER Chapters, in: Income Taxation, Trans-Atlantic Public Economics Seminar (TAPES), pages 185-215, National Bureau of Economic Research, Inc.
    5. Sebastian Calonico & Matias D. Cattaneo & Rocio Titiunik, 2014. "Robust data-driven inference in the regression-discontinuity design," Stata Journal, StataCorp LP, vol. 14(4), pages 909-946, December.
    6. Raj Chetty & Adam Looney & Kory Kroft, 2009. "Salience and Taxation: Theory and Evidence," American Economic Review, American Economic Association, vol. 99(4), pages 1145-1177, September.
    7. Per Engström & Katarina Nordblom & Henry Ohlsson & Annika Persson, 2015. "Tax Compliance and Loss Aversion," American Economic Journal: Economic Policy, American Economic Association, vol. 7(4), pages 132-164, November.
    8. Sebastian Calonico & Matias D. Cattaneo & Rocio Titiunik, 2014. "Robust Nonparametric Confidence Intervals for Regression‐Discontinuity Designs," Econometrica, Econometric Society, vol. 82, pages 2295-2326, November.
    9. David Card & Andrew Johnston & Pauline Leung & Alexandre Mas & Zhuan Pei, 2015. "The Effect of Unemployment Benefits on the Duration of Unemployment Insurance Receipt: New Evidence from a Regression Kink Design in Missouri, 2003-2013," American Economic Review, American Economic Association, vol. 105(5), pages 126-130, May.
    10. Michihito Ando, 2017. "How much should we trust regression-kink-design estimates?," Empirical Economics, Springer, vol. 53(3), pages 1287-1322, November.
    11. Jens Ludwig & Douglas L. Miller, 2007. "Does Head Start Improve Children's Life Chances? Evidence from a Regression Discontinuity Design," The Quarterly Journal of Economics, Oxford University Press, vol. 122(1), pages 159-208.
    12. Ho, Daniel E. & Imai, Kosuke, 2006. "Randomization Inference With Natural Experiments: An Analysis of Ballot Effects in the 2003 California Recall Election," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 888-900, September.
    13. Sebastian Calonico & Matias D. Cattaneo & Rocío Titiunik, 2015. "Optimal Data-Driven Regression Discontinuity Plots," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1753-1769, December.
    14. Jonas Kolsrud & Camille Landais & Peter Nilsson & Johannes Spinnewijn, 2018. "The Optimal Timing of Unemployment Benefits: Theory and Evidence from Sweden," American Economic Review, American Economic Association, vol. 108(4-5), pages 985-1033, April.
    15. Guido Imbens & Karthik Kalyanaraman, 2012. "Optimal Bandwidth Choice for the Regression Discontinuity Estimator," Review of Economic Studies, Oxford University Press, vol. 79(3), pages 933-959.
    16. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, Oxford University Press, vol. 119(1), pages 249-275.
    17. Lee, David S., 2008. "Randomized experiments from non-random selection in U.S. House elections," Journal of Econometrics, Elsevier, vol. 142(2), pages 675-697, February.
    18. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    19. Andrew Gelman & Guido Imbens, 2019. "Why High-Order Polynomials Should Not Be Used in Regression Discontinuity Designs," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 37(3), pages 447-456, July.
    20. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2014. "Finite Population Causal Standard Errors," NBER Working Papers 20325, National Bureau of Economic Research, Inc.
    21. Guido W. Imbens & Paul R. Rosenbaum, 2005. "Robust, accurate confidence intervals with a weak instrument: quarter of birth and education," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(1), pages 109-126, January.
    22. D. G. C. Britto, 2016. "Unemployment Insurance and the Duration of Employment: Evidence from a Regression Kink Design," Working Papers wp1058, Dipartimento Scienze Economiche, Universita' di Bologna.
    23. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    24. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    2. Ganong, Peter & Jäger, Simon, 2014. "A Permutation Test and Estimation Alternatives for the Regression Kink Design," IZA Discussion Papers 8282, Institute of Labor Economics (IZA).
    3. Bana, Sarah & Bedard, Kelly & Rossin-Slater, Maya, 2018. "The Impacts of Paid Family Leave Benefits: Regression Kink Evidence from California Administrative Data," IZA Discussion Papers 11381, Institute of Labor Economics (IZA).
    4. Sarah H. Bana & Kelly Bedard & Maya Rossin‐Slater, 2020. "The Impacts of Paid Family Leave Benefits: Regression Kink Evidence from California Administrative Data," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(4), pages 888-929, September.
    5. repec:hrv:faseco:34222894 is not listed on IDEAS
    6. Michihito Ando, 2017. "How much should we trust regression-kink-design estimates?," Empirical Economics, Springer, vol. 53(3), pages 1287-1322, November.
    7. Huber, Martin, 2019. "An introduction to flexible methods for policy evaluation," FSES Working Papers 504, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    8. Adam C. Sales & Ben B. Hansen, 2020. "Limitless Regression Discontinuity," Journal of Educational and Behavioral Statistics, , vol. 45(2), pages 143-174, April.
    9. Feng, Li & Figlio, David & Sass, Tim, 2018. "School accountability and teacher mobility," Journal of Urban Economics, Elsevier, vol. 103(C), pages 1-17.
    10. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    11. Xu, Ke-Li, 2018. "A semi-nonparametric estimator of regression discontinuity design with discrete duration outcomes," Journal of Econometrics, Elsevier, vol. 206(1), pages 258-278.
    12. Van den Berge, Wiljan & Jongen, Egbert L. W. & van der Wiel, Karen, 2017. "Using Tax Deductions to Promote Lifelong Learning: Real and Shifting Responses," IZA Discussion Papers 10885, Institute of Labor Economics (IZA).
    13. Nicholas Barton & Tessa Bold & Justin Sandefur, 2017. "Measuring Rents from Public Employment: Regression Discontinuity Evidence from Kenya - Working Paper 457," Working Papers 457, Center for Global Development.
    14. Petri Böckerman & Ohto Kanninen & Ilpo Suoniemi, 2018. "A kink that makes you sick: The effect of sick pay on absence," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 33(4), pages 568-579, June.
    15. Timothy B Armstrong & Michal Kolesár, 2018. "A Simple Adjustment for Bandwidth Snooping," Review of Economic Studies, Oxford University Press, vol. 85(2), pages 732-765.
    16. Yoichi Arai & Taisuke Otsu & Myung Hwan Seo, 2021. "Regression Discontinuity Design with Potentially Many Covariates," Papers 2109.08351, arXiv.org.
    17. Chen, Heng & Fan, Yanqin, 2019. "Identification and wavelet estimation of weighted ATE under discontinuous and kink incentive assignment mechanisms," Journal of Econometrics, Elsevier, vol. 212(2), pages 476-502.
    18. Cappelletti, Giuseppe & Guazzarotti, Giovanni & Tommasino, Pietro, 2017. "The stock market effects of a securities transaction tax: Quasi-experimental evidence from Italy," Journal of Financial Stability, Elsevier, vol. 31(C), pages 81-92.
    19. Bertanha, Marinho & Moreira, Marcelo J., 2020. "Impossible inference in econometrics: Theory and applications," Journal of Econometrics, Elsevier, vol. 218(2), pages 247-270.
    20. Tomi Kyyrä & Hanna Pesola, 2020. "The Effects of UI Benefits on Unemployment and Subsequent Outcomes: Evidence from a Kinked Benefit Rule," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 82(5), pages 1135-1160, October.
    21. 'Agoston Reguly, 2021. "Heterogeneous Treatment Effects in Regression Discontinuity Designs," Papers 2106.11640, arXiv.org, revised Oct 2021.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:qsh:wpaper:174531. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/cbrssus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Richard Brandon The email address of this maintainer does not seem to be valid anymore. Please ask Richard Brandon to update the entry or send us the correct address (email available below). General contact details of provider: https://edirc.repec.org/data/cbrssus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.