IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp8282.html
   My bibliography  Save this paper

A Permutation Test and Estimation Alternatives for the Regression Kink Design

Author

Listed:
  • Ganong, Peter

    (Harvard University)

  • Jäger, Simon

    (Massachusetts Institute of Technology)

Abstract

The Regression Kink (RK) design is an increasingly popular empirical method, with more than 20 studies circulated using RK in the last 5 years since the initial circulation of Card, Lee, Pei and Weber (2012). We document empirically that these estimates, which typically use local linear regression, are highly sensitive to curvature in the underlying relationship between the outcome and the assignment variable. As an alternative inference procedure, motivated by randomization inference, we propose that researchers construct a distribution of placebo estimates in regions without a policy kink. We apply our procedure to three empirical RK applications – two administrative UI datasets with true policy kinks and the 1980 Census, which has no policy kinks – and we find that statistical significance based on conventional p-values may be spurious. In contrast, our permutation test reinforces the asymptotic inference results of a recent Regression Discontinuity study and a Difference-in-Difference study. Finally, we propose estimating RK models with a modified cubic splines framework and test the performance of different estimators in a simulation exercise. Cubic specifications – in particular recently proposed robust estimators (Calonico, Cattaneo and Titiunik 2014) – yield short interval lengths with good coverage rates.

Suggested Citation

  • Ganong, Peter & Jäger, Simon, 2014. "A Permutation Test and Estimation Alternatives for the Regression Kink Design," IZA Discussion Papers 8282, Institute of Labor Economics (IZA).
  • Handle: RePEc:iza:izadps:dp8282
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp8282.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jennifer R. Peck, 2017. "Can Hiring Quotas Work? The Effect of the Nitaqat Program on the Saudi Private Sector," American Economic Journal: Economic Policy, American Economic Association, vol. 9(2), pages 316-347, May.
    2. Fe, Eduardo & Hollingsworth, Bruce, 2012. "Estimating the eect of retirement on mental health via panel discontinuity designs," MPRA Paper 38162, University Library of Munich, Germany.
    3. David E. Card & David S. Lee & Zhuan Pei & Andrea Weber, 2012. "Nonlinear Policy Rules and the Identification and Estimation of Causal Effects in a Generalized Regression Kink Design," NRN working papers 2012-14, The Austrian Center for Labor Economics and the Analysis of the Welfare State, Johannes Kepler University Linz, Austria.
    4. Sebastian Calonico & Matias D. Cattaneo & Rocio Titiunik, 2014. "Robust Nonparametric Confidence Intervals for Regression‐Discontinuity Designs," Econometrica, Econometric Society, vol. 82, pages 2295-2326, November.
    5. Michihito Ando, 2017. "How much should we trust regression-kink-design estimates?," Empirical Economics, Springer, vol. 53(3), pages 1287-1322, November.
    6. Heléne Lundqvist & Matz Dahlberg & Eva Mörk, 2014. "Stimulating Local Public Employment: Do General Grants Work?," American Economic Journal: Economic Policy, American Economic Association, vol. 6(1), pages 167-192, February.
    7. Maisy Wong, 2013. "Estimating Ethnic Preferences Using Ethnic Housing Quotas in Singapore," Review of Economic Studies, Oxford University Press, vol. 80(3), pages 1178-1214.
    8. Marianne Simonsen & Lars Skipper & Niels Skipper, 2016. "Price Sensitivity of Demand for Prescription Drugs: Exploiting a Regression Kink Design," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(2), pages 320-337, March.
    9. Caughey, Devin & Sekhon, Jasjeet S., 2011. "Elections and the Regression Discontinuity Design: Lessons from Close U.S. House Races, 1942–2008," Political Analysis, Cambridge University Press, vol. 19(4), pages 385-408.
    10. Dong, Yingying, 2010. "Jumpy or Kinky? Regression Discontinuity without the Discontinuity," MPRA Paper 25461, University Library of Munich, Germany.
    11. Dong, Yingying, 2010. "Jumpy or Kinky? Regression Discontinuity without the Discontinuity," MPRA Paper 25427, University Library of Munich, Germany.
    12. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    13. Per Engström & Katarina Nordblom & Henry Ohlsson & Annika Persson, 2015. "Tax Compliance and Loss Aversion," American Economic Journal: Economic Policy, American Economic Association, vol. 7(4), pages 132-164, November.
    14. Guido W. Imbens & Paul R. Rosenbaum, 2005. "Robust, accurate confidence intervals with a weak instrument: quarter of birth and education," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(1), pages 109-126, January.
    15. Yixiao Sun, 2005. "Adaptive Estimation of the Regression Discontinuity Model," Econometrics 0506003, University Library of Munich, Germany.
    16. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, Oxford University Press, vol. 119(1), pages 249-275.
    17. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    18. J. D. Tena & Jan Fidrmuc, 2013. "National minimum wage and employment of young workers in the UK," Discussion Papers 12, Central European Labour Studies Institute (CELSI).
    19. Arindrajit Dube & T. William Lester & Michael Reich, 2010. "Minimum Wage Effects Across State Borders: Estimates Using Contiguous Counties," The Review of Economics and Statistics, MIT Press, vol. 92(4), pages 945-964, November.
    20. Sebastian Garmann, 2014. "The causal effect of coalition governments on fiscal policies: evidence from a Regression Kink Design," Applied Economics, Taylor & Francis Journals, vol. 46(36), pages 4490-4507, December.
    21. Andrew Hanson, 2012. "The Incidence of the Mortgage Interest Deduction: Evidence from the Market for Home Purchase Loans," Public Finance Review, , vol. 40(3), pages 339-359, May.
    22. Ho, Daniel E. & Imai, Kosuke, 2006. "Randomization Inference With Natural Experiments: An Analysis of Ballot Effects in the 2003 California Recall Election," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 888-900, September.
    23. Petri Böckerman & Ohto Kanninen & Ilpo Suoniemi, 2014. "A Kink that Makes You Sick: The Incentive Effect of Sick Pay on Absence," Working Papers 292, Työn ja talouden tutkimus LABORE, The Labour Institute for Economic Research LABORE.
    24. Kristensen, Søren Rud & Fe, Eduardo & Bech, Mickael & Mainz, Jan, 2013. "Is the quality of hospital care price sensitive? Regression kink estimates from a volume dependent price setting scheme," DaCHE discussion papers 2013:4, University of Southern Denmark, Dache - Danish Centre for Health Economics.
    25. Lee, David S., 2008. "Randomized experiments from non-random selection in U.S. House elections," Journal of Econometrics, Elsevier, vol. 142(2), pages 675-697, February.
    26. Benjamin M. Marx & Lesley J. Turner, 2015. "Borrowing Trouble? Student Loans, the Cost of Borrowing, and Implications for the Effectiveness of Need-Based Grant Aid," NBER Working Papers 20850, National Bureau of Economic Research, Inc.
    27. Will Dobbie & Paige Marta Skiba, 2013. "Information Asymmetries in Consumer Credit Markets: Evidence from Payday Lending," American Economic Journal: Applied Economics, American Economic Association, vol. 5(4), pages 256-282, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jonas Kolsrud & Camille Landais & Peter Nilsson & Johannes Spinnewijn, 2018. "The Optimal Timing of Unemployment Benefits: Theory and Evidence from Sweden," American Economic Review, American Economic Association, vol. 108(4-5), pages 985-1033, April.
    2. David Card & David S. Lee & Zhuan Pei & Andrea Weber, 2015. "Inference on Causal Effects in a Generalized Regression Kink Design," Econometrica, Econometric Society, vol. 83, pages 2453-2483, November.
    3. Jonas Kolsrud & Camille Landais & Peter Nilsson & Johannes Spinnewijn, 2015. "The Optimal Timing of UI Benefits: Theory and Evidence from Sweden," CEP Discussion Papers dp1361, Centre for Economic Performance, LSE.
    4. David Card & Andrew Johnston & Pauline Leung & Alexandre Mas & Zhuan Pei, 2015. "The Effect of Unemployment Benefits on the Duration of Unemployment Insurance Receipt: New Evidence from a Regression Kink Design in Missouri, 2003-2013," American Economic Review, American Economic Association, vol. 105(5), pages 126-130, May.
    5. Petri Bockerman & Ohto Kanninen & Ilpo Suoniemi, 2014. "A Kink that Makes you Sick: the Effect of Sick Pay on Absence in a Social Insurance System," Discussion Papers 97, Aboa Centre for Economics.
    6. David Card & David S. Lee & Zhuan Pei & Andrea Weber, 2017. "Regression Kink Design: Theory and Practice," Advances in Econometrics, in: Regression Discontinuity Designs, volume 38, pages 341-382, Emerald Group Publishing Limited.
    7. Camille Landais, 2015. "Assessing the Welfare Effects of Unemployment Benefits Using the Regression Kink Design," American Economic Journal: Economic Policy, American Economic Association, vol. 7(4), pages 243-278, November.
    8. George B. Bulman & Caroline M. Hoxby, 2015. "The Returns to the Federal Tax Credits for Higher Education," Tax Policy and the Economy, University of Chicago Press, vol. 29(1), pages 13-88.
    9. Alexander Gelber & Timothy J. Moore & Alexander Strand, 2017. "The Effect of Disability Insurance Payments on Beneficiaries' Earnings," American Economic Journal: Economic Policy, American Economic Association, vol. 9(3), pages 229-261, August.
    10. Therese C. Scharlemann & Stephen H. Shore, 2015. "The Effect of Negative Equity on Mortgage Default: Evidence from HAMP PRA," Working Papers 15-06, Office of Financial Research, US Department of the Treasury.
    11. D. G. C. Britto, 2016. "Unemployment Insurance and the Duration of Employment: Evidence from a Regression Kink Design," Working Papers wp1058, Dipartimento Scienze Economiche, Universita' di Bologna.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jaeger, Simon C & Ganong, Peter Nathan, 2014. "A Permutation Test and Estimation Alternatives for the Regression Kink Design," Scholarly Articles 34222894, Harvard University Department of Economics.
    2. Peter Ganong & Simon Jäger, 2018. "A Permutation Test for the Regression Kink Design," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 494-504, April.
    3. Sarah H. Bana & Kelly Bedard & Maya Rossin‐Slater, 2020. "The Impacts of Paid Family Leave Benefits: Regression Kink Evidence from California Administrative Data," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(4), pages 888-929, September.
    4. Quoc-Anh Do & Yen-Teik Lee & Bang Dang Nguyen, 2016. "Directors as Connectors: The Impact of the External Networks of Directors on Firms," Sciences Po publications 52, Sciences Po.
    5. Michihito Ando, 2017. "How much should we trust regression-kink-design estimates?," Empirical Economics, Springer, vol. 53(3), pages 1287-1322, November.
    6. Barbara Engels & Johannes Geyer & Peter Haan, 2016. "Pension Incentives and Early Retirement," Discussion Papers of DIW Berlin 1617, DIW Berlin, German Institute for Economic Research.
    7. Quoc-Anh Do & Yen-Teik Lee & Bang Dang Nguyen, 2016. "Directors as Connectors: The Impact of the External Networks of Directors on Firms," Post-Print hal-03393196, HAL.
    8. Huber, Martin, 2019. "An introduction to flexible methods for policy evaluation," FSES Working Papers 504, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    9. Sebastian Garmann, 2014. "The causal effect of coalition governments on fiscal policies: evidence from a Regression Kink Design," Applied Economics, Taylor & Francis Journals, vol. 46(36), pages 4490-4507, December.
    10. repec:hal:spmain:info:hdl:2441/5q8d3q8agf8hdbs42laqdfujkb is not listed on IDEAS
    11. Adam C. Sales & Ben B. Hansen, 2020. "Limitless Regression Discontinuity," Journal of Educational and Behavioral Statistics, , vol. 45(2), pages 143-174, April.
    12. Eduardo Fé & Bruce Hollingsworth, 2016. "Short- and long-run estimates of the local effects of retirement on health," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(4), pages 1051-1067, October.
    13. David Card & David S. Lee & Zhuan Pei & Andrea Weber, 2015. "Inference on Causal Effects in a Generalized Regression Kink Design," Econometrica, Econometric Society, vol. 83, pages 2453-2483, November.
    14. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    15. Camille Landais, 2015. "Assessing the Welfare Effects of Unemployment Benefits Using the Regression Kink Design," American Economic Journal: Economic Policy, American Economic Association, vol. 7(4), pages 243-278, November.
    16. Likai Chen & Georg Keilbar & Liangjun Su & Weining Wang, 2023. "Tests for Many Treatment Effects in Regression Discontinuity Panel Data Models," Papers 2312.01162, arXiv.org.
    17. Mauricio Villamizar‐Villegas & Freddy A. Pinzon‐Puerto & Maria Alejandra Ruiz‐Sanchez, 2022. "A comprehensive history of regression discontinuity designs: An empirical survey of the last 60 years," Journal of Economic Surveys, Wiley Blackwell, vol. 36(4), pages 1130-1178, September.
    18. Prakash, Nishith & Rockmore, Marc & Uppal, Yogesh, 2019. "Do criminally accused politicians affect economic outcomes? Evidence from India," Journal of Development Economics, Elsevier, vol. 141(C).
    19. Bartalotti Otávio, 2019. "Regression Discontinuity and Heteroskedasticity Robust Standard Errors: Evidence from a Fixed-Bandwidth Approximation," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-26, January.
    20. David Slichter, 2023. "The employment effects of the minimum wage: A selection ratio approach to measuring treatment effects," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(3), pages 334-357, April.
    21. Baum-Snow, Nathaniel & Ferreira, Fernando, 2015. "Causal Inference in Urban and Regional Economics," Handbook of Regional and Urban Economics, in: Gilles Duranton & J. V. Henderson & William C. Strange (ed.), Handbook of Regional and Urban Economics, edition 1, volume 5, chapter 0, pages 3-68, Elsevier.

    More about this item

    Keywords

    cubic splines; randomization inference; placebo test;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C13 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Estimation: General
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C31 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions; Social Interaction Models

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp8282. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.