IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp8282.html
   My bibliography  Save this paper

A Permutation Test and Estimation Alternatives for the Regression Kink Design

Author

Listed:
  • Ganong, Peter

    (Harvard University)

  • Jäger, Simon

    (Massachusetts Institute of Technology)

Abstract

The Regression Kink (RK) design is an increasingly popular empirical method, with more than 20 studies circulated using RK in the last 5 years since the initial circulation of Card, Lee, Pei and Weber (2012). We document empirically that these estimates, which typically use local linear regression, are highly sensitive to curvature in the underlying relationship between the outcome and the assignment variable. As an alternative inference procedure, motivated by randomization inference, we propose that researchers construct a distribution of placebo estimates in regions without a policy kink. We apply our procedure to three empirical RK applications – two administrative UI datasets with true policy kinks and the 1980 Census, which has no policy kinks – and we find that statistical significance based on conventional p-values may be spurious. In contrast, our permutation test reinforces the asymptotic inference results of a recent Regression Discontinuity study and a Difference-in-Difference study. Finally, we propose estimating RK models with a modified cubic splines framework and test the performance of different estimators in a simulation exercise. Cubic specifications – in particular recently proposed robust estimators (Calonico, Cattaneo and Titiunik 2014) – yield short interval lengths with good coverage rates.

Suggested Citation

  • Ganong, Peter & Jäger, Simon, 2014. "A Permutation Test and Estimation Alternatives for the Regression Kink Design," IZA Discussion Papers 8282, Institute of Labor Economics (IZA).
  • Handle: RePEc:iza:izadps:dp8282
    as

    Download full text from publisher

    File URL: https://ftp.iza.org/dp8282.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jennifer R. Peck, 2017. "Can Hiring Quotas Work? The Effect of the Nitaqat Program on the Saudi Private Sector," American Economic Journal: Economic Policy, American Economic Association, vol. 9(2), pages 316-347, May.
    2. Fe, Eduardo & Hollingsworth, Bruce, 2012. "Estimating the eect of retirement on mental health via panel discontinuity designs," MPRA Paper 38162, University Library of Munich, Germany.
    3. David E. Card & David S. Lee & Zhuan Pei & Andrea Weber, 2012. "Nonlinear Policy Rules and the Identification and Estimation of Causal Effects in a Generalized Regression Kink Design," NRN working papers 2012-14, The Austrian Center for Labor Economics and the Analysis of the Welfare State, Johannes Kepler University Linz, Austria.
    4. Per Engström & Katarina Nordblom & Henry Ohlsson & Annika Persson, 2015. "Tax Compliance and Loss Aversion," American Economic Journal: Economic Policy, American Economic Association, vol. 7(4), pages 132-164, November.
    5. Sebastian Calonico & Matias D. Cattaneo & Rocio Titiunik, 2014. "Robust Nonparametric Confidence Intervals for Regression‐Discontinuity Designs," Econometrica, Econometric Society, vol. 82, pages 2295-2326, November.
    6. Michihito Ando, 2017. "How much should we trust regression-kink-design estimates?," Empirical Economics, Springer, vol. 53(3), pages 1287-1322, November.
    7. Sebastian Garmann, 2014. "The causal effect of coalition governments on fiscal policies: evidence from a Regression Kink Design," Applied Economics, Taylor & Francis Journals, vol. 46(36), pages 4490-4507, December.
    8. Heléne Lundqvist & Matz Dahlberg & Eva Mörk, 2014. "Stimulating Local Public Employment: Do General Grants Work?," American Economic Journal: Economic Policy, American Economic Association, vol. 6(1), pages 167-192, February.
    9. Andrew Hanson, 2012. "The Incidence of the Mortgage Interest Deduction: Evidence from the Market for Home Purchase Loans," Public Finance Review, , vol. 40(3), pages 339-359, May.
    10. Ho, Daniel E. & Imai, Kosuke, 2006. "Randomization Inference With Natural Experiments: An Analysis of Ballot Effects in the 2003 California Recall Election," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 888-900, September.
    11. Maisy Wong, 2013. "Estimating Ethnic Preferences Using Ethnic Housing Quotas in Singapore," Review of Economic Studies, Oxford University Press, vol. 80(3), pages 1178-1214.
    12. Caughey, Devin & Sekhon, Jasjeet S., 2011. "Elections and the Regression Discontinuity Design: Lessons from Close U.S. House Races, 1942–2008," Political Analysis, Cambridge University Press, vol. 19(4), pages 385-408.
    13. Dong, Yingying, 2010. "Jumpy or Kinky? Regression Discontinuity without the Discontinuity," MPRA Paper 25461, University Library of Munich, Germany.
    14. Dong, Yingying, 2010. "Jumpy or Kinky? Regression Discontinuity without the Discontinuity," MPRA Paper 25427, University Library of Munich, Germany.
    15. Petri Böckerman & Ohto Kanninen & Ilpo Suoniemi, 2014. "A Kink that Makes You Sick: The Incentive Effect of Sick Pay on Absence," Working Papers 292, Palkansaajien tutkimuslaitos, Labour Institute for Economic Research.
    16. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, Oxford University Press, vol. 119(1), pages 249-275.
    17. Kristensen, Søren Rud & Fe, Eduardo & Bech, Mickael & Mainz, Jan, 2013. "Is the quality of hospital care price sensitive? Regression kink estimates from a volume dependent price setting scheme," DaCHE discussion papers 2013:4, University of Southern Denmark, Dache - Danish Centre for Health Economics.
    18. Lee, David S., 2008. "Randomized experiments from non-random selection in U.S. House elections," Journal of Econometrics, Elsevier, vol. 142(2), pages 675-697, February.
    19. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    20. Jan Fidrmuc & J. D. Tena, 2013. "National Minimum Wage and Employment of Young Workers in the UK," CESifo Working Paper Series 4286, CESifo.
    21. Benjamin M. Marx & Lesley J. Turner, 2015. "Borrowing Trouble? Student Loans, the Cost of Borrowing, and Implications for the Effectiveness of Need-Based Grant Aid," NBER Working Papers 20850, National Bureau of Economic Research, Inc.
    22. Guido W. Imbens & Paul R. Rosenbaum, 2005. "Robust, accurate confidence intervals with a weak instrument: quarter of birth and education," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(1), pages 109-126, January.
    23. Will Dobbie & Paige Marta Skiba, 2013. "Information Asymmetries in Consumer Credit Markets: Evidence from Payday Lending," American Economic Journal: Applied Economics, American Economic Association, vol. 5(4), pages 256-282, October.
    24. Yixiao Sun, 2005. "Adaptive Estimation of the Regression Discontinuity Model," Econometrics 0506003, University Library of Munich, Germany.
    25. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jonas Kolsrud & Camille Landais & Peter Nilsson & Johannes Spinnewijn, 2018. "The Optimal Timing of Unemployment Benefits: Theory and Evidence from Sweden," American Economic Review, American Economic Association, vol. 108(4-5), pages 985-1033, April.
    2. David Card & David S. Lee & Zhuan Pei & Andrea Weber, 2015. "Inference on Causal Effects in a Generalized Regression Kink Design," Econometrica, Econometric Society, vol. 83, pages 2453-2483, November.
    3. Jonas Kolsrud & Camille Landais & Peter Nilsson & Johannes Spinnewijn, 2015. "The Optimal Timing of UI Benefits: Theory and Evidence from Sweden," CEP Discussion Papers dp1361, Centre for Economic Performance, LSE.
    4. David Card & Andrew Johnston & Pauline Leung & Alexandre Mas & Zhuan Pei, 2015. "The Effect of Unemployment Benefits on the Duration of Unemployment Insurance Receipt: New Evidence from a Regression Kink Design in Missouri, 2003-2013," American Economic Review, American Economic Association, vol. 105(5), pages 126-130, May.
    5. Petri Bockerman & Ohto Kanninen & Ilpo Suoniemi, 2014. "A Kink that Makes you Sick: the Effect of Sick Pay on Absence in a Social Insurance System," Discussion Papers 97, Aboa Centre for Economics.
    6. David Card & David S. Lee & Zhuan Pei & Andrea Weber, 2017. "Regression Kink Design: Theory and Practice," Advances in Econometrics, in: Matias D. Cattaneo & Juan Carlos Escanciano (ed.), Regression Discontinuity Designs, volume 38, pages 341-382, Emerald Publishing Ltd.
    7. Camille Landais, 2015. "Assessing the Welfare Effects of Unemployment Benefits Using the Regression Kink Design," American Economic Journal: Economic Policy, American Economic Association, vol. 7(4), pages 243-278, November.
    8. George B. Bulman & Caroline M. Hoxby, 2015. "The Returns to the Federal Tax Credits for Higher Education," Tax Policy and the Economy, University of Chicago Press, vol. 29(1), pages 13-88.
    9. Alexander Gelber & Timothy J. Moore & Alexander Strand, 2017. "The Effect of Disability Insurance Payments on Beneficiaries' Earnings," American Economic Journal: Economic Policy, American Economic Association, vol. 9(3), pages 229-261, August.
    10. Therese C. Scharlemann & Stephen H. Shore, 2015. "The Effect of Negative Equity on Mortgage Default: Evidence from HAMP PRA," Working Papers 15-06, Office of Financial Research, US Department of the Treasury.
    11. D. G. C. Britto, 2016. "Unemployment Insurance and the Duration of Employment: Evidence from a Regression Kink Design," Working Papers wp1058, Dipartimento Scienze Economiche, Universita' di Bologna.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:hrv:faseco:34222894 is not listed on IDEAS
    2. Peter Ganong & Simon Jäger, 2018. "A Permutation Test for the Regression Kink Design," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 494-504, April.
    3. Bana, Sarah & Bedard, Kelly & Rossin-Slater, Maya, 2018. "The Impacts of Paid Family Leave Benefits: Regression Kink Evidence from California Administrative Data," IZA Discussion Papers 11381, Institute of Labor Economics (IZA).
    4. Sarah H. Bana & Kelly Bedard & Maya Rossin‐Slater, 2020. "The Impacts of Paid Family Leave Benefits: Regression Kink Evidence from California Administrative Data," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(4), pages 888-929, September.
    5. Adam C. Sales & Ben B. Hansen, 2020. "Limitless Regression Discontinuity," Journal of Educational and Behavioral Statistics, , vol. 45(2), pages 143-174, April.
    6. Eduardo Fé & Bruce Hollingsworth, 2016. "Short- and long-run estimates of the local effects of retirement on health," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(4), pages 1051-1067, October.
    7. Prakash, Nishith & Rockmore, Marc & Uppal, Yogesh, 2019. "Do criminally accused politicians affect economic outcomes? Evidence from India," Journal of Development Economics, Elsevier, vol. 141(C).
    8. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    9. Hansen, Benjamin & Nguyen, Tuan & Waddell, Glen R., 2017. "Benefit Generosity and Injury Duration: Quasi-Experimental Evidence from Regression Kinks," IZA Discussion Papers 10621, Institute of Labor Economics (IZA).
    10. Michihito Ando, 2017. "How much should we trust regression-kink-design estimates?," Empirical Economics, Springer, vol. 53(3), pages 1287-1322, November.
    11. Huber, Martin, 2019. "An introduction to flexible methods for policy evaluation," FSES Working Papers 504, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    12. Bartalotti Otávio, 2019. "Regression Discontinuity and Heteroskedasticity Robust Standard Errors: Evidence from a Fixed-Bandwidth Approximation," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-26, January.
    13. Marco Alberto De Benedetto & Maria De Paola, 2017. "Candidates’ Education and Turnout: Evidence from Italian Municipal Elections," German Economic Review, Verein für Socialpolitik, vol. 18(1), pages 22-50, February.
    14. van der Klaauw, Bas, 2014. "From micro data to causality: Forty years of empirical labor economics," Labour Economics, Elsevier, vol. 30(C), pages 88-97.
    15. Quoc-Anh Do & Yen-Teik Lee & Bang Dang Nguyen, 2016. "Directors as Connectors: The Impact of the External Networks of Directors on Firms," Sciences Po publications 52, Sciences Po.
    16. Qian Tang & Andrew B. Whinston, 2020. "Do Reputational Sanctions Deter Negligence in Information Security Management? A Field Quasi‐Experiment," Production and Operations Management, Production and Operations Management Society, vol. 29(2), pages 410-427, February.
    17. Baum-Snow, Nathaniel & Ferreira, Fernando, 2015. "Causal Inference in Urban and Regional Economics," Handbook of Regional and Urban Economics, in: Gilles Duranton & J. V. Henderson & William C. Strange (ed.), Handbook of Regional and Urban Economics, edition 1, volume 5, chapter 0, pages 3-68, Elsevier.
    18. Yoichi Arai & Hidehiko Ichimura, 2018. "Simultaneous selection of optimal bandwidths for the sharp regression discontinuity estimator," Quantitative Economics, Econometric Society, vol. 9(1), pages 441-482, March.
    19. Murphy, Richard & Wyness, Gill, 2015. "Testing means-tested aid," LSE Research Online Documents on Economics 66060, London School of Economics and Political Science, LSE Library.
    20. Ivan A Canay & Vishal Kamat, 2018. "Approximate Permutation Tests and Induced Order Statistics in the Regression Discontinuity Design," Review of Economic Studies, Oxford University Press, vol. 85(3), pages 1577-1608.
    21. Thushyanthan Baskaran & Sonia Bhalotra & Brian Min & Yogesh Uppal, 2018. "Women legislators and economic performance," WIDER Working Paper Series wp-2018-47, World Institute for Development Economic Research (UNU-WIDER).

    More about this item

    Keywords

    randomization inference; placebo test; cubic splines;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C13 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Estimation: General
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C31 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions; Social Interaction Models

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp8282. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.