IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2212.04043.html
   My bibliography  Save this paper

Optimal Model Selection in RDD and Related Settings Using Placebo Zones

Author

Listed:
  • Nathan Kettlewell
  • Peter Siminski

Abstract

We propose a new model-selection algorithm for Regression Discontinuity Design, Regression Kink Design, and related IV estimators. Candidate models are assessed within a 'placebo zone' of the running variable, where the true effects are known to be zero. The approach yields an optimal combination of bandwidth, polynomial, and any other choice parameters. It can also inform choices between classes of models (e.g. RDD versus cohort-IV) and any other choices, such as covariates, kernel, or other weights. We outline sufficient conditions under which the approach is asymptotically optimal. The approach also performs favorably under more general conditions in a series of Monte Carlo simulations. We demonstrate the approach in an evaluation of changes to Minimum Supervised Driving Hours in the Australian state of New South Wales. We also re-evaluate evidence on the effects of Head Start and Minimum Legal Drinking Age. Our Stata commands implement the procedure and compare its performance to other approaches.

Suggested Citation

  • Nathan Kettlewell & Peter Siminski, 2022. "Optimal Model Selection in RDD and Related Settings Using Placebo Zones," Papers 2212.04043, arXiv.org.
  • Handle: RePEc:arx:papers:2212.04043
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2212.04043
    File Function: Latest version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Zhuan Pei & David S. Lee & David Card & Andrea Weber, 2022. "Local Polynomial Order in Regression Discontinuity Designs," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 40(3), pages 1259-1267, June.
    2. Jason M. Lindo & Peter Siminski & Oleg Yerokhin, 2016. "Breaking The Link Between Legal Access To Alcohol And Motor Vehicle Accidents: Evidence From New South Wales," Health Economics, John Wiley & Sons, Ltd., vol. 25(7), pages 908-928, July.
    3. David Card & David S. Lee & Zhuan Pei & Andrea Weber, 2015. "Inference on Causal Effects in a Generalized Regression Kink Design," Econometrica, Econometric Society, vol. 83, pages 2453-2483, November.
    4. Sebastian Calonico & Matias D. Cattaneo & Max H. Farrell & Rocío Titiunik, 2019. "Regression Discontinuity Designs Using Covariates," The Review of Economics and Statistics, MIT Press, vol. 101(3), pages 442-451, July.
    5. Imbens, Guido & van der Klaauw, Wilbert, 1995. "Evaluating the Cost of Conscription in The Netherlands," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(2), pages 207-215, April.
    6. Cornelissen, Thomas & Dustmann, Christian & Raute, Anna & Schönberg, Uta, 2016. "From LATE to MTE: Alternative methods for the evaluation of policy interventions," Labour Economics, Elsevier, vol. 41(C), pages 47-60.
    7. John Bound & Sarah Turner, 2002. "Going to War and Going to College: Did World War II and the G.I. Bill Increase Educational Attainment for Returning Veterans?," Journal of Labor Economics, University of Chicago Press, vol. 20(4), pages 784-815, October.
    8. David S. Lee & Thomas Lemieux, 2010. "Regression Discontinuity Designs in Economics," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 281-355, June.
    9. Ari Hyytinen & Jaakko Meriläinen & Tuukka Saarimaa & Otto Toivanen & Janne Tukiainen, 2018. "When does regression discontinuity design work? Evidence from random election outcomes," Quantitative Economics, Econometric Society, vol. 9(2), pages 1019-1051, July.
    10. Matias D. Cattaneo & Michael Jansson & Xinwei Ma, 2020. "Simple Local Polynomial Density Estimators," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 115(531), pages 1449-1455, July.
    11. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    12. David Card & David S. Lee & Zhuan Pei & Andrea Weber, 2017. "Regression Kink Design: Theory and Practice," Advances in Econometrics, in: Regression Discontinuity Designs, volume 38, pages 341-382, Emerald Group Publishing Limited.
    13. Andrew Gelman & Guido Imbens, 2019. "Why High-Order Polynomials Should Not Be Used in Regression Discontinuity Designs," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 37(3), pages 447-456, July.
    14. Sebastian Calonico & Matias D. Cattaneo & Rocio Titiunik, 2014. "Robust Nonparametric Confidence Intervals for Regression‐Discontinuity Designs," Econometrica, Econometric Society, vol. 82, pages 2295-2326, November.
    15. McCrary, Justin, 2008. "Manipulation of the running variable in the regression discontinuity design: A density test," Journal of Econometrics, Elsevier, vol. 142(2), pages 698-714, February.
    16. Duncan D. Chaplin & Thomas D. Cook & Jelena Zurovac & Jared S. Coopersmith & Mariel M. Finucane & Lauren N. Vollmer & Rebecca E. Morris, 2018. "The Internal And External Validity Of The Regression Discontinuity Design: A Meta‐Analysis Of 15 Within‐Study Comparisons," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 37(2), pages 403-429, March.
    17. Cousley, Alex & Siminski, Peter & Ville, Simon, 2017. "The Effects of World War II Military Service: Evidence from Australia," The Journal of Economic History, Cambridge University Press, vol. 77(3), pages 838-865, September.
    18. Peter Ganong & Simon Jäger, 2018. "A Permutation Test for the Regression Kink Design," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 494-504, April.
    19. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, December.
    20. Doug Miller & Jens Ludwig, 2005. "Does Head Start Improve Children?s Life Chances? Evidence from a Regression Discontinuity Design," Working Papers 534, University of California, Davis, Department of Economics.
    21. Jens Ludwig & Douglas L. Miller, 2007. "Does Head Start Improve Children's Life Chances? Evidence from a Regression Discontinuity Design," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(1), pages 159-208.
    22. Burt S. Barnow & Matias D. Cattaneo & Rocío Titiunik & Gonzalo Vazquez‐Bare, 2017. "Comparing Inference Approaches for RD Designs: A Reexamination of the Effect of Head Start on Child Mortality," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 36(3), pages 643-681, June.
    23. Dee, Thomas S. & Grabowski, David C. & Morrisey, Michael A., 2005. "Graduated driver licensing and teen traffic fatalities," Journal of Health Economics, Elsevier, vol. 24(3), pages 571-589, May.
    24. Sebastian Calonico & Matias D. Cattaneo & Rocío Titiunik, 2015. "Optimal Data-Driven Regression Discontinuity Plots," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1753-1769, December.
    25. Hahn, Jinyong & Todd, Petra & Van der Klaauw, Wilbert, 2001. "Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design," Econometrica, Econometric Society, vol. 69(1), pages 201-209, January.
    26. Rebecca A. Maynard & Kenneth A. Couch & Coady Wing & Thomas D. Cook, 2013. "Strengthening The Regression Discontinuity Design Using Additional Design Elements: A Within‐Study Comparison," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 32(4), pages 853-877, September.
    27. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    28. Hall, Peter G. & Racine, Jeffrey S., 2015. "Infinite order cross-validated local polynomial regression," Journal of Econometrics, Elsevier, vol. 185(2), pages 510-525.
    29. Doug Miller & Jens Ludwig, 2005. "Does Head Start Improve Children?s Life Chances? Evidence from a Regression Discontinuity Design," Working Papers 54, University of California, Davis, Department of Economics.
    30. Christopher Carpenter & Carlos Dobkin, 2009. "The Effect of Alcohol Consumption on Mortality: Regression Discontinuity Evidence from the Minimum Drinking Age," American Economic Journal: Applied Economics, American Economic Association, vol. 1(1), pages 164-182, January.
    31. Karaca-Mandic, Pinar & Ridgeway, Greg, 2010. "Behavioral impact of graduated driver licensing on teenage driving risk and exposure," Journal of Health Economics, Elsevier, vol. 29(1), pages 48-61, January.
    32. Matias D. Cattaneo & Nicolas Idrobo & Rocio Titiunik, 2019. "A Practical Introduction to Regression Discontinuity Designs: Foundations," Papers 1911.09511, arXiv.org.
    33. Joshua D. Angrist & Victor Lavy, 1999. "Using Maimonides' Rule to Estimate the Effect of Class Size on Scholastic Achievement," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(2), pages 533-575.
    34. Joshua D. Angrist & Jörn-Steffen Pischke, 2015. "The path from cause to effect: mastering 'metrics," CentrePiece - The magazine for economic performance 442, Centre for Economic Performance, LSE.
    35. Gilpin, Gregory, 2019. "Teen driver licensure provisions, licensing, and vehicular fatalities," Journal of Health Economics, Elsevier, vol. 66(C), pages 54-70.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nathan Kettlewell & Yuting Zhang, 2024. "Age penalties and take‐up of private health insurance," Health Economics, John Wiley & Sons, Ltd., vol. 33(4), pages 636-651, April.
    2. Kettlewell, Nathan & Zhang, Yuting, 2024. "Financial incentives and private health insurance demand on the extensive and intensive margins," Journal of Health Economics, Elsevier, vol. 94(C).
    3. Kettlewell, Nathan & Zhang, Yuting, 2023. "Financial Incentives and Private Health Insurance Demand on the Extensive and Intensive Margins," IZA Discussion Papers 16248, Institute of Labor Economics (IZA).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mauricio Villamizar‐Villegas & Freddy A. Pinzon‐Puerto & Maria Alejandra Ruiz‐Sanchez, 2022. "A comprehensive history of regression discontinuity designs: An empirical survey of the last 60 years," Journal of Economic Surveys, Wiley Blackwell, vol. 36(4), pages 1130-1178, September.
    2. Blaise Melly & Rafael Lalive, 2020. "Estimation, Inference, and Interpretation in the Regression Discontinuity Design," Diskussionsschriften dp2016, Universitaet Bern, Departement Volkswirtschaft.
    3. Matias D. Cattaneo & Rocío Titiunik, 2022. "Regression Discontinuity Designs," Annual Review of Economics, Annual Reviews, vol. 14(1), pages 821-851, August.
    4. Ari Hyytinen & Jaakko Meriläinen & Tuukka Saarimaa & Otto Toivanen & Janne Tukiainen, 2018. "When does regression discontinuity design work? Evidence from random election outcomes," Quantitative Economics, Econometric Society, vol. 9(2), pages 1019-1051, July.
    5. Babii, Andrii & Kumar, Rohit, 2023. "Isotonic regression discontinuity designs," Journal of Econometrics, Elsevier, vol. 234(2), pages 371-393.
    6. Cuong Viet Nguyen & Finn Tarp, 2023. "Cash Transfers and Labor Supply: New Evidence on Impacts and Mechanisms," DERG working paper series 23-18, University of Copenhagen. Department of Economics. Development Economics Research Group (DERG).
    7. Jin-young Choi & Myoung-jae Lee, 2017. "Regression discontinuity: review with extensions," Statistical Papers, Springer, vol. 58(4), pages 1217-1246, December.
    8. 'Agoston Reguly, 2021. "Heterogeneous Treatment Effects in Regression Discontinuity Designs," Papers 2106.11640, arXiv.org, revised Oct 2021.
    9. Marinho Bertanha & Guido W. Imbens, 2020. "External Validity in Fuzzy Regression Discontinuity Designs," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(3), pages 593-612, July.
    10. Burgherr, David, 2022. "Behavioral Responses to a Pension Savings Mandate : Quasi-experimental Evidence from Swiss Tax Data," CAGE Online Working Paper Series 645, Competitive Advantage in the Global Economy (CAGE).
    11. repec:oup:emjrnl:v:23:y:2020:i:2:p:192-210. is not listed on IDEAS
    12. Crespo Cristian, 2020. "Beyond Manipulation: Administrative Sorting in Regression Discontinuity Designs," Journal of Causal Inference, De Gruyter, vol. 8(1), pages 164-181, January.
    13. Likai Chen & Georg Keilbar & Liangjun Su & Weining Wang, 2023. "Tests for Many Treatment Effects in Regression Discontinuity Panel Data Models," Papers 2312.01162, arXiv.org.
    14. Sebastian Calonico & Matias D. Cattaneo & Max H. Farrell, 2018. "Optimal Bandwidth Choice for Robust Bias Corrected Inference in Regression Discontinuity Designs," Papers 1809.00236, arXiv.org, revised Jan 2020.
    15. Yoichi Arai & Yu‐Chin Hsu & Toru Kitagawa & Ismael Mourifié & Yuanyuan Wan, 2022. "Testing identifying assumptions in fuzzy regression discontinuity designs," Quantitative Economics, Econometric Society, vol. 13(1), pages 1-28, January.
    16. Myung Hwan Seo & Yoichi Arai & Taisuke Otsu, 2021. "Regression Discontinuity Design with Potentially Many Covariates," Working Paper Series no142, Institute of Economic Research, Seoul National University.
    17. Yang He & Otávio Bartalotti, 2020. "Wild bootstrap for fuzzy regression discontinuity designs: obtaining robust bias-corrected confidence intervals [Using Maimonides’ rule to estimate the effect of class size on scholastic achievemen," The Econometrics Journal, Royal Economic Society, vol. 23(2), pages 211-231.
    18. Makkar, Karan, 2023. "Defector Politicians and Economic Growth: Evidence from India," European Journal of Political Economy, Elsevier, vol. 79(C).
    19. Philip Gleason & Alexandra Resch & Jillian Berk, 2018. "RD or Not RD: Using Experimental Studies to Assess the Performance of the Regression Discontinuity Approach," Evaluation Review, , vol. 42(1), pages 3-33, February.
    20. Cappelletti, Giuseppe & Ponte Marques, Aurea & Salleo, Carmelo & Martín, Diego Vila, 2020. "How do banking groups react to macroprudential policies? Cross-border spillover effects of higher capital buffers on lending, risk-taking and internal markets," Working Paper Series 2497, European Central Bank.
    21. Travis A. Smith & Pourya Valizadeh, 2024. "Aging out of WIC and child nutrition: Evidence from a regression discontinuity design," American Journal of Agricultural Economics, John Wiley & Sons, vol. 106(2), pages 904-924, March.

    More about this item

    JEL classification:

    • C13 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Estimation: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • I18 - Health, Education, and Welfare - - Health - - - Government Policy; Regulation; Public Health

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2212.04043. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.