IDEAS home Printed from https://ideas.repec.org/p/zbw/i4rdps/17.html
   My bibliography  Save this paper

A Reply to Comment by Bonander et al. (2023)

Author

Listed:
  • Altindag, Onur
  • Erten, Bilge
  • Keskin, Pinar

Abstract

In Altindag et al. (2022), we estimate the effects of an age-specific lockdown policy on mobility and mental health outcomes among adults aged 65 and older in Turkey using a regression discontinuity design. Bonander et al. (2023) successfully replicate all our main findings. They argue that the estimates for mobility outcomes are all robust to alternative sensitivity checks while some of the estimates for mental health-which were statistically significant around the 5-9 percent level-lose significance at the conventional level of 10 percent in the more conservative specifications. In this reply, we provide approximately 7,000 additional estimates that comprise a near universe of RD estimates for all our outcomes, each possible monthly bandwidth, and each possible combination of covariate adjustment, kernel selection, estimation methodology, standard error adjustment, and kernel weighting selection. This comprehensive analysis shows that our original results are robust to these choices. We show that Bonander et al. (2023) rely on a selection of very narrow bandwidths that produce highly sensitive and uninformative estimates due to overfitting. We also show that Bonander et al. (2023) report imprecise estimates, which are outliers in the distribution of all estimates that can be reported. We conclude that broader statistical tests are more informative for robustness checks.

Suggested Citation

  • Altindag, Onur & Erten, Bilge & Keskin, Pinar, 2023. "A Reply to Comment by Bonander et al. (2023)," I4R Discussion Paper Series 17, The Institute for Replication (I4R).
  • Handle: RePEc:zbw:i4rdps:17
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/268497/1/I4R-DP017.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. David Card & David S. Lee & Zhuan Pei & Andrea Weber, 2015. "Inference on Causal Effects in a Generalized Regression Kink Design," Econometrica, Econometric Society, vol. 83, pages 2453-2483, November.
    2. Onur Altindag & Bilge Erten & Pinar Keskin, 2022. "Mental Health Costs of Lockdowns: Evidence from Age-Specific Curfews in Turkey," American Economic Journal: Applied Economics, American Economic Association, vol. 14(2), pages 320-343, April.
    3. Jens Ludwig & Douglas L. Miller, 2007. "Does Head Start Improve Children's Life Chances? Evidence from a Regression Discontinuity Design," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(1), pages 159-208.
    4. Guido Imbens & Karthik Kalyanaraman, 2012. "Optimal Bandwidth Choice for the Regression Discontinuity Estimator," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 79(3), pages 933-959.
    5. Sebastian Calonico & Matias D. Cattaneo & Max H. Farrell & Roc ́ıo Titiunik, 2017. "rdrobust: Software for regression-discontinuity designs," Stata Journal, StataCorp LP, vol. 17(2), pages 372-404, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sebastian Calonico & Matias D. Cattaneo & Max H. Farrell, 2018. "Optimal Bandwidth Choice for Robust Bias Corrected Inference in Regression Discontinuity Designs," Papers 1809.00236, arXiv.org, revised Jan 2020.
    2. Myung Hwan Seo & Yoichi Arai & Taisuke Otsu, 2021. "Regression Discontinuity Design with Potentially Many Covariates," Working Paper Series no142, Institute of Economic Research, Seoul National University.
    3. Sebastian Calonico & Matias D. Cattaneo & Max H. Farrell & Rocío Titiunik, 2019. "Regression Discontinuity Designs Using Covariates," The Review of Economics and Statistics, MIT Press, vol. 101(3), pages 442-451, July.
    4. Hansen, Benjamin & Miller, Keaton & Weber, Caroline, 2020. "Federalism, partial prohibition, and cross-border sales: Evidence from recreational marijuana," Journal of Public Economics, Elsevier, vol. 187(C).
    5. Liepmann, Hannah & Pignatti, Clemente, 2024. "Welfare effects of unemployment benefits when informality is high," Journal of Public Economics, Elsevier, vol. 229(C).
    6. Peter Ganong & Simon Jäger, 2018. "A Permutation Test for the Regression Kink Design," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 494-504, April.
    7. Timothy B Armstrong & Michal Kolesár, 2018. "A Simple Adjustment for Bandwidth Snooping," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 85(2), pages 732-765.
    8. Blaise Melly & Rafael Lalive, 2020. "Estimation, Inference, and Interpretation in the Regression Discontinuity Design," Diskussionsschriften dp2016, Universitaet Bern, Departement Volkswirtschaft.
    9. He, Yang & Bartalotti, Otávio, 2019. "Wild Bootstrap for Fuzzy Regression Discontinuity Designs: Obtaining Robust Bias-Corrected Confidence Intervals," IZA Discussion Papers 12801, Institute of Labor Economics (IZA).
    10. Bartalotti Otávio, 2019. "Regression Discontinuity and Heteroskedasticity Robust Standard Errors: Evidence from a Fixed-Bandwidth Approximation," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-26, January.
    11. Majbouri, Mahdi, 2017. "Sir! I'd Rather Go to School, Sir!," IZA Discussion Papers 10787, Institute of Labor Economics (IZA).
    12. Jin-young Choi & Myoung-jae Lee, 2017. "Regression discontinuity: review with extensions," Statistical Papers, Springer, vol. 58(4), pages 1217-1246, December.
    13. Zhuan Pei & David S. Lee & David Card & Andrea Weber, 2022. "Local Polynomial Order in Regression Discontinuity Designs," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 40(3), pages 1259-1267, June.
    14. repec:oup:emjrnl:v:23:y:2020:i:2:p:211-231. is not listed on IDEAS
    15. Matias D. Cattaneo & Rocío Titiunik, 2022. "Regression Discontinuity Designs," Annual Review of Economics, Annual Reviews, vol. 14(1), pages 821-851, August.
    16. Deng, Taotao & Hu, Yukun & Ma, Mulan, 2019. "Regional policy and tourism: A quasi-natural experiment," Annals of Tourism Research, Elsevier, vol. 74(C), pages 1-16.
    17. Ximing Wu, 2021. "Hierarchical Gaussian Process Models for Regression Discontinuity/Kink under Sharp and Fuzzy Designs," Papers 2110.00921, arXiv.org, revised Feb 2022.
    18. YANAGI, Takahide & 柳, 貴英, 2015. "Regression Discontinuity Designs with Nonclassical Measurement Error," Discussion Papers 2015-09, Graduate School of Economics, Hitotsubashi University.
    19. Dong, Yingying, 2010. "Jumpy or Kinky? Regression Discontinuity without the Discontinuity," MPRA Paper 25461, University Library of Munich, Germany.
    20. Yoichi Arai & Hidehiko Ichimura, 2018. "Simultaneous selection of optimal bandwidths for the sharp regression discontinuity estimator," Quantitative Economics, Econometric Society, vol. 9(1), pages 441-482, March.
    21. Chen, Yi & Zhao, Yi, 2022. "The timing of first marriage and subsequent life outcomes: Evidence from a natural experiment," Journal of Comparative Economics, Elsevier, vol. 50(3), pages 713-731.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:i4rdps:17. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://www.i4replication.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.