IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2303.11721.html
   My bibliography  Save this paper

Using Forests in Multivariate Regression Discontinuity Designs

Author

Listed:
  • Yiqi Liu
  • Yuan Qi

Abstract

We discuss estimation and inference of conditional treatment effects in regression discontinuity designs with multiple scores. Aside from the commonly used local linear regression approach and a minimax-optimal estimator recently proposed by Imbens and Wager (2019), we consider two estimators based on random forests -- honest regression forests and local linear forests -- whose construction resembles that of standard local regressions, with theoretical validity following from results in Wager and Athey (2018) and Friedberg et al. (2020). We design a systematic Monte Carlo study with data generating processes built both from functional forms that we specify and from Wasserstein Generative Adversarial Networks that can closely mimic the observed data. We find that no single estimator dominates across all simulations: (i) local linear regressions perform well in univariate settings, but can undercover when multivariate scores are transformed into a univariate score -- which is commonly done in practice -- possibly due to the "zero-density" issue of the collapsed univariate score at the transformed cutoff; (ii) good performance of the minimax-optimal estimator depends on accurate estimation of a nuisance parameter and its current implementation only accepts up to two scores; (iii) forest-based estimators are not designed for estimation at boundary points and can suffer from bias in finite sample, but their flexibility in modeling multivariate scores opens the door to a wide range of empirical applications in multivariate regression discontinuity designs.

Suggested Citation

  • Yiqi Liu & Yuan Qi, 2023. "Using Forests in Multivariate Regression Discontinuity Designs," Papers 2303.11721, arXiv.org, revised Jul 2024.
  • Handle: RePEc:arx:papers:2303.11721
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2303.11721
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Erich Battistin & Agar Brugiavini & Enrico Rettore & Guglielmo Weber, 2009. "The Retirement Consumption Puzzle: Evidence from a Regression Discontinuity Approach," American Economic Review, American Economic Association, vol. 99(5), pages 2209-2226, December.
    2. Timothy B. Armstrong & Michal Kolesár, 2018. "Optimal Inference in a Class of Regression Models," Econometrica, Econometric Society, vol. 86(2), pages 655-683, March.
    3. Sandra E. Black, 1999. "Do Better Schools Matter? Parental Valuation of Elementary Education," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(2), pages 577-599.
    4. Sebastian Calonico & Matias D. Cattaneo & Rocio Titiunik, 2014. "Robust Nonparametric Confidence Intervals for Regression‐Discontinuity Designs," Econometrica, Econometric Society, vol. 82, pages 2295-2326, November.
    5. Brian A. Jacob & Lars Lefgren, 2004. "Remedial Education and Student Achievement: A Regression-Discontinuity Analysis," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 226-244, February.
    6. Stefan Wager & Susan Athey, 2018. "Estimation and Inference of Heterogeneous Treatment Effects using Random Forests," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1228-1242, July.
    7. Sebastian Calonico & Matias D. Cattaneo & Max H. Farrell & Rocío Titiunik, 2019. "Regression Discontinuity Designs Using Covariates," The Review of Economics and Statistics, MIT Press, vol. 101(3), pages 442-451, July.
    8. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    9. Keele, Luke J. & Titiunik, Rocío, 2015. "Geographic Boundaries as Regression Discontinuities," Political Analysis, Cambridge University Press, vol. 23(1), pages 127-155, January.
    10. Johannes F. Schmieder & Till von Wachter & Stefan Bender, 2012. "The Effects of Extended Unemployment Insurance Over the Business Cycle: Evidence from Regression Discontinuity Estimates Over 20 Years," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 127(2), pages 701-752.
    11. Papay, John P. & Willett, John B. & Murnane, Richard J., 2011. "Extending the regression-discontinuity approach to multiple assignment variables," Journal of Econometrics, Elsevier, vol. 161(2), pages 203-207, April.
    12. Hahn, Jinyong & Todd, Petra & Van der Klaauw, Wilbert, 2001. "Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design," Econometrica, Econometric Society, vol. 69(1), pages 201-209, January.
    13. Guido Imbens & Karthik Kalyanaraman, 2012. "Optimal Bandwidth Choice for the Regression Discontinuity Estimator," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 79(3), pages 933-959.
    14. Damon Clark & Paco Martorell, 2014. "The Signaling Value of a High School Diploma," Journal of Political Economy, University of Chicago Press, vol. 122(2), pages 282-318.
    15. Bradley Efron, 2014. "Estimation and Accuracy After Model Selection," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(507), pages 991-1007, September.
    16. David S. Lee & Thomas Lemieux, 2010. "Regression Discontinuity Designs in Economics," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 281-355, June.
    17. Johannes F. Schmieder† & Till von Wachter & Stefan Bender, 2011. "The Effects Of Extended Unemployment Insurance Over The Business Cycle: Evidence From Regression Discontinuity Estimates Over Twenty Years," Boston University - Department of Economics - Working Papers Series WP2011-063, Boston University - Department of Economics.
    18. Sexton, Joseph & Laake, Petter, 2009. "Standard errors for bagged and random forest estimators," Computational Statistics & Data Analysis, Elsevier, vol. 53(3), pages 801-811, January.
    19. Lee, David S., 2008. "Randomized experiments from non-random selection in U.S. House elections," Journal of Econometrics, Elsevier, vol. 142(2), pages 675-697, February.
    20. Guido Imbens & Stefan Wager, 2019. "Optimized Regression Discontinuity Designs," The Review of Economics and Statistics, MIT Press, vol. 101(2), pages 264-278, May.
    21. Choi, Jin-young & Lee, Myoung-jae, 2018. "Regression Discontinuity with Multiple Running Variables Allowing Partial Effects," Political Analysis, Cambridge University Press, vol. 26(3), pages 258-274, July.
    22. McCrary, Justin, 2008. "Manipulation of the running variable in the regression discontinuity design: A density test," Journal of Econometrics, Elsevier, vol. 142(2), pages 698-714, February.
    23. Matsudaira, Jordan D., 2008. "Mandatory summer school and student achievement," Journal of Econometrics, Elsevier, vol. 142(2), pages 829-850, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Blaise Melly & Rafael Lalive, 2020. "Estimation, Inference, and Interpretation in the Regression Discontinuity Design," Diskussionsschriften dp2016, Universitaet Bern, Departement Volkswirtschaft.
    2. Jin-young Choi & Myoung-jae Lee, 2017. "Regression discontinuity: review with extensions," Statistical Papers, Springer, vol. 58(4), pages 1217-1246, December.
    3. Mauricio Villamizar‐Villegas & Freddy A. Pinzon‐Puerto & Maria Alejandra Ruiz‐Sanchez, 2022. "A comprehensive history of regression discontinuity designs: An empirical survey of the last 60 years," Journal of Economic Surveys, Wiley Blackwell, vol. 36(4), pages 1130-1178, September.
    4. Bertanha, Marinho & Moreira, Marcelo J., 2020. "Impossible inference in econometrics: Theory and applications," Journal of Econometrics, Elsevier, vol. 218(2), pages 247-270.
    5. Matias D. Cattaneo & Rocío Titiunik, 2022. "Regression Discontinuity Designs," Annual Review of Economics, Annual Reviews, vol. 14(1), pages 821-851, August.
    6. Guido Imbens & Stefan Wager, 2019. "Optimized Regression Discontinuity Designs," The Review of Economics and Statistics, MIT Press, vol. 101(2), pages 264-278, May.
    7. Huber, Martin, 2019. "An introduction to flexible methods for policy evaluation," FSES Working Papers 504, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    8. Christina Korting & Carl Lieberman & Jordan Matsudaira & Zhuan Pei & Yi Shen, 2023. "Visual Inference and Graphical Representation in Regression Discontinuity Designs," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 138(3), pages 1977-2019.
    9. Yoichi Arai & Yu‐Chin Hsu & Toru Kitagawa & Ismael Mourifié & Yuanyuan Wan, 2022. "Testing identifying assumptions in fuzzy regression discontinuity designs," Quantitative Economics, Econometric Society, vol. 13(1), pages 1-28, January.
    10. Yang He & Otávio Bartalotti, 2020. "Wild bootstrap for fuzzy regression discontinuity designs: obtaining robust bias-corrected confidence intervals," The Econometrics Journal, Royal Economic Society, vol. 23(2), pages 211-231.
    11. Bartalotti, Otávio C. & Calhoun, Gray & He, Yang, 2016. "Bootstrap Confidence Intervals for Sharp Regression Discontinuity Designs with the Uniform Kernel," Staff General Research Papers Archive 3394, Iowa State University, Department of Economics.
    12. Marinho Bertanha & Guido W. Imbens, 2020. "External Validity in Fuzzy Regression Discontinuity Designs," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(3), pages 593-612, July.
    13. Masayuki Sawada & Takuya Ishihara & Daisuke Kurisu & Yasumasa Matsuda, 2024. "Local-Polynomial Estimation for Multivariate Regression Discontinuity Designs," Papers 2402.08941, arXiv.org.
    14. Holbein, John B. & Ladd, Helen F., 2017. "Accountability pressure: Regression discontinuity estimates of how No Child Left Behind influenced student behavior," Economics of Education Review, Elsevier, vol. 58(C), pages 55-67.
    15. Giuseppe Francesco Gori & Patrizia Lattarulo & Marco Mariani, 2021. "The Expediting Effect of Monitoring on Infrastructural Works. A Regression-Discontinuity Approach with Multiple Assignment Variables," Papers 2102.09625, arXiv.org.
    16. Dong, Yingying, 2010. "Jumpy or Kinky? Regression Discontinuity without the Discontinuity," MPRA Paper 25461, University Library of Munich, Germany.
    17. Atı̇la Abdulkadı̇roğlu & Joshua D. Angrist & Yusuke Narita & Parag Pathak, 2022. "Breaking Ties: Regression Discontinuity Design Meets Market Design," Econometrica, Econometric Society, vol. 90(1), pages 117-151, January.
    18. Crespo Cristian, 2020. "Beyond Manipulation: Administrative Sorting in Regression Discontinuity Designs," Journal of Causal Inference, De Gruyter, vol. 8(1), pages 164-181, January.
    19. Hsu, Yu-Chin & Shiu, Ji-Liang & Wan, Yuanyuan, 2024. "Testing identification conditions of LATE in fuzzy regression discontinuity designs," Journal of Econometrics, Elsevier, vol. 241(1).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2303.11721. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.