IDEAS home Printed from https://ideas.repec.org/p/mpr/mprres/a4f1d03eb7bf427a8983d4736a4f8f10.html
   My bibliography  Save this paper

Statistical Power for Regression Discontinuity Designs in Education: Empirical Estimates of Design Effects Relative to Randomized Controlled Trials

Author

Listed:
  • John Deke
  • Lisa Dragoset

Abstract

Using data from four previously published education studies, this working paper finds that a study using a regression discontinuity design needs between 9 and 17 times as many schools or students as a randomized controlled trial to produce an impact with the same level of statistical precision.

Suggested Citation

  • John Deke & Lisa Dragoset, "undated". "Statistical Power for Regression Discontinuity Designs in Education: Empirical Estimates of Design Effects Relative to Randomized Controlled Trials," Mathematica Policy Research Reports a4f1d03eb7bf427a8983d4736, Mathematica Policy Research.
    • Handle: RePEc:mpr:mprres:a4f1d03eb7bf427a8983d4736a4f8f10
    as

    Download full text from publisher

    File URL: https://www.mathematica.org/-/media/publications/pdfs/education/statpowerregdis_wp.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. repec:mpr:mprres:6372 is not listed on IDEAS
    2. repec:mpr:mprres:6174 is not listed on IDEAS
    3. repec:mpr:mprres:6192 is not listed on IDEAS
    4. Peter Z. Schochet, "undated". "Technical Methods Report: Statistical Power for Regression Discontinuity Designs in Education Evaluations," Mathematica Policy Research Reports 61fb6c057561451a8a6074508, Mathematica Policy Research.
    5. repec:mpr:mprres:7723 is not listed on IDEAS
    6. Jill Constantine & Daniel Player & Tim Silva & Kristin Hallgren & Mary Grider & John Deke, 2009. "An Evaluation of Teachers Trained Through Different Routes to Certification," Mathematica Policy Research Reports 1a9e2910a01843c2babbc8c64, Mathematica Policy Research.
    7. Cook, Thomas D., 2008. ""Waiting for Life to Arrive": A history of the regression-discontinuity design in Psychology, Statistics and Economics," Journal of Econometrics, Elsevier, vol. 142(2), pages 636-654, February.
    8. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    9. repec:mpr:mprres:6181 is not listed on IDEAS
    10. Larissa Campuzano & Mark Dynarski & Roberto Agodini & Kristina Rall, "undated". "Effectiveness of Reading and Mathematics Software Products: Findings from Two Student Cohorts," Mathematica Policy Research Reports c6255408bda44f82910991c48, Mathematica Policy Research.
    11. repec:mpr:mprres:6814 is not listed on IDEAS
    12. Lee, David S. & Card, David, 2008. "Regression discontinuity inference with specification error," Journal of Econometrics, Elsevier, vol. 142(2), pages 655-674, February.
    13. repec:mpr:mprres:6585 is not listed on IDEAS
    14. Duncan, Keith, 2010. "A history of corporate governance around the world: Family business groups to professional managers, RK Morck (ed.) (2007), A National Bureau of Economic Research Conference report, The University of ," Journal of Management & Organization, Cambridge University Press, vol. 15(3), pages 395-397, March.
    15. Jill Constantine, 2009. "Evaluation of Teachers Trained Through Different Routes to Certification," Mathematica Policy Research Reports 6e759a791c77408a816add035, Mathematica Policy Research.
    16. Roberto Agodini & Barbara Harris & Sally Atkins-Burnett & Sheila Heaviside & Timothy Novak & Robert Murphy, "undated". "Achievement Effects of Four Early Elementary School Math Curricula: Findings from First Graders in 39 Schools," Mathematica Policy Research Reports 467194cd2d6f4cbaba7e23745, Mathematica Policy Research.
    17. repec:mpr:mprres:6175 is not listed on IDEAS
    18. repec:mpr:mprres:7338 is not listed on IDEAS
    19. Roberto Agodini, 2009. "Achievement Effects of Four Early Elementary School Math Curricula: Findings from First Graders in 39 Schools (Conference Paper)," Mathematica Policy Research Reports a08d661306a843fa89aff4c30, Mathematica Policy Research.
    20. repec:mpr:mprres:7337 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. John Deke & Thomas Wei & Tim Kautz, "undated". "Asymdystopia: The Threat of Small Biases in Evaluations of Education Interventions that Need to be Powered to Detect Small Impacts," Mathematica Policy Research Reports f0ff8f86e3c34dc8baaf22b56, Mathematica Policy Research.
    2. Kim, Hyuncheol Bryant & Lee, Suejin & Lim, Wilfredo, 2017. "Knowing Is Not Half the Battle: Impacts of the National Health Screening Program in Korea," IZA Discussion Papers 10650, Institute of Labor Economics (IZA).
    3. Sarah Dolfin & Nan Maxwell & Ankita Patnaik, "undated". "WHD Compliance Strategies: Directions for Future Research," Mathematica Policy Research Reports b7a5ca876e0b448f9b9c0850e, Mathematica Policy Research.
    4. de Hoyos, Rafael & Garcia-Moreno, Vicente A. & Patrinos, Harry Anthony, 2017. "The impact of an accountability intervention with diagnostic feedback: Evidence from Mexico," Economics of Education Review, Elsevier, vol. 58(C), pages 123-140.
    5. Berg, Heléne, 2020. "On the returns to holding political office (Is it worth it?)," Journal of Economic Behavior & Organization, Elsevier, vol. 178(C), pages 840-865.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:mpr:mprres:7486 is not listed on IDEAS
    2. Lisa Dragoset & Jaime Thomas & Mariesa Herrmann & John Deke & Susanne James-Burdumy & Cheryl Graczewski & Andrea Boyle & Rachel Upton & Courtney Tanenbaum & Jessica Giffin, "undated". "School Improvement Grants: Implementation and Effectiveness (Final Report)," Mathematica Policy Research Reports 76bce3f4bb0944f29a481fae0, Mathematica Policy Research.
    3. repec:mpr:mprres:6569 is not listed on IDEAS
    4. Lizabeth M. Malone & Charlotte Cabili & Jamila Henderson & Andrea Mraz Esposito & Kathleen Coolahan & Juliette Henke & Subuhi Asheer & Meghan O'Toole & Sally Atkins-Burnett & Kimberly Boller, "undated". "Compendium of Student, Teacher, and Classroom Measures Used in NCEE Evaluations of Educational Interventions, Volume II: Technical Details, Measure Profiles, and Glossary," Mathematica Policy Research Reports ceaa887498d84907b20e88da1, Mathematica Policy Research.
    5. John Deke & Thomas Wei & Tim Kautz, "undated". "Asymdystopia: The Threat of Small Biases in Evaluations of Education Interventions that Need to be Powered to Detect Small Impacts," Mathematica Policy Research Reports f0ff8f86e3c34dc8baaf22b56, Mathematica Policy Research.
    6. David S. Lee & Thomas Lemieux, 2010. "Regression Discontinuity Designs in Economics," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 281-355, June.
    7. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    8. Mauricio Villamizar‐Villegas & Freddy A. Pinzon‐Puerto & Maria Alejandra Ruiz‐Sanchez, 2022. "A comprehensive history of regression discontinuity designs: An empirical survey of the last 60 years," Journal of Economic Surveys, Wiley Blackwell, vol. 36(4), pages 1130-1178, September.
    9. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    10. Chad D. Meyerhoefer & Muzhe Yang, 2011. "The Relationship between Food Assistance and Health: A Review of the Literature and Empirical Strategies for Identifying Program Effects," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 33(3), pages 304-344.
    11. Olivier Bargain & Delphine Boutin, 2021. "Minimum Age Regulation and Child Labor: New Evidence from Brazil," The World Bank Economic Review, World Bank, vol. 35(1), pages 234-260.
    12. Mazzutti, Caio Cícero Toledo Piza da Costa, 2016. "Three essays on the causal impacts of child labour laws in Brazil," Economics PhD Theses 0616, Department of Economics, University of Sussex Business School.
    13. Jin-young Choi & Myoung-jae Lee, 2017. "Regression discontinuity: review with extensions," Statistical Papers, Springer, vol. 58(4), pages 1217-1246, December.
    14. Cunguara, Benedito & Darnhofer, Ika, 2011. "Assessing the impact of improved agricultural technologies on household income in rural Mozambique," Food Policy, Elsevier, vol. 36(3), pages 378-390, June.
    15. repec:mpr:mprres:7461 is not listed on IDEAS
    16. Mäkinen, Taneli & Li, Fan & Mercatanti, Andrea & Silvestrini, Andrea, 2022. "Causal analysis of central bank holdings of corporate bonds under interference," Economic Modelling, Elsevier, vol. 113(C).
    17. Martin Pereyra & Diego Aboal & Flavia Rovira, 2021. "How effective are training and mentorship programs for entrepreneurs at promoting entrepreneurial activity? An impact evaluation," SN Business & Economics, Springer, vol. 1(7), pages 1-21, July.
    18. Matias D. Cattaneo & Rocío Titiunik, 2022. "Regression Discontinuity Designs," Annual Review of Economics, Annual Reviews, vol. 14(1), pages 821-851, August.
    19. de Lazzer, Jakob, 2016. "Non-monotonic Selection Issues in Electoral Regression Discontinuity Designs," VfS Annual Conference 2016 (Augsburg): Demographic Change 145845, Verein für Socialpolitik / German Economic Association.
    20. Philip M. Gleason & Alexandra M. Resch & Jillian A. Berk, "undated". "Replicating Experimental Impact Estimates Using a Regression Discontinuity Approach," Mathematica Policy Research Reports c7ecf8b35c8d402491d09b2d0, Mathematica Policy Research.
    21. Persson, Petra & Qiu, Xinyao & Rossin-Slater, Maya, 2021. "Family Spillover Effects of Marginal Diagnoses: The Case of ADHD," IZA Discussion Papers 14020, Institute of Labor Economics (IZA).
    22. KAMKOUM, Arnaud Cedric, 2023. "The Federal Reserve’s Response to the Global Financial Crisis and its Effects: An Interrupted Time-Series Analysis of the Impact of its Quantitative Easing Programs," Thesis Commons d7pvg, Center for Open Science.
    23. Jason M. Lindo & Nicholas J. Sanders & Philip Oreopoulos, 2010. "Ability, Gender, and Performance Standards: Evidence from Academic Probation," American Economic Journal: Applied Economics, American Economic Association, vol. 2(2), pages 95-117, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:mpr:mprres:a4f1d03eb7bf427a8983d4736a4f8f10. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joanne Pfleiderer or Cindy George (email available below). General contact details of provider: https://edirc.repec.org/data/mathius.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.