IDEAS home Printed from https://ideas.repec.org/p/iae/iaewps/wp2016n37.html
   My bibliography  Save this paper

Economic Research and Education Policy: Project STAR and Class Size Reduction

Author

Listed:
  • Moshe Justman

    (Melbourne Institute of Applied Economic and Social Research, The University of Melbourne; and Department of Economics, Ben Gurion University of the Negev)

Abstract

The use of randomized controlled trials (RCTs) and related randomization strategies to eliminate selection biases in establishing causality is a key element of the “modern experimentalist paradigm” (MEP). Yet, its emphasis on precisely identifying causal factors often limits its capacity to provide an evidence base for policy. We illustrate this through a detailed look at Project STAR, an extensively analyzed, well-funded, large-scale, rigorous RCT commissioned by the Tennessee legislature to help it decide whether to mandate statewide class-size reductions (CSR) from kindergarten to the third grade. Project STAR randomly assigned students to classes of different size and compared test results across these classes, to obtain an unbiased answer to the research question, “Does reducing class size improve test scores?” However, this shed little light on whether reducing class size was a good use of increased education financing. Analyses of Project STAR ignored general equilibrium effects of CSR on both the demand for teachers and the value of test scores. Moreover, its emphasis on estimating average class-size effects in a particular setting diverted attention from their heterogeneity, and the need to understand how class size affects learning, and how its effect is moderated by circumstances. Rather than considering the full chain of evidence necessary for shaping class-size policy, Project STAR concentrated its effort on maximizing the accuracy of a single link in that chain; internal validity trumped policy relevance.

Suggested Citation

  • Moshe Justman, 2016. "Economic Research and Education Policy: Project STAR and Class Size Reduction," Melbourne Institute Working Paper Series wp2016n37, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.
  • Handle: RePEc:iae:iaewps:wp2016n37
    as

    Download full text from publisher

    File URL: http://melbourneinstitute.unimelb.edu.au/downloads/working_paper_series/wp2016n37.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    2. repec:fth:prinin:357 is not listed on IDEAS
    3. Weili Ding & Steven F. Lehrer, 2010. "Estimating Treatment Effects from Contaminated Multiperiod Education Experiments: The Dynamic Impacts of Class Size Reductions," The Review of Economics and Statistics, MIT Press, vol. 92(1), pages 31-42, February.
    4. Miguel Urquiola & Eric Verhoogen, 2009. "Class-Size Caps, Sorting, and the Regression-Discontinuity Design," American Economic Review, American Economic Association, vol. 99(1), pages 179-215, March.
    5. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    6. Janet Currie & Duncan Thomas, 1999. "Early Test Scores, Socioeconomic Status and Future Outcomes," NBER Working Papers 6943, National Bureau of Economic Research, Inc.
    7. Rothstein, Jesse & Von Wachter, Till, 2016. "Social Experiments in the Labor Market," Department of Economics, Working Paper Series qt7957p9g6, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
    8. David Card & Alan Krueger, 1996. "Labor Market Effects of School Quality: Theory and Evidence," Working Papers 736, Princeton University, Department of Economics, Industrial Relations Section..
    9. Angus Deaton & Nancy Cartwright, 2016. "Understanding and Misunderstanding Randomized Controlled Trials," Working Papers august_25.pdf, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
    10. Alan B. Krueger, 2003. "Economic Considerations and Class Size," Economic Journal, Royal Economic Society, vol. 113(485), pages 34-63, February.
    11. Necker, Sarah, 2014. "Scientific misbehavior in economics," Research Policy, Elsevier, vol. 43(10), pages 1747-1759.
    12. David Greenberg & Mark Shroder & Matthew Onstott, 1999. "The Social Experiment Market," Journal of Economic Perspectives, American Economic Association, vol. 13(3), pages 157-172, Summer.
    13. David Card & Alan B. Krueger, 1996. "Labor Market Effects of School Quality: Theory and Evidence," NBER Working Papers 5450, National Bureau of Economic Research, Inc.
    14. Krueger, Alan B & Whitmore, Diane M, 2001. "The Effect of Attending a Small Class in the Early Grades on College-Test Taking and Middle School Test Results: Evidence from Project STAR," Economic Journal, Royal Economic Society, vol. 111(468), pages 1-28, January.
    15. Guido W. Imbens, 2010. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 399-423, June.
    16. Aviva Aron-Dine & Liran Einav & Amy Finkelstein, 2013. "The RAND Health Insurance Experiment, Three Decades Later," Journal of Economic Perspectives, American Economic Association, vol. 27(1), pages 197-222, Winter.
    17. Funkhouser, Edward, 2009. "The effect of kindergarten classroom size reduction on second grade student achievement: Evidence from California," Economics of Education Review, Elsevier, vol. 28(3), pages 403-414, June.
    18. Alan B. Krueger, 1999. "Experimental Estimates of Education Production Functions," The Quarterly Journal of Economics, Oxford University Press, vol. 114(2), pages 497-532.
    19. James J. Heckman, 2000. "Causal Parameters and Policy Analysis in Economics: A Twentieth Century Retrospective," The Quarterly Journal of Economics, Oxford University Press, vol. 115(1), pages 45-97.
    20. Murnane, Richard J & Willett, John B & Levy, Frank, 1995. "The Growing Importance of Cognitive Skills in Wage Determination," The Review of Economics and Statistics, MIT Press, vol. 77(2), pages 251-266, May.
    21. Kasy, Maximilian, 2016. "Why Experimenters Might Not Always Want to Randomize, and What They Could Do Instead," Political Analysis, Cambridge University Press, vol. 24(3), pages 324-338, July.
    22. Abhijit Banerjee & Sylvain Chassang & Erik Snowberg, 2016. "Decision Theoretic Approaches to Experiment Design and External Validity," NBER Working Papers 22167, National Bureau of Economic Research, Inc.
    23. Joshua D. Angrist & Victor Lavy, 1999. "Using Maimonides' Rule to Estimate the Effect of Class Size on Scholastic Achievement," The Quarterly Journal of Economics, Oxford University Press, vol. 114(2), pages 533-575.
    24. Cho, Hyunkuk & Glewwe, Paul & Whitler, Melissa, 2012. "Do reductions in class size raise students’ test scores? Evidence from population variation in Minnesota's elementary schools," Economics of Education Review, Elsevier, vol. 31(3), pages 77-95.
    25. Stephen V. Cameron & James J. Heckman, 1998. "Life Cycle Schooling and Dynamic Selection Bias: Models and Evidence for Five Cohorts of American Males," Journal of Political Economy, University of Chicago Press, vol. 106(2), pages 262-333, April.
    26. Jens Ludwig & Jeffrey R. Kling & Sendhil Mullainathan, 2011. "Mechanism Experiments and Policy Evaluations," Journal of Economic Perspectives, American Economic Association, vol. 25(3), pages 17-38, Summer.
    27. Eric A. Hanushek, "undated". "The Evidence on Class Size," Wallis Working Papers WP10, University of Rochester - Wallis Institute of Political Economy.
    28. Caroline M. Hoxby, 2000. "The Effects of Class Size on Student Achievement: New Evidence from Population Variation," The Quarterly Journal of Economics, Oxford University Press, vol. 115(4), pages 1239-1285.
    29. Graham McKee & Katharine Sims & Steven Rivkin, 2015. "Disruption, learning, and the heterogeneous benefits of smaller classes," Empirical Economics, Springer, vol. 48(3), pages 1267-1286, May.
    30. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    31. Edward P. Lazear, 2001. "Educational Production," The Quarterly Journal of Economics, Oxford University Press, vol. 116(3), pages 777-803.
    32. Neal, Derek A & Johnson, William R, 1996. "The Role of Premarket Factors in Black-White Wage Differences," Journal of Political Economy, University of Chicago Press, vol. 104(5), pages 869-895, October.
    33. James J. Heckman & Vytlacil, Edward J., 2007. "Econometric Evaluation of Social Programs, Part I: Causal Models, Structural Models and Econometric Policy Evaluation," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 70, Elsevier.
    34. Christopher Jepsen & Steven Rivkin, 2009. "Class Size Reduction and Student Achievement: The Potential Tradeoff between Teacher Quality and Class Size," Journal of Human Resources, University of Wisconsin Press, vol. 44(1).
    35. Donald B. Rubin, 2005. "Causal Inference Using Potential Outcomes: Design, Modeling, Decisions," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 322-331, March.
    36. Stephen V. Cameron & James J. Heckman, 1998. "Life Cycle Schooling and Dynamic Selection Bias: Models and Evidence for Five Cohorts," NBER Working Papers 6385, National Bureau of Economic Research, Inc.
    37. Chingos, Matthew M., 2012. "The impact of a universal class-size reduction policy: Evidence from Florida's statewide mandate," Economics of Education Review, Elsevier, vol. 31(5), pages 543-562.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Justman, Moshe, 2018. "Randomized controlled trials informing public policy: Lessons from project STAR and class size reduction," European Journal of Political Economy, Elsevier, vol. 54(C), pages 167-174.
    2. Matthew M. Chingos & Kenneth A. Couch, 2013. "Class Size and Student Outcomes: Research and Policy Implications," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 32(2), pages 411-438, March.
    3. Marie Connolly & Catherine Haeck, 2018. "Le lien entre la taille des classes et les compétences cognitives et non cognitives," CIRANO Project Reports 2018rp-18, CIRANO.
    4. Maximilian Bach & Stephan Sievert, 2019. "Birth Cohort Size Variation and the Estimation of Class Size Effects," Discussion Papers of DIW Berlin 1817, DIW Berlin, German Institute for Economic Research.
    5. Petra E. Todd & Kenneth I. Wolpin, 2007. "The Production of Cognitive Achievement in Children: Home, School, and Racial Test Score Gaps," Journal of Human Capital, University of Chicago Press, vol. 1(1), pages 91-136.
    6. Alan B. Krueger, 2003. "Economic Considerations and Class Size," Economic Journal, Royal Economic Society, vol. 113(485), pages 34-63, February.
    7. Wößmann, Ludger, 2001. "New Evidence on the Missing Resource-Performance Link in Education," Kiel Working Papers 1051, Kiel Institute for the World Economy (IfW Kiel).
    8. Graham McKee & Katharine Sims & Steven Rivkin, 2015. "Disruption, learning, and the heterogeneous benefits of smaller classes," Empirical Economics, Springer, vol. 48(3), pages 1267-1286, May.
    9. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    10. Michael Gilraine & Hugh Macartney & Robert McMillan, 2018. "Estimating the Direct and Indirect Effects of Major Education Reforms," NBER Working Papers 24191, National Bureau of Economic Research, Inc.
    11. Hægeland, Torbjørn & Raaum, Oddbjørn & Salvanes, Kjell G., 2012. "Pennies from heaven? Using exogenous tax variation to identify effects of school resources on pupil achievement," Economics of Education Review, Elsevier, vol. 31(5), pages 601-614.
    12. Margaret Stevens & Kathryn Graddy, 2003. "The Impact of School Inputs on Student Performance: An Empirical Study of Private Schools in the United Kingdom," Economics Series Working Papers 146, University of Oxford, Department of Economics.
    13. Peter Fredriksson & Björn Öckert & Hessel Oosterbeek, 2013. "Long-Term Effects of Class Size," The Quarterly Journal of Economics, Oxford University Press, vol. 128(1), pages 249-285.
    14. Stevens, Margaret & Graddy, Kathryn, 2003. "The Impact of School Inputs on Student Performance: An Empirical Study of Private Schools in the UK," CEPR Discussion Papers 3776, C.E.P.R. Discussion Papers.
    15. Raj Chetty & John N. Friedman & Nathaniel Hilger & Emmanuel Saez & Diane Whitmore Schanzenbach & Danny Yagan, 2011. "How Does Your Kindergarten Classroom Affect Your Earnings? Evidence from Project Star," The Quarterly Journal of Economics, Oxford University Press, vol. 126(4), pages 1593-1660.
    16. Petra E. Todd & Kenneth I. Wolpin, 2003. "On The Specification and Estimation of The Production Function for Cognitive Achievement," Economic Journal, Royal Economic Society, vol. 113(485), pages 3-33, February.
    17. Hanushek, Eric A. & Luque, Javier A., 2003. "Efficiency and equity in schools around the world," Economics of Education Review, Elsevier, vol. 22(5), pages 481-502, October.
    18. Cohen-Zada, Danny & Gradstein, Mark & Reuven, Ehud, 2013. "Allocation of students in public schools: Theory and new evidence," Economics of Education Review, Elsevier, vol. 34(C), pages 96-106.
    19. Han, Joseph & Ryu, Keunkwan, 2017. "Effects of class size reduction in upper grades: Evidence from Seoul, Korea," Economics of Education Review, Elsevier, vol. 60(C), pages 68-85.
    20. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.

    More about this item

    Keywords

    Class size; Project STAR; randomized controlled trials; field experiments; internal validity; external validity; modern experimentalist paradigm;
    All these keywords.

    JEL classification:

    • C54 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Quantitative Policy Modeling
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iae:iaewps:wp2016n37. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sheri Carnegie (email available below). General contact details of provider: https://edirc.repec.org/data/mimelau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.