IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/26480.html
   My bibliography  Save this paper

Teacher Effects on Student Achievement and Height: A Cautionary Tale

Author

Listed:
  • Marianne Bitler
  • Sean Corcoran
  • Thurston Domina
  • Emily Penner

Abstract

Estimates of teacher “value-added” suggest teachers vary substantially in their ability to promote student learning. Prompted by this finding, many states and school districts have adopted value-added measures as indicators of teacher job performance. In this paper, we conduct a new test of the validity of value-added models. Using administrative student data from New York City, we apply commonly estimated value-added models to an outcome teachers cannot plausibly affect: student height. We find the standard deviation of teacher effects on height is nearly as large as that for math and reading achievement, raising obvious questions about validity. Subsequent analysis finds these “effects” are largely spurious variation (noise), rather than bias resulting from sorting on unobserved factors related to achievement. Given the difficulty of differentiating signal from noise in real-world teacher effect estimates, this paper serves as a cautionary tale for their use in practice.

Suggested Citation

  • Marianne Bitler & Sean Corcoran & Thurston Domina & Emily Penner, 2019. "Teacher Effects on Student Achievement and Height: A Cautionary Tale," NBER Working Papers 26480, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:26480
    Note: CH ED
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w26480.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Thomas J. Kane & Douglas O. Staiger, 2008. "Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation," NBER Working Papers 14607, National Bureau of Economic Research, Inc.
    2. Peter Z. Schochet & Hanley S. Chiang, 2013. "What Are Error Rates for Classifying Teacher and School Performance Using Value-Added Models?," Journal of Educational and Behavioral Statistics, , vol. 38(2), pages 142-171, April.
    3. Josh Kinsler, 2012. "Assessing Rothstein's critique of teacher value‐added models," Quantitative Economics, Econometric Society, vol. 3(2), pages 333-362, July.
    4. Dale Ballou & William Sanders & Paul Wright, 2004. "Controlling for Student Background in Value-Added Assessment of Teachers," Journal of Educational and Behavioral Statistics, , vol. 29(1), pages 37-65, March.
    5. Andrew Bacher-Hicks & Mark J. Chin & Thomas J. Kane & Douglas O. Staiger, 2017. "An Evaluation of Bias in Three Measures of Teacher Quality: Value-Added, Classroom Observations, and Student Surveys," NBER Working Papers 23478, National Bureau of Economic Research, Inc.
    6. Peter Z. Schochet & Hanley S. Chiang, "undated". "What Are Error Rates for Classifying Teacher and School Performance Using Value-Added Models?," Mathematica Policy Research Reports 8cc459dd9c574c3d832ed4182, Mathematica Policy Research.
    7. Jonah E. Rockoff, 2004. "The Impact of Individual Teachers on Student Achievement: Evidence from Panel Data," American Economic Review, American Economic Association, vol. 94(2), pages 247-252, May.
    8. Jonah E. Rockoff & Douglas O. Staiger & Thomas J. Kane & Eric S. Taylor, 2012. "Information and Employee Evaluation: Evidence from a Randomized Intervention in Public Schools," American Economic Review, American Economic Association, vol. 102(7), pages 3184-3213, December.
    9. Daniel F. McCaffrey & Tim R. Sass & J. R. Lockwood & Kata Mihaly, 2009. "The Intertemporal Variability of Teacher Effect Estimates," Education Finance and Policy, MIT Press, vol. 4(4), pages 572-606, October.
    10. Daniel Aaronson & Lisa Barrow & William Sander, 2007. "Teachers and Student Achievement in the Chicago Public High Schools," Journal of Labor Economics, University of Chicago Press, vol. 25(1), pages 95-135.
    11. Kane, Thomas J. & Rockoff, Jonah E. & Staiger, Douglas O., 2008. "What does certification tell us about teacher effectiveness? Evidence from New York City," Economics of Education Review, Elsevier, vol. 27(6), pages 615-631, December.
    12. Cory Koedel & Julian R. Betts, 2011. "Does Student Sorting Invalidate Value-Added Models of Teacher Effectiveness? An Extended Analysis of the Rothstein Critique," Education Finance and Policy, MIT Press, vol. 6(1), pages 18-42, January.
    13. Andrew Bacher-Hicks & Thomas J. Kane & Douglas O. Staiger, 2014. "Validating Teacher Effect Estimates Using Changes in Teacher Assignments in Los Angeles," NBER Working Papers 20657, National Bureau of Economic Research, Inc.
    14. Jesse Rothstein, 2010. "Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 175-214.
    15. repec:mpr:mprres:6971 is not listed on IDEAS
    16. Jun Ishii & Steven G. Rivkin, 2009. "Impediments to the Estimation of Teacher Value Added," Education Finance and Policy, MIT Press, vol. 4(4), pages 520-536, October.
    17. repec:mpr:mprres:7762 is not listed on IDEAS
    18. repec:mpr:mprres:7020 is not listed on IDEAS
    19. Eric A. Hanushek & Steven G. Rivkin, 2010. "Generalizations about Using Value-Added Measures of Teacher Quality," American Economic Review, American Economic Association, vol. 100(2), pages 267-271, May.
    20. repec:mpr:mprres:6829 is not listed on IDEAS
    21. Eric Isenberg & Heinrich Hock, 2011. "Design of Value-Added Models for IMPACT and TEAM in DC Public Schools, 2010-2011 School Year," Mathematica Policy Research Reports 885c9869ad064392aaa24d9e4, Mathematica Policy Research.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Araujo P., Maria Daniela & Quis, Johanna Sophie, 2021. "Parents can tell! Evidence on classroom quality differences in German primary schools," BERG Working Paper Series 172, Bamberg University, Bamberg Economic Research Group.
    2. Michael Gilraine & Jiaying Gu & Robert McMillan, 2020. "A New Method for Estimating Teacher Value-Added," NBER Working Papers 27094, National Bureau of Economic Research, Inc.
    3. Yue-Yi Hwa & Clare Leaver, 2021. "Management in education systems," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 37(2), pages 367-391.
    4. Michael Gilraine & Jiaying Gu & Robert McMillan, 2021. "A Nonparametric Method for Estimating Teacher Value-Added," Working Papers tecipa-689, University of Toronto, Department of Economics.
    5. Soonwoo Kwon, 2023. "Optimal Shrinkage Estimation of Fixed Effects in Linear Panel Data Models," Papers 2308.12485, arXiv.org, revised Oct 2023.
    6. Pauline Givord & Milena Suarez Castillo, 2021. "What Makes a Good High School? Measuring School Effects beyond the Average," Economie et Statistique / Economics and Statistics, Institut National de la Statistique et des Etudes Economiques (INSEE), issue 528-529, pages 29-45.
    7. Naven, Matthew, 2020. "Within-School Heterogeneity in Quality: Do Schools Provide Equal Value Added to All Students?," MPRA Paper 100123, University Library of Munich, Germany.
    8. John Jerrim & Sam Sims & Rebecca Allen, 2021. "The mental health and wellbeing of teachers in England," DoQSS Working Papers 21-01r, Quantitative Social Science - UCL Social Research Institute, University College London.
    9. Michael J. Andrews & Aaron K. Chatterji & Scott Stern, 2021. "Introduction: Beyond 140 Characters," NBER Chapters, in: The Role of Innovation and Entrepreneurship in Economic Growth, pages 1-28, National Bureau of Economic Research, Inc.
    10. Araujo P., María Daniela & Quis, Johanna Sophie, 2021. "Teacher Effects in Germany: Evidence from Elementary School," VfS Annual Conference 2021 (Virtual Conference): Climate Economics 242457, Verein für Socialpolitik / German Economic Association.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Koedel, Cory & Mihaly, Kata & Rockoff, Jonah E., 2015. "Value-added modeling: A review," Economics of Education Review, Elsevier, vol. 47(C), pages 180-195.
    2. Dan Goldhaber & Michael Hansen, 2013. "Is it Just a Bad Class? Assessing the Long-term Stability of Estimated Teacher Performance," Economica, London School of Economics and Political Science, vol. 80(319), pages 589-612, July.
    3. Koedel Cory & Leatherman Rebecca & Parsons Eric, 2012. "Test Measurement Error and Inference from Value-Added Models," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 12(1), pages 1-37, November.
    4. Araujo P., Maria Daniela & Quis, Johanna Sophie, 2021. "Parents can tell! Evidence on classroom quality differences in German primary schools," BERG Working Paper Series 172, Bamberg University, Bamberg Economic Research Group.
    5. Araujo P., María Daniela & Quis, Johanna Sophie, 2021. "Teacher Effects in Germany: Evidence from Elementary School," VfS Annual Conference 2021 (Virtual Conference): Climate Economics 242457, Verein für Socialpolitik / German Economic Association.
    6. Allison Atteberry & Susanna Loeb & James Wyckoff, 2013. "Do First Impressions Matter? Improvement in Early Career Teacher Effectiveness," NBER Working Papers 19096, National Bureau of Economic Research, Inc.
    7. Sean Corcoran & Dan Goldhaber, 2013. "Value Added and Its Uses: Where You Stand Depends on Where You Sit," Education Finance and Policy, MIT Press, vol. 8(3), pages 418-434, July.
    8. Marine de Talancé, 2015. "Better Teachers, Better Results? Evidence from Rural Pakistan," Working Papers DT/2015/21, DIAL (Développement, Institutions et Mondialisation).
    9. Canales, Andrea & Maldonado, Luis, 2018. "Teacher quality and student achievement in Chile: Linking teachers' contribution and observable characteristics," International Journal of Educational Development, Elsevier, vol. 60(C), pages 33-50.
    10. Stacy, Brian & Guarino, Cassandra & Wooldridge, Jeffrey, 2018. "Does the precision and stability of value-added estimates of teacher performance depend on the types of students they serve?," Economics of Education Review, Elsevier, vol. 64(C), pages 50-74.
    11. Hanushek, Eric A., 2011. "The economic value of higher teacher quality," Economics of Education Review, Elsevier, vol. 30(3), pages 466-479, June.
    12. Naven, Matthew, 2019. "Human-Capital Formation During Childhood and Adolescence: Evidence from School Quality and Postsecondary Success in California," MPRA Paper 97716, University Library of Munich, Germany.
    13. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    14. Eric S. Taylor & John H. Tyler, 2011. "The Effect of Evaluation on Performance: Evidence from Longitudinal Student Achievement Data of Mid-career Teachers," NBER Working Papers 16877, National Bureau of Economic Research, Inc.
    15. Lavy, Victor & Megalokonomou, Rigissa, 2024. "Alternative Measures of Teachers' Value Added and Impact on Short and Long-Term Outcomes: Evidence from Random Assignment," IZA Discussion Papers 17121, Institute of Labor Economics (IZA).
    16. Nirav Mehta, 2014. "Targeting the Wrong Teachers: Estimating Teacher Quality for Use in Accountability Regimes," University of Western Ontario, Centre for Human Capital and Productivity (CHCP) Working Papers 20143, University of Western Ontario, Centre for Human Capital and Productivity (CHCP).
    17. Josh Kinsler, 2016. "Teacher Complementarities in Test Score Production: Evidence from Primary School," Journal of Labor Economics, University of Chicago Press, vol. 34(1), pages 29-61.
    18. Hanushek, Eric A. & Rivkin, Steven G. & Schiman, Jeffrey C., 2016. "Dynamic effects of teacher turnover on the quality of instruction," Economics of Education Review, Elsevier, vol. 55(C), pages 132-148.
    19. Peter Z. Schochet & Hanley S. Chiang, 2013. "What Are Error Rates for Classifying Teacher and School Performance Using Value-Added Models?," Journal of Educational and Behavioral Statistics, , vol. 38(2), pages 142-171, April.
    20. Harris, Douglas N. & Sass, Tim R., 2014. "Skills, productivity and the evaluation of teacher performance," Economics of Education Review, Elsevier, vol. 40(C), pages 183-204.

    More about this item

    JEL classification:

    • I2 - Health, Education, and Welfare - - Education
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:26480. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.