IDEAS home Printed from https://ideas.repec.org/a/tpr/edfpol/v6y2011i1p18-42.html
   My bibliography  Save this article

Does Student Sorting Invalidate Value-Added Models of Teacher Effectiveness? An Extended Analysis of the Rothstein Critique

Author

Listed:
  • Cory Koedel

    (Department of Economics, University of Missouri)

  • Julian R. Betts

    (Department of Economics, University of California, San Diego and NBER)

Abstract

Value-added modeling continues to gain traction as a tool for measuring teacher performance. However, recent research questions the validity of the value-added approach by showing that it does not mitigate student-teacher sorting bias (its presumed primary benefit). Our study explores this critique in more detail. Although we find that estimated teacher effects from some value-added models are severely biased, we also show that a sufficiently complex value-added model that evaluates teachers over multiple years reduces the sorting bias problem to statistical insignificance. One implication of our findings is that data from the first year or two of classroom teaching for novice teachers may be insufficient to make reliable judgments about quality. Overall, our results suggest that in some cases value-added modeling will continue to provide useful information about the effectiveness of educational inputs. © 2011 Association for Education Finance and Policy

Suggested Citation

  • Cory Koedel & Julian R. Betts, 2011. "Does Student Sorting Invalidate Value-Added Models of Teacher Effectiveness? An Extended Analysis of the Rothstein Critique," Education Finance and Policy, MIT Press, vol. 6(1), pages 18-42, January.
    • Handle: RePEc:tpr:edfpol:v:6:y:2011:i:1:p:18-42
    as

    Download full text from publisher

    File URL: http://www.mitpressjournals.org/doi/pdf/10.1162/EDFP_a_00027
    Download Restriction: Access to PDF is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Thomas J. Kane & Douglas O. Staiger, 2008. "Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation," NBER Working Papers 14607, National Bureau of Economic Research, Inc.
    2. Thomas J. Kane & Douglas O. Staiger, 2002. "The Promise and Pitfalls of Using Imprecise School Accountability Measures," Journal of Economic Perspectives, American Economic Association, vol. 16(4), pages 91-114, Fall.
    3. Daniel F. McCaffrey & Tim R. Sass & J. R. Lockwood & Kata Mihaly, 2009. "The Intertemporal Variability of Teacher Effect Estimates," Education Finance and Policy, MIT Press, vol. 4(4), pages 572-606, October.
    4. Sass, Tim R. & Semykina, Anastasia & Harris, Douglas N., 2014. "Value-added models and the measurement of teacher productivity," Economics of Education Review, Elsevier, vol. 38(C), pages 9-23.
    5. Eric A. Hanushek & John F. Kain & Steven G. Rivkin & Daniel M. O'Brien, 2005. "The Market for Teacher Quality," Discussion Papers 04-025, Stanford Institute for Economic Policy Research.
    6. Daniel Aaronson & Lisa Barrow & William Sander, 2007. "Teachers and Student Achievement in the Chicago Public High Schools," Journal of Labor Economics, University of Chicago Press, vol. 25(1), pages 95-135.
    7. Eric A. Hanushek, 1996. "Measuring Investment in Education," Journal of Economic Perspectives, American Economic Association, vol. 10(4), pages 9-30, Fall.
    8. Cory Koedel & Julian Betts, 2007. "Re-Examining the Role of Teacher Quality In the Educational Production Function," Working Papers 0708, Department of Economics, University of Missouri.
    9. Charles T. Clotfelter & Helen F. Ladd & Jacob L. Vigdor, 2006. "Teacher-Student Matching and the Assessment of Teacher Effectiveness," Journal of Human Resources, University of Wisconsin Press, vol. 41(4).
    10. Brian A. Jacob & Lars Lefgren, 2005. "Principals as Agents: Subjective Performance Measurement in Education," NBER Working Papers 11463, National Bureau of Economic Research, Inc.
    11. Jonah E. Rockoff, 2004. "The Impact of Individual Teachers on Student Achievement: Evidence from Panel Data," American Economic Review, American Economic Association, vol. 94(2), pages 247-252, May.
    12. Michael J. Podgursky & Matthew G. Springer, 2007. "Teacher performance pay: A review," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 26(4), pages 909-950.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Allison Atteberry & Susanna Loeb & James Wyckoff, 2013. "Do First Impressions Matter? Improvement in Early Career Teacher Effectiveness," NBER Working Papers 19096, National Bureau of Economic Research, Inc.
    2. Koedel Cory & Leatherman Rebecca & Parsons Eric, 2012. "Test Measurement Error and Inference from Value-Added Models," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 12(1), pages 1-37, November.
    3. Cory Koedel & Julian Betts, 2010. "Value Added to What? How a Ceiling in the Testing Instrument Influences Value-Added Estimation," Education Finance and Policy, MIT Press, vol. 5(1), pages 54-81, January.
    4. Dan Goldhaber & Michael Hansen, 2013. "Is it Just a Bad Class? Assessing the Long-term Stability of Estimated Teacher Performance," Economica, London School of Economics and Political Science, vol. 80(319), pages 589-612, July.
    5. repec:pri:cepsud:159rothstein is not listed on IDEAS
    6. Stacy, Brian & Guarino, Cassandra & Wooldridge, Jeffrey, 2018. "Does the precision and stability of value-added estimates of teacher performance depend on the types of students they serve?," Economics of Education Review, Elsevier, vol. 64(C), pages 50-74.
    7. Koedel, Cory, 2009. "An empirical analysis of teacher spillover effects in secondary school," Economics of Education Review, Elsevier, vol. 28(6), pages 682-692, December.
    8. Koedel, Cory & Mihaly, Kata & Rockoff, Jonah E., 2015. "Value-added modeling: A review," Economics of Education Review, Elsevier, vol. 47(C), pages 180-195.
    9. Jesse Rothstein, 2010. "Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 175-214.
    10. Azam, Mehtabul & Kingdon, Geeta Gandhi, 2015. "Assessing teacher quality in India," Journal of Development Economics, Elsevier, vol. 117(C), pages 74-83.
    11. Hanushek, Eric A. & Rivkin, Steven G. & Schiman, Jeffrey C., 2016. "Dynamic effects of teacher turnover on the quality of instruction," Economics of Education Review, Elsevier, vol. 55(C), pages 132-148.
    12. Lindsay Fox, 2016. "Playing to Teachers’ Strengths: Using Multiple Measures of Teacher Effectiveness to Improve Teacher Assignments," Education Finance and Policy, MIT Press, vol. 11(1), pages 70-96, Winter.
    13. Douglas O. Staiger & Jonah E. Rockoff, 2010. "Searching for Effective Teachers with Imperfect Information," Journal of Economic Perspectives, American Economic Association, vol. 24(3), pages 97-118, Summer.
    14. Jesse Rothstein, 2007. "Do Value-Added Models Add Value? Tracking, Fixed Effects, and Causal Inference," Working Papers 1036, Princeton University, Department of Economics, Center for Economic Policy Studies..
    15. Pau Balart & Antonio Cabrales, 2015. "La evaluación docente basada en el resultado como vía de mejora del sistema educativo," Studies on the Spanish Economy eee2015-13, FEDEA.
    16. Peter Z. Schochet & Hanley S. Chiang, 2013. "What Are Error Rates for Classifying Teacher and School Performance Using Value-Added Models?," Journal of Educational and Behavioral Statistics, , vol. 38(2), pages 142-171, April.
    17. Cory Koedel & Julian Betts, 2007. "Re-Examining the Role of Teacher Quality In the Educational Production Function," Working Papers 0708, Department of Economics, University of Missouri.
    18. Cory Koedel & Julian Betts, 2007. "Teacher Quality and Educational Production in Secondary School," Working Papers 0707, Department of Economics, University of Missouri.
    19. Susanna Loeb & Demetra Kalogrides & Tara Béteille, 2011. "Effective Schools: Teacher Hiring, Assignment, Development, and Retention," NBER Working Papers 17177, National Bureau of Economic Research, Inc.
    20. Condie, Scott & Lefgren, Lars & Sims, David, 2014. "Teacher heterogeneity, value-added and education policy," Economics of Education Review, Elsevier, vol. 40(C), pages 76-92.
    21. Nirav Mehta, 2019. "Measuring quality for use in incentive schemes: The case of “shrinkage” estimators," Quantitative Economics, Econometric Society, vol. 10(4), pages 1537-1577, November.

    More about this item

    Keywords

    student sorting; teacher effectiveness; value-added modeling;
    All these keywords.

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I22 - Health, Education, and Welfare - - Education - - - Educational Finance; Financial Aid

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tpr:edfpol:v:6:y:2011:i:1:p:18-42. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Kelly McDougall (email available below). General contact details of provider: https://direct.mit.edu/journals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.