IDEAS home Printed from https://ideas.repec.org/a/eee/ecoedu/v64y2018icp50-74.html
   My bibliography  Save this article

Does the precision and stability of value-added estimates of teacher performance depend on the types of students they serve?

Author

Listed:
  • Stacy, Brian
  • Guarino, Cassandra
  • Wooldridge, Jeffrey

Abstract

In this paper, we investigate how the precision and year-to-year stability of a teacher’s value-added estimate relate to student characteristics. We find that teachers serving initially higher performing students have more precise value-added estimates and in most cases have higher year-to-year stability levels than teachers with lower performing students. We also decompose the variation in value-added estimates into components that reflect persistent and transitory variation in true teacher performance as well as variation caused by imprecision in the estimates. We find that teachers with lower performing students have less precision in their estimates and more transitory variation in value-added from year to year than other teachers. Our estimates imply that if teachers serving initially lower performing students had levels of precision and transitory variation in their value-added estimates equal to those serving higher performing students, the year-to-year stability in their estimates would actually exceed that of teachers with initially higher performing students.

Suggested Citation

  • Stacy, Brian & Guarino, Cassandra & Wooldridge, Jeffrey, 2018. "Does the precision and stability of value-added estimates of teacher performance depend on the types of students they serve?," Economics of Education Review, Elsevier, vol. 64(C), pages 50-74.
    • Handle: RePEc:eee:ecoedu:v:64:y:2018:i:c:p:50-74
    DOI: 10.1016/j.econedurev.2018.04.001
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0272775717300596
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.econedurev.2018.04.001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Thomas J. Kane & Douglas O. Staiger, 2008. "Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation," NBER Working Papers 14607, National Bureau of Economic Research, Inc.
    2. Thomas J. Kane & Douglas O. Staiger, 2002. "The Promise and Pitfalls of Using Imprecise School Accountability Measures," Journal of Economic Perspectives, American Economic Association, vol. 16(4), pages 91-114, Fall.
    3. Peter Z. Schochet & Hanley S. Chiang, 2013. "What Are Error Rates for Classifying Teacher and School Performance Using Value-Added Models?," Journal of Educational and Behavioral Statistics, , vol. 38(2), pages 142-171, April.
    4. Daniel F. McCaffrey & Tim R. Sass & J. R. Lockwood & Kata Mihaly, 2009. "The Intertemporal Variability of Teacher Effect Estimates," Education Finance and Policy, MIT Press, vol. 4(4), pages 572-606, October.
    5. Sass, Tim R. & Semykina, Anastasia & Harris, Douglas N., 2014. "Value-added models and the measurement of teacher productivity," Economics of Education Review, Elsevier, vol. 38(C), pages 9-23.
    6. Cassandra M. Guarino & Mark D. Reckase & Jeffrey M. Woolrdige, 2014. "Can Value-Added Measures of Teacher Performance Be Trusted?," Education Finance and Policy, MIT Press, vol. 10(1), pages 117-156, November.
    7. Daniel Aaronson & Lisa Barrow & William Sander, 2007. "Teachers and Student Achievement in the Chicago Public High Schools," Journal of Labor Economics, University of Chicago Press, vol. 25(1), pages 95-135.
    8. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    9. Heinrich Hock & Eric Isenberg, "undated". "Methods for Accounting for Co-Teaching in Value-Added Models," Mathematica Policy Research Reports e53ebf9f792e48dca345a4bba, Mathematica Policy Research.
    10. Dan Goldhaber & Michael Hansen, 2013. "Is it Just a Bad Class? Assessing the Long-term Stability of Estimated Teacher Performance," Economica, London School of Economics and Political Science, vol. 80(319), pages 589-612, July.
    11. Dale Ballou & William Sanders & Paul Wright, 2004. "Controlling for Student Background in Value-Added Assessment of Teachers," Journal of Educational and Behavioral Statistics, , vol. 29(1), pages 37-65, March.
    12. Dan Goldhaber & Duncan Dunbar Chaplin, "undated". "Assessing the "Rothstein Falsification Test": Does It Really Show Teacher Value-Added Models Are Biased?," Mathematica Policy Research Reports 93f34c834817419a9efccecdf, Mathematica Policy Research.
    13. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2011. "The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood," NBER Working Papers 17699, National Bureau of Economic Research, Inc.
    14. Guarino, Cassandra M. & Reckase, Mark D. & Stacy, Brian & Wooldridge, Jeffrey M., 2014. "A Comparison of Growth Percentile and Value-Added Models of Teacher Performance," IZA Discussion Papers 7973, Institute of Labor Economics (IZA).
    15. Thomas S. Dee & James Wyckoff, 2015. "Incentives, Selection, and Teacher Performance: Evidence from IMPACT," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 34(2), pages 267-297, March.
    16. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood," American Economic Review, American Economic Association, vol. 104(9), pages 2633-2679, September.
    17. Jesse Rothstein, 2010. "Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement," The Quarterly Journal of Economics, Oxford University Press, vol. 125(1), pages 175-214.
    18. Jason A. Grissom & Susanna Loeb & Nathaniel A. Nakashima, 2014. "Strategic Involuntary Teacher Transfers and Teacher Performance: Examining Equity and Efficiency," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(1), pages 112-140, January.
    19. Brian A. Jacob & Lars Lefgren, 2008. "Can Principals Identify Effective Teachers? Evidence on Subjective Performance Evaluation in Education," Journal of Labor Economics, University of Chicago Press, vol. 26(1), pages 101-136.
    20. Cory Koedel & Julian Betts, 2007. "Re-Examining the Role of Teacher Quality In the Educational Production Function," Working Papers 0708, Department of Economics, University of Missouri.
    21. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, December.
    22. Eric A. Hanushek, 1979. "Conceptual and Empirical Issues in the Estimation of Educational Production Functions," Journal of Human Resources, University of Wisconsin Press, vol. 14(3), pages 351-388.
    23. Peter Z. Schochet & Hanley S. Chiang, "undated". "What Are Error Rates for Classifying Teacher and School Performance Using Value-Added Models?," Mathematica Policy Research Reports 8cc459dd9c574c3d832ed4182, Mathematica Policy Research.
    24. Petra E. Todd & Kenneth I. Wolpin, 2003. "On The Specification and Estimation of The Production Function for Cognitive Achievement," Economic Journal, Royal Economic Society, vol. 113(485), pages 3-33, February.
    25. repec:mpr:mprres:7762 is not listed on IDEAS
    26. repec:mpr:mprres:7482 is not listed on IDEAS
    27. J. R. Lockwood & Daniel F. McCaffrey, 2014. "Correcting for Test Score Measurement Error in ANCOVA Models for Estimating Treatment Effects," Journal of Educational and Behavioral Statistics, , vol. 39(1), pages 22-52, February.
    28. Cassandra M. Guarino & Michelle Maxfield & Mark D. Reckase & Paul N. Thompson & Jeffrey M. Wooldridge, 2015. "An Evaluation of Empirical Bayes’s Estimation of Value-Added Teacher Performance Measures," Journal of Educational and Behavioral Statistics, , vol. 40(2), pages 190-222, April.
    29. Matthew P. Steinberg & Morgaen L. Donaldson, 2016. "The New Educational Accountability: Understanding the Landscape of Teacher Evaluation in the Post-NCLB Era," Education Finance and Policy, MIT Press, vol. 11(3), pages 340-359, Summer.
    30. Daniel F. McCaffrey & J. R. Lockwood & Daniel Koretz & Thomas A. Louis & Laura Hamilton, 2004. "Models for Value-Added Modeling of Teacher Effects," Journal of Educational and Behavioral Statistics, , vol. 29(1), pages 67-101, March.
    31. Guarino, Cassandra M. & Reckase, Mark D. & Stacy, Brian & Wooldridge, Jeffrey M., 2014. "Evaluating Specification Tests in the Context of Value-Added Estimation," IZA Discussion Papers 7974, Institute of Labor Economics (IZA).
    32. Wiswall, Matthew, 2013. "The dynamics of teacher quality," Journal of Public Economics, Elsevier, vol. 100(C), pages 61-78.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Cory Koedel & Jiaxi Li, 2016. "The Efficiency Implications Of Using Proportional Evaluations To Shape The Teaching Workforce," Contemporary Economic Policy, Western Economic Association International, vol. 34(1), pages 47-62, January.
    2. Esteban M. Aucejo & Patrick Coate & Jane Cooley Fruehwirth & Sean Kelly & Zachary Mozenter, 2018. "Teacher effectiveness and classroom composition," CEP Discussion Papers dp1574, Centre for Economic Performance, LSE.
    3. Mariesa Herrmann & Elias Walsh & Eric Isenberg & Alexandra Resch, 2013. "Shrinkage of Value-Added Estimates and Characteristics of Students with Hard-to-Predict Achievement Levels," Mathematica Policy Research Reports 2b140369be0242ac83eeb5b0a, Mathematica Policy Research.
    4. repec:mpr:mprres:7817 is not listed on IDEAS
    5. Eric Parsons & Cory Koedel & Li Tan, 2019. "Accounting for Student Disadvantage in Value-Added Models," Journal of Educational and Behavioral Statistics, , vol. 44(2), pages 144-179, April.
    6. Backes, Ben & Cowan, James & Goldhaber, Dan & Koedel, Cory & Miller, Luke C. & Xu, Zeyu, 2018. "The common core conundrum: To what extent should we worry that changes to assessments will affect test-based measures of teacher performance?," Economics of Education Review, Elsevier, vol. 62(C), pages 48-65.
    7. repec:mpr:mprres:7748 is not listed on IDEAS
    8. Elias Walsh & Stephen Lipscomb, "undated". "Classroom Observations from Phase 2 of the Pennsylvania Teacher Evaluation Pilot: Assessing Internal Consistency, Score Variation, and Relationships with Value Added," Mathematica Policy Research Reports a6b29a4a217f42a09d5206cfe, Mathematica Policy Research.
    9. Oketch, Moses & Rolleston, Caine & Rossiter, Jack, 2021. "Diagnosing the learning crisis: What can value-added analysis contribute?," International Journal of Educational Development, Elsevier, vol. 87(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Koedel, Cory & Mihaly, Kata & Rockoff, Jonah E., 2015. "Value-added modeling: A review," Economics of Education Review, Elsevier, vol. 47(C), pages 180-195.
    2. Backes, Ben & Cowan, James & Goldhaber, Dan & Koedel, Cory & Miller, Luke C. & Xu, Zeyu, 2018. "The common core conundrum: To what extent should we worry that changes to assessments will affect test-based measures of teacher performance?," Economics of Education Review, Elsevier, vol. 62(C), pages 48-65.
    3. Stacy, Brian, 2014. "Ranking Teachers when Teacher Value-Added is Heterogeneous Across Students," EconStor Preprints 104743, ZBW - Leibniz Information Centre for Economics.
    4. Vosters, Kelly N. & Guarino, Cassandra M. & Wooldridge, Jeffrey M., 2018. "Understanding and evaluating the SAS® EVAAS® Univariate Response Model (URM) for measuring teacher effectiveness," Economics of Education Review, Elsevier, vol. 66(C), pages 191-205.
    5. Cassandra M. Guarino & Mark D. Reckase & Jeffrey M. Woolrdige, 2014. "Can Value-Added Measures of Teacher Performance Be Trusted?," Education Finance and Policy, MIT Press, vol. 10(1), pages 117-156, November.
    6. Dan Goldhaber & Michael Hansen, 2013. "Is it Just a Bad Class? Assessing the Long-term Stability of Estimated Teacher Performance," Economica, London School of Economics and Political Science, vol. 80(319), pages 589-612, July.
    7. Guarino, Cassandra M. & Reckase, Mark D. & Stacy, Brian & Wooldridge, Jeffrey M., 2014. "A Comparison of Growth Percentile and Value-Added Models of Teacher Performance," IZA Discussion Papers 7973, Institute of Labor Economics (IZA).
    8. Eric Parsons & Cory Koedel & Li Tan, 2019. "Accounting for Student Disadvantage in Value-Added Models," Journal of Educational and Behavioral Statistics, , vol. 44(2), pages 144-179, April.
    9. Gershenson, Seth, 2021. "Identifying and Producing Effective Teachers," IZA Discussion Papers 14096, Institute of Labor Economics (IZA).
    10. Harris, Douglas N. & Sass, Tim R., 2014. "Skills, productivity and the evaluation of teacher performance," Economics of Education Review, Elsevier, vol. 40(C), pages 183-204.
    11. Koedel Cory & Leatherman Rebecca & Parsons Eric, 2012. "Test Measurement Error and Inference from Value-Added Models," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 12(1), pages 1-37, November.
    12. Nirav Mehta, 2019. "Measuring quality for use in incentive schemes: The case of “shrinkage” estimators," Quantitative Economics, Econometric Society, vol. 10(4), pages 1537-1577, November.
    13. Azam, Mehtabul & Kingdon, Geeta Gandhi, 2015. "Assessing teacher quality in India," Journal of Development Economics, Elsevier, vol. 117(C), pages 74-83.
    14. Goldhaber, Dan & Cowan, James & Walch, Joe, 2013. "Is a good elementary teacher always good? Assessing teacher performance estimates across subjects," Economics of Education Review, Elsevier, vol. 36(C), pages 216-228.
    15. Araujo P., Maria Daniela & Quis, Johanna Sophie, 2021. "Parents can tell! Evidence on classroom quality differences in German primary schools," BERG Working Paper Series 172, Bamberg University, Bamberg Economic Research Group.
    16. Seth Gershenson & Diane Whitmore Schanzenbach, 2016. "Linking Teacher Quality, Student Attendance, and Student Achievement," Education Finance and Policy, MIT Press, vol. 11(2), pages 125-149, Spring.
    17. Araujo P., María Daniela & Quis, Johanna Sophie, 2021. "Teacher Effects in Germany: Evidence from Elementary School," VfS Annual Conference 2021 (Virtual Conference): Climate Economics 242457, Verein für Socialpolitik / German Economic Association.
    18. Steven Dieterle & Cassandra M. Guarino & Mark D. Reckase & Jeffrey M. Wooldridge, 2015. "How do Principals Assign Students to Teachers? Finding Evidence in Administrative Data and the Implications for Value Added," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 34(1), pages 32-58, January.
    19. Allison Atteberry & Susanna Loeb & James Wyckoff, 2013. "Do First Impressions Matter? Improvement in Early Career Teacher Effectiveness," NBER Working Papers 19096, National Bureau of Economic Research, Inc.
    20. M. Caridad Araujo & Pedro Carneiro & Yyannú Cruz-Aguayo & Norbert Schady, 2016. "Teacher Quality and Learning Outcomes in Kindergarten," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(3), pages 1415-1453.

    More about this item

    Keywords

    Teacher value-added; Precision; Stability; Student achievement; Teacher accountability;
    All these keywords.

    JEL classification:

    • I2 - Health, Education, and Welfare - - Education
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy
    • J3 - Labor and Demographic Economics - - Wages, Compensation, and Labor Costs
    • C01 - Mathematical and Quantitative Methods - - General - - - Econometrics
    • C33 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Models with Panel Data; Spatio-temporal Models

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecoedu:v:64:y:2018:i:c:p:50-74. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/econedurev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.