IDEAS home Printed from https://ideas.repec.org/p/mpr/mprres/2b140369be0242ac83eeb5b0afea95da.html

Shrinkage of Value-Added Estimates and Characteristics of Students with Hard-to-Predict Achievement Levels

Author

Listed:
  • Mariesa Herrmann
  • Elias Walsh
  • Eric Isenberg
  • Alexandra Resch

Abstract

This working paper investigates how empirical Bayes shrinkage, an approach commonly used in implementing teacher accountability systems, affects the value-added estimates of teachers of students with hard-to-predict achievement levels, such as students who have low prior achievement and receive free lunch. Teachers of these students tend to have less precise value-added estimates than teachers of other types of students. Shrinkage increases their estimates’ precision and reduces the absolute value of their value-added estimates. However, this paper found shrinkage has no statistically significant effect on the relative probability that teachers of hard-to-predict students receive value-added estimates that fall in the extremes of the value-added distribution and, as a result, receive consequences in the accountability system.

Suggested Citation

  • Mariesa Herrmann & Elias Walsh & Eric Isenberg & Alexandra Resch, 2013. "Shrinkage of Value-Added Estimates and Characteristics of Students with Hard-to-Predict Achievement Levels," Mathematica Policy Research Reports 2b140369be0242ac83eeb5b0a, Mathematica Policy Research.
    • Handle: RePEc:mpr:mprres:2b140369be0242ac83eeb5b0afea95da
    as

    Download full text from publisher

    File URL: https://www.mathematica.org/-/media/publications/pdfs/education/value-added_shrinkage_wp.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Cassandra M. Guarino & Michelle Maxfield & Mark D. Reckase & Paul N. Thompson & Jeffrey M. Wooldridge, 2015. "An Evaluation of Empirical Bayes’s Estimation of Value-Added Teacher Performance Measures," Journal of Educational and Behavioral Statistics, , vol. 40(2), pages 190-222, April.
    2. Thomas J. Kane & Douglas O. Staiger, 2008. "Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation," NBER Working Papers 14607, National Bureau of Economic Research, Inc.
    3. Thomas J. Kane & Douglas O. Staiger, 2002. "The Promise and Pitfalls of Using Imprecise School Accountability Measures," Journal of Economic Perspectives, American Economic Association, vol. 16(4), pages 91-114, Fall.
    4. Stacy, Brian & Guarino, Cassandra & Wooldridge, Jeffrey, 2018. "Does the precision and stability of value-added estimates of teacher performance depend on the types of students they serve?," Economics of Education Review, Elsevier, vol. 64(C), pages 50-74.
    5. Heinrich Hock & Eric Isenberg, "undated". "Methods for Accounting for Co-Teaching in Value-Added Models," Mathematica Policy Research Reports e53ebf9f792e48dca345a4bba, Mathematica Policy Research.
    6. Jesse Rothstein, 2010. "Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 175-214.
    7. Daniel Aaronson & Lisa Barrow & William Sander, 2007. "Teachers and Student Achievement in the Chicago Public High Schools," Journal of Labor Economics, University of Chicago Press, vol. 25(1), pages 95-135.
    8. Koedel Cory & Leatherman Rebecca & Parsons Eric, 2012. "Test Measurement Error and Inference from Value-Added Models," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 12(1), pages 1-37, November.
    9. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2011. "The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood," NBER Working Papers 17699, National Bureau of Economic Research, Inc.
    10. repec:mpr:mprres:7482 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Elias Walsh & Albert Y. Liu & Dallas Dotter, "undated". "Measuring Teacher and School Value Added in Oklahoma, 2012-2013 School Year," Mathematica Policy Research Reports 96a4bc02e74346c7895798a9c, Mathematica Policy Research.
    2. von Hippel, Paul T. & Bellows, Laura & Osborne, Cynthia & Lincove, Jane Arnold & Mills, Nick, 2016. "Teacher quality differences between teacher preparation programs: How big? How reliable? Which programs are different?," Economics of Education Review, Elsevier, vol. 53(C), pages 31-45.
    3. repec:mpr:mprres:7817 is not listed on IDEAS
    4. Cory Koedel & Jiaxi Li, 2016. "The Efficiency Implications Of Using Proportional Evaluations To Shape The Teaching Workforce," Contemporary Economic Policy, Western Economic Association International, vol. 34(1), pages 47-62, January.
    5. Eric Isenberg & Elias Walsh, 2014. "Measuring Teacher Value Added in DC, 2012-2013 School Year," Mathematica Policy Research Reports b319ed849495477791cef8b8c, Mathematica Policy Research.
    6. Elias Walsh & Stephen Lipscomb, "undated". "Classroom Observations from Phase 2 of the Pennsylvania Teacher Evaluation Pilot: Assessing Internal Consistency, Score Variation, and Relationships with Value Added," Mathematica Policy Research Reports a6b29a4a217f42a09d5206cfe, Mathematica Policy Research.
    7. Elias Walsh & Albert Y. Liu & Dallas Dotter, "undated". "Measuring Teacher and School Value Added in Oklahoma, 2013-2014 School Year," Mathematica Policy Research Reports fa9f4bb2356545758516a7c89, Mathematica Policy Research.
    8. Koedel, Cory & Mihaly, Kata & Rockoff, Jonah E., 2015. "Value-added modeling: A review," Economics of Education Review, Elsevier, vol. 47(C), pages 180-195.
    9. repec:mpr:mprres:7916 is not listed on IDEAS

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:mpr:mprres:7748 is not listed on IDEAS
    2. Stacy, Brian & Guarino, Cassandra & Wooldridge, Jeffrey, 2018. "Does the precision and stability of value-added estimates of teacher performance depend on the types of students they serve?," Economics of Education Review, Elsevier, vol. 64(C), pages 50-74.
    3. Eric Parsons & Cory Koedel & Li Tan, 2019. "Accounting for Student Disadvantage in Value-Added Models," Journal of Educational and Behavioral Statistics, , vol. 44(2), pages 144-179, April.
    4. Eric Isenberg & Heinrich Hock, "undated". "Measuring School and Teacher Value Added in DC, 2011-2012 School Year," Mathematica Policy Research Reports 429d075abf05480eb0fbb4831, Mathematica Policy Research.
    5. Cassandra M. Guarino & Mark D. Reckase & Jeffrey M. Woolrdige, 2014. "Can Value-Added Measures of Teacher Performance Be Trusted?," Education Finance and Policy, MIT Press, vol. 10(1), pages 117-156, November.
    6. Koedel Cory & Leatherman Rebecca & Parsons Eric, 2012. "Test Measurement Error and Inference from Value-Added Models," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 12(1), pages 1-37, November.
    7. Steven Dieterle & Cassandra M. Guarino & Mark D. Reckase & Jeffrey M. Wooldridge, 2015. "How do Principals Assign Students to Teachers? Finding Evidence in Administrative Data and the Implications for Value Added," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 34(1), pages 32-58, January.
    8. Allison Atteberry & Susanna Loeb & James Wyckoff, 2013. "Do First Impressions Matter? Improvement in Early Career Teacher Effectiveness," NBER Working Papers 19096, National Bureau of Economic Research, Inc.
    9. repec:mpr:mprres:8135 is not listed on IDEAS
    10. Koedel, Cory & Mihaly, Kata & Rockoff, Jonah E., 2015. "Value-added modeling: A review," Economics of Education Review, Elsevier, vol. 47(C), pages 180-195.
    11. Dana Rotz & Matthew Johnson & Brian Gill, "undated". "Value-Added Models for the Pittsburgh Public Schools, 2012-13 School Year," Mathematica Policy Research Reports 11cf6795ac3842b996637b08e, Mathematica Policy Research.
    12. Backes, Ben & Cowan, James & Goldhaber, Dan & Koedel, Cory & Miller, Luke C. & Xu, Zeyu, 2018. "The common core conundrum: To what extent should we worry that changes to assessments will affect test-based measures of teacher performance?," Economics of Education Review, Elsevier, vol. 62(C), pages 48-65.
    13. Eric Isenberg & Heinrich Hock, 2010. "Measuring School and Teacher Value Added for IMPACT and TEAM in DC Public Schools," Mathematica Policy Research Reports 42dd1ff2d7eb46948f98d8e9c, Mathematica Policy Research.
    14. Douglas O. Staiger & Jonah E. Rockoff, 2010. "Searching for Effective Teachers with Imperfect Information," Journal of Economic Perspectives, American Economic Association, vol. 24(3), pages 97-118, Summer.
    15. Goldhaber, Dan & Cowan, James & Walch, Joe, 2013. "Is a good elementary teacher always good? Assessing teacher performance estimates across subjects," Economics of Education Review, Elsevier, vol. 36(C), pages 216-228.
    16. Tim R. Sass, 2015. "Licensure and Worker Quality: A Comparison of Alternative Routes to Teaching," Journal of Law and Economics, University of Chicago Press, vol. 58(1), pages 1-35.
    17. Marc van der Steeg & Sander Gerritsen, 2013. "Teacher evaluations and pupil achievement: Evidence from classroom observations," CPB Discussion Paper 230, CPB Netherlands Bureau for Economic Policy Analysis.
    18. repec:mpr:mprres:7587 is not listed on IDEAS
    19. C. Kirabo Jackson, 2014. "Teacher Quality at the High School Level: The Importance of Accounting for Tracks," Journal of Labor Economics, University of Chicago Press, vol. 32(4), pages 645-684.
    20. Marc van der Steeg & Sander Gerritsen, 2013. "Teacher evaluations and pupil achievement: Evidence from classroom observations," CPB Discussion Paper 230.rdf, CPB Netherlands Bureau for Economic Policy Analysis.
    21. Richard K. Mansfield, 2015. "Teacher Quality and Student Inequality," Journal of Labor Economics, University of Chicago Press, vol. 33(3), pages 751-788.
    22. Victor Lavy & Rigissa Megalokonomou, 2024. "Alternative Measures of Teachers’ Value Added and Impact on Short and Long-Term Outcomes: Evidence From Random Assignment," Monash Economics Working Papers 2024-12, Monash University, Department of Economics.
    23. Cory Koedel & Mark Ehlert & Eric Parsons & Michael Podgursky, 2012. "Selecting Growth Measures for School and Teacher Evaluations," Working Papers 1210, Department of Economics, University of Missouri.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:mpr:mprres:2b140369be0242ac83eeb5b0afea95da. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joanne Pfleiderer or Cindy George The email address of this maintainer does not seem to be valid anymore. Please ask Cindy George to update the entry or send us the correct address (email available below). General contact details of provider: https://edirc.repec.org/data/mathius.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.