IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/22805.html
   My bibliography  Save this paper

The Compositional Effect of Rigorous Teacher Evaluation on Workforce Quality

Author

Listed:
  • Julie Berry Cullen
  • Cory Koedel
  • Eric Parsons

Abstract

Improving public sector workforce quality is challenging in sectors such as education where worker productivity is difficult to assess and manager incentives are muted by political and bureaucratic constraints. In this paper, we study how providing improved information to principals about teacher effectiveness and encouraging them to use the information in personnel decisions affects the composition of teacher turnovers. Our setting is the Houston Independent School District, which recently implemented a rigorous teacher evaluation system. Prior to the new system, teacher effectiveness was negatively correlated with district exit and we show that the policy significantly strengthened this relationship, primarily by increasing the relative likelihood of exit for teachers in the bottom quintile of the quality distribution. Low-performing teachers working in low-achieving schools were especially likely to leave. However, despite the success, the implied change to the quality of the workforce overall is too small to have a detectable impact on student achievement.

Suggested Citation

  • Julie Berry Cullen & Cory Koedel & Eric Parsons, 2016. "The Compositional Effect of Rigorous Teacher Evaluation on Workforce Quality," NBER Working Papers 22805, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:22805
    Note: ED LS PE
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w22805.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Markus Nagler & Marc Piopiunik & Martin R. West, 2020. "Weak Markets, Strong Teachers: Recession at Career Start and Teacher Effectiveness," Journal of Labor Economics, University of Chicago Press, vol. 38(2), pages 453-500.
    2. Tom Ahn & Jacob L. Vigdor, 2021. "When Incentives Matter Too Much: Explaining Significant Responses to Irrelevant Information," Journal of Human Capital, University of Chicago Press, vol. 15(4), pages 629-664.
    3. Harris, Douglas N. & Sass, Tim R., 2011. "Teacher training, teacher quality and student achievement," Journal of Public Economics, Elsevier, vol. 95(7-8), pages 798-812, August.
    4. David Blazar, 2018. "Validating Teacher Effects on Students’ Attitudes and Behaviors: Evidence from Random Assignment of Teachers to Students," Education Finance and Policy, MIT Press, vol. 13(3), pages 281-309, Summer.
    5. Roland G. Fryer, Jr., 2014. "Injecting Charter School Best Practices into Traditional Public Schools: Evidence from Field Experiments," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(3), pages 1355-1407.
    6. Lauren Sartain & Matthew P. Steinberg, 2016. "Teachers’ Labor Market Responses to Performance Evaluation Reform: Experimental Evidence from Chicago Public Schools," Journal of Human Resources, University of Wisconsin Press, vol. 51(3), pages 615-655.
    7. Daniel Aaronson & Lisa Barrow & William Sander, 2007. "Teachers and Student Achievement in the Chicago Public High Schools," Journal of Labor Economics, University of Chicago Press, vol. 25(1), pages 95-135.
    8. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    9. Dan Goldhaber & Betheny Gross & Daniel Player, 2011. "Teacher career paths, teacher quality, and persistence in the classroom: Are public schools keeping their best?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 30(1), pages 57-87, December.
    10. Scott A. Imberman & Michael F. Lovenheim, 2015. "Incentive Strength and Teacher Productivity: Evidence from a Group-Based Teacher Incentive Pay System," The Review of Economics and Statistics, MIT Press, vol. 97(2), pages 364-386, May.
    11. Erik Hanushek & F. Welch (ed.), 2006. "Handbook of the Economics of Education," Handbook of the Economics of Education, Elsevier, edition 1, volume 1, number 1, June.
    12. Hanushek, Eric A. & Rivkin, Steven G. & Schiman, Jeffrey C., 2016. "Dynamic effects of teacher turnover on the quality of instruction," Economics of Education Review, Elsevier, vol. 55(C), pages 132-148.
    13. Brehm, Margaret & Imberman, Scott A. & Lovenheim, Michael F., 2017. "Achievement effects of individual performance incentives in a teacher merit pay tournament," Labour Economics, Elsevier, vol. 44(C), pages 133-150.
    14. Dale Ballou & William Sanders & Paul Wright, 2004. "Controlling for Student Background in Value-Added Assessment of Teachers," Journal of Educational and Behavioral Statistics, , vol. 29(1), pages 37-65, March.
    15. Jesse Rothstein, 2015. "Teacher Quality Policy When Supply Matters," American Economic Review, American Economic Association, vol. 105(1), pages 100-130, January.
    16. Kristin F. Butcher & Patrick J. McEwan & Akila Weerapana, 2014. "The Effects of an Anti-grade-Inflation Policy at Wellesley College," Journal of Economic Perspectives, American Economic Association, vol. 28(3), pages 189-204, Summer.
    17. Andrew Bacher-Hicks & Thomas J. Kane & Douglas O. Staiger, 2014. "Validating Teacher Effect Estimates Using Changes in Teacher Assignments in Los Angeles," NBER Working Papers 20657, National Bureau of Economic Research, Inc.
    18. David Greenberg & John McCall, 1974. "Teacher Mobility and Allocation," Journal of Human Resources, University of Wisconsin Press, vol. 9(4), pages 480-502.
    19. Thomas S. Dee & James Wyckoff, 2015. "Incentives, Selection, and Teacher Performance: Evidence from IMPACT," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 34(2), pages 267-297, March.
    20. Hanushek, Eric A., 2011. "The economic value of higher teacher quality," Economics of Education Review, Elsevier, vol. 30(3), pages 466-479, June.
    21. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood," American Economic Review, American Economic Association, vol. 104(9), pages 2633-2679, September.
    22. Thomas J. Kane & Eric S. Taylor & John H. Tyler & Amy L. Wooten, 2011. "Identifying Effective Classroom Practices Using Student Achievement Data," Journal of Human Resources, University of Wisconsin Press, vol. 46(3), pages 587-613.
    23. Brian A. Jacob & Lars Lefgren, 2008. "Can Principals Identify Effective Teachers? Evidence on Subjective Performance Evaluation in Education," Journal of Labor Economics, University of Chicago Press, vol. 26(1), pages 101-136.
    24. Matthew A. Kraft, 2019. "Teacher Effects on Complex Cognitive Skills and Social-Emotional Competencies," Journal of Human Resources, University of Wisconsin Press, vol. 54(1), pages 1-36.
    25. Erik Hanushek & F. Welch (ed.), 2006. "Handbook of the Economics of Education," Handbook of the Economics of Education, Elsevier, edition 1, volume 2, number 2, June.
    26. Li Feng & Tim R. Sass, 2017. "Teacher Quality and Teacher Mobility," Education Finance and Policy, MIT Press, vol. 12(3), pages 396-418, Summer.
    27. C. Kirabo Jackson, 2018. "What Do Test Scores Miss? The Importance of Teacher Effects on Non–Test Score Outcomes," Journal of Political Economy, University of Chicago Press, vol. 126(5), pages 2072-2107.
    28. Jonah E. Rockoff & Douglas O. Staiger & Thomas J. Kane & Eric S. Taylor, 2012. "Information and Employee Evaluation: Evidence from a Randomized Intervention in Public Schools," American Economic Review, American Economic Association, vol. 102(7), pages 3184-3213, December.
    29. Gregory F. Branch & Eric A. Hanushek & Steven G. Rivkin, 2012. "Estimating the Effect of Leaders on Public Sector Productivity: The Case of School Principals," NBER Working Papers 17803, National Bureau of Economic Research, Inc.
    30. Backes, Ben & Cowan, James & Goldhaber, Dan & Koedel, Cory & Miller, Luke C. & Xu, Zeyu, 2018. "The common core conundrum: To what extent should we worry that changes to assessments will affect test-based measures of teacher performance?," Economics of Education Review, Elsevier, vol. 62(C), pages 48-65.
    31. Douglas O. Staiger & Jonah E. Rockoff, 2010. "Searching for Effective Teachers with Imperfect Information," Journal of Economic Perspectives, American Economic Association, vol. 24(3), pages 97-118, Summer.
    32. Eric A. Hanushek & Steven G. Rivkin, 2010. "Generalizations about Using Value-Added Measures of Teacher Quality," American Economic Review, American Economic Association, vol. 100(2), pages 267-271, May.
    33. Michael Bates, 2016. "Public and Private Learning in the Market for Teachers: Evidence from the Adoption of Value-Added Measures," Working Papers 201616, University of California at Riverside, Department of Economics.
    34. Jason A. Grissom & Michelle Reininger, 2012. "Who Comes Back? A Longitudinal Analysis of the Reentry Behavior of Exiting Teachers," Education Finance and Policy, MIT Press, vol. 7(4), pages 425-454, September.
    35. Melinda Adnot & Thomas Dee & Veronica Katz & James Wyckoff, 2016. "Teacher Turnover, Teacher Quality, and Student Achievement in DCPS," NBER Working Papers 21922, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Liebowitz, David D., 2021. "Teacher evaluation for accountability and growth: Should policy treat them as complements or substitutes?," Labour Economics, Elsevier, vol. 71(C).
    2. Bobby Chung & Jian Zou, 2021. "Teacher Licensing, Teacher Supply, and Student Achievement: Nationwide Implementation of edTPA," Working Papers 2021-039, Human Capital and Economic Opportunity Working Group.
    3. Eric Parsons & Cory Koedel & Li Tan, 2019. "Accounting for Student Disadvantage in Value-Added Models," Journal of Educational and Behavioral Statistics, , vol. 44(2), pages 144-179, April.
    4. Kraft, Matthew A. & Brunner, Eric J. & Dougherty, Shaun M. & Schwegman, David J., 2020. "Teacher accountability reforms and the supply and quality of new teachers," Journal of Public Economics, Elsevier, vol. 188(C).
    5. Cory Koedel & Jiaxi Li & Matthew G. Springer & Li Tan, 2018. "Teacher Performance Ratings and Professional Improvement," Working Papers 1808, Department of Economics, University of Missouri.
    6. Bergman, Peter & Hill, Matthew J., 2018. "The effects of making performance information public: Regression discontinuity evidence from Los Angeles teachers," Economics of Education Review, Elsevier, vol. 66(C), pages 104-113.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Johnston, Andrew C., 2021. "Preferences, Selection, and the Structure of Teacher Pay," IZA Discussion Papers 14831, Institute of Labor Economics (IZA).
    2. Gershenson, Seth, 2021. "Identifying and Producing Effective Teachers," IZA Discussion Papers 14096, Institute of Labor Economics (IZA).
    3. Koedel, Cory & Mihaly, Kata & Rockoff, Jonah E., 2015. "Value-added modeling: A review," Economics of Education Review, Elsevier, vol. 47(C), pages 180-195.
    4. Dhushyanth Raju, 2017. "Public School Teacher Management in Sri Lanka," South Asia Economic Journal, Institute of Policy Studies of Sri Lanka, vol. 18(1), pages 39-63, March.
    5. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.
    6. Michael Dinerstein & Isaac M. Opper, 2022. "Screening with Multitasking," CESifo Working Paper Series 9869, CESifo.
    7. Hanushek, Eric A. & Rivkin, Steven G. & Schiman, Jeffrey C., 2016. "Dynamic effects of teacher turnover on the quality of instruction," Economics of Education Review, Elsevier, vol. 55(C), pages 132-148.
    8. Asma Benhenda, 2018. "Teacher Screening, On the Job Evaluations and Performancee," DoQSS Working Papers 18-06, Quantitative Social Science - UCL Social Research Institute, University College London.
    9. Naven, Matthew, 2019. "Human-Capital Formation During Childhood and Adolescence: Evidence from School Quality and Postsecondary Success in California," MPRA Paper 97716, University Library of Munich, Germany.
    10. Sean Corcoran & Dan Goldhaber, 2013. "Value Added and Its Uses: Where You Stand Depends on Where You Sit," Education Finance and Policy, MIT Press, vol. 8(3), pages 418-434, July.
    11. Simon Briole, 2019. "From Teacher Quality to Teaching Quality: Instructional Productivity and Teaching Practices in the US," Working Papers halshs-01993616, HAL.
    12. Cory Koedel & Jiaxi Li & Matthew G. Springer & Li Tan, 2016. "The Impact of Performance Ratings on Job Satisfaction for Public School Teachers," Working Papers 1617, Department of Economics, University of Missouri.
    13. Hinrichs, Peter, 2021. "What kind of teachers are schools looking for? Evidence from a randomized field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 186(C), pages 395-411.
    14. repec:hal:spmain:info:hdl:2441/1jgbspo1909q48svne93o55rca is not listed on IDEAS
    15. Marc Steeg & Sander Gerritsen, 2016. "Teacher Evaluations and Pupil Achievement Gains: Evidence from Classroom Observations," De Economist, Springer, vol. 164(4), pages 419-443, December.
    16. Alexis Le Chapelain, 2014. "Market for Education and Student Achievement," Sciences Po publications info:hdl:2441/1jgbspo1909, Sciences Po.
    17. Jacob, Brian A. & Rockoff, Jonah E. & Taylor, Eric S. & Lindy, Benjamin & Rosen, Rachel, 2018. "Teacher applicant hiring and teacher performance: Evidence from DC public schools," Journal of Public Economics, Elsevier, vol. 166(C), pages 81-97.
    18. Brian Jacob & Jonah E. Rockoff & Eric S. Taylor & Benjamin Lindy & Rachel Rosen, 2016. "Teacher Applicant Hiring and Teacher Performance: Evidence from DC Public Schools," NBER Working Papers 22054, National Bureau of Economic Research, Inc.
    19. Pope, Nolan G., 2019. "The effect of teacher ratings on teacher performance," Journal of Public Economics, Elsevier, vol. 172(C), pages 84-110.
    20. Berlinski, Samuel & Ramos, Alejandra, 2020. "Teacher mobility and merit pay: Evidence from a voluntary public award program," Journal of Public Economics, Elsevier, vol. 186(C).
    21. Cory Koedel & Eric Parsons & Michael Podgursky & Mark Ehlert, 2015. "Teacher Preparation Programs and Teacher Quality: Are There Real Differences Across Programs?," Education Finance and Policy, MIT Press, vol. 10(4), pages 508-534, October.

    More about this item

    JEL classification:

    • H75 - Public Economics - - State and Local Government; Intergovernmental Relations - - - State and Local Government: Health, Education, and Welfare
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy
    • J45 - Labor and Demographic Economics - - Particular Labor Markets - - - Public Sector Labor Markets

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:22805. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.