IDEAS home Printed from https://ideas.repec.org/p/ces/ceswps/_9263.html
   My bibliography  Save this paper

Measuring and Summarizing the Multiple Dimensions of Teacher Effectiveness

Author

Listed:
  • Christine Mulhern
  • Isaac M. Opper

Abstract

There is an emerging consensus that teachers impact multiple student outcomes, but it remains unclear how to summarize these multiple dimensions of teacher effectiveness into simple metrics that can be used for research or personnel decisions. Here, we discuss the implications of estimating teacher effects in a multidimensional empirical Bayes framework and illustrate how to appropriately use these noisy estimates to assess the dimensionality and predictive power of the true teacher effects. Empirically, our principal components analysis indicates that the multiple dimensions can be efficiently summarized by a small number of measures; for example, one dimension explains over half the variation in the teacher effects on all the dimensions we observe. Summary measures based on the first principal component lead to similar rankings of teachers as summary measures weighting short-term effects by their prediction of long-term outcomes. We conclude by discussing the practical implications of using summary measures of effectiveness and, specifically, how to ensure that the policy implementation is fair when different sets of measures are observed for different teachers.

Suggested Citation

  • Christine Mulhern & Isaac M. Opper, 2021. "Measuring and Summarizing the Multiple Dimensions of Teacher Effectiveness," CESifo Working Paper Series 9263, CESifo.
    • Handle: RePEc:ces:ceswps:_9263
    as

    Download full text from publisher

    File URL: https://www.ifo.de/DocDL/cesifo1_wp9263.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Joshua D. Angrist & Peter Hull & Parag Pathak & Christopher Walters, 2020. "Simple and Credible Value-Added Estimation Using Centralized School Assignment," Working Papers 2020-186, Becker Friedman Institute for Research In Economics.
    2. Thomas J. Kane & Douglas O. Staiger, 2008. "Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation," NBER Working Papers 14607, National Bureau of Economic Research, Inc.
    3. Amemiya, Yasuo, 1985. "Instrumental variable estimator for the nonlinear errors-in-variables model," Journal of Econometrics, Elsevier, vol. 28(3), pages 273-289, June.
    4. Seth Gershenson & Diane Whitmore Schanzenbach, 2016. "Linking Teacher Quality, Student Attendance, and Student Achievement," Education Finance and Policy, MIT Press, vol. 11(2), pages 125-149, Spring.
    5. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    6. Yingyao Hu & Susanne M. Schennach, 2008. "Instrumental Variable Treatment of Nonclassical Measurement Error Models," Econometrica, Econometric Society, vol. 76(1), pages 195-216, January.
    7. Joshua D. Angrist & Peter D. Hull & Parag A. Pathak & Christopher R. Walters, 2017. "Leveraging Lotteries for School Value-Added: Testing and Estimation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(2), pages 871-919.
    8. Hausman, Jerry A. & Newey, Whitney K. & Ichimura, Hidehiko & Powell, James L., 1991. "Identification and estimation of polynomial errors-in-variables models," Journal of Econometrics, Elsevier, vol. 50(3), pages 273-295, December.
    9. Michael Dinerstein & Rigissa Megalokonomou & Constantine Yannelis, 2020. "Human Capital Depreciation," Working Papers 2020-146, Becker Friedman Institute for Research In Economics.
    10. Seth Gershenson & Cassandra M. D. Hart & Joshua Hyman & Constance A. Lindsay & Nicholas W. Papageorge, 2022. "The Long-Run Impacts of Same-Race Teachers," American Economic Journal: Economic Policy, American Economic Association, vol. 14(4), pages 300-342, November.
    11. Thomas S. Dee & James Wyckoff, 2015. "Incentives, Selection, and Teacher Performance: Evidence from IMPACT," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 34(2), pages 267-297, March.
    12. Brian A. Jacob & Lars Lefgren, 2008. "Can Principals Identify Effective Teachers? Evidence on Subjective Performance Evaluation in Education," Journal of Labor Economics, University of Chicago Press, vol. 26(1), pages 101-136.
    13. Matthew A. Kraft, 2019. "Teacher Effects on Complex Cognitive Skills and Social-Emotional Competencies," Journal of Human Resources, University of Wisconsin Press, vol. 54(1), pages 1-36.
    14. Joshua D. Angrist & Peter D. Hull & Parag A. Pathak & Christopher R. Walters, 2017. "Leveraging Lotteries for School Value-Added: Testing and Estimation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(2), pages 871-919.
    15. Hugh Macartney, 2016. "The Dynamic Effects of Educational Accountability," Journal of Labor Economics, University of Chicago Press, vol. 34(1), pages 1-28.
    16. Bacher-Hicks, Andrew & Chin, Mark J. & Kane, Thomas J. & Staiger, Douglas O., 2019. "An experimental evaluation of three teacher quality measures: Value-added, classroom observations, and student surveys," Economics of Education Review, Elsevier, vol. 73(C).
    17. Raj Chetty & Nathaniel Hendren, 2018. "The Impacts of Neighborhoods on Intergenerational Mobility II: County-Level Estimates," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(3), pages 1163-1228.
    18. Arthur Lewbel, 1998. "Semiparametric Latent Variable Model Estimation with Endogenous or Mismeasured Regressors," Econometrica, Econometric Society, vol. 66(1), pages 105-122, January.
    19. Helen F. Ladd & Lucy C. Sorensen, 2017. "Returns to Teacher Experience: Student Achievement and Motivation in Middle School," Education Finance and Policy, MIT Press, vol. 12(2), pages 241-279, Spring.
    20. C. Kirabo Jackson, 2018. "What Do Test Scores Miss? The Importance of Teacher Effects on Non–Test Score Outcomes," Journal of Political Economy, University of Chicago Press, vol. 126(5), pages 2072-2107.
    21. Jonah E. Rockoff & Douglas O. Staiger & Thomas J. Kane & Eric S. Taylor, 2012. "Information and Employee Evaluation: Evidence from a Randomized Intervention in Public Schools," American Economic Review, American Economic Association, vol. 102(7), pages 3184-3213, December.
    22. C. B. Begg & D. H. Y. Leung, 2000. "On the use of surrogate end points in randomized trials," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 163(1), pages 15-28.
    23. Michael Gilraine & Jiaying Gu & Robert McMillan, 2020. "A New Method for Estimating Teacher Value-Added," NBER Working Papers 27094, National Bureau of Economic Research, Inc.
    24. John P. Papay & Eric S. Taylor & John H. Tyler & Mary E. Laski, 2020. "Learning Job Skills from Colleagues at Work: Evidence from a Field Experiment Using Teacher Performance Data," American Economic Journal: Economic Policy, American Economic Association, vol. 12(1), pages 359-388, February.
    25. Hong, Han & Tamer, Elie, 2003. "A simple estimator for nonlinear error in variable models," Journal of Econometrics, Elsevier, vol. 117(1), pages 1-19, November.
    26. Isaac M. Opper, 2019. "Does Helping John Help Sue? Evidence of Spillovers in Education," American Economic Review, American Economic Association, vol. 109(3), pages 1080-1115, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Michael Dinerstein & Isaac M. Opper, 2022. "Screening with Multitasking," CESifo Working Paper Series 9869, CESifo.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gershenson, Seth, 2021. "Identifying and Producing Effective Teachers," IZA Discussion Papers 14096, Institute of Labor Economics (IZA).
    2. Liebowitz, David D., 2021. "Teacher evaluation for accountability and growth: Should policy treat them as complements or substitutes?," Labour Economics, Elsevier, vol. 71(C).
    3. Antoine Deeb, 2021. "A Framework for Using Value-Added in Regressions," Papers 2109.01741, arXiv.org, revised Oct 2021.
    4. Michael Dinerstein & Isaac M. Opper, 2022. "Screening with Multitasking," CESifo Working Paper Series 9869, CESifo.
    5. Tian Xie, 2025. "Automatic Inference for Value-Added Regressions," Papers 2503.19178, arXiv.org.
    6. Julie Berry Cullen & Cory Koedel & Eric Parsons, 2021. "The Compositional Effect of Rigorous Teacher Evaluation on Workforce Quality," Education Finance and Policy, MIT Press, vol. 16(1), pages 7-41, Winter.
    7. Mookerjee, Sulagna & Slichter, David, 2023. "Test scores, schools, and the geography of economic opportunity," Journal of Urban Economics, Elsevier, vol. 137(C).
    8. Timothy B. Armstrong & Michal Kolesár & Mikkel Plagborg‐Møller, 2022. "Robust Empirical Bayes Confidence Intervals," Econometrica, Econometric Society, vol. 90(6), pages 2567-2602, November.
    9. Bruhn, Jesse & Imberman, Scott & Winters, Marcus, 2022. "Regulatory arbitrage in teacher hiring and retention: Evidence from Massachusetts Charter Schools," Journal of Public Economics, Elsevier, vol. 215(C).
    10. Timothy B. Armstrong & Michal Koles'ar & Mikkel Plagborg-M{o}ller, 2020. "Robust Empirical Bayes Confidence Intervals," Papers 2004.03448, arXiv.org, revised May 2022.
    11. Nirav Mehta, 2019. "Measuring quality for use in incentive schemes: The case of “shrinkage” estimators," Quantitative Economics, Econometric Society, vol. 10(4), pages 1537-1577, November.
    12. Matthew A. Kraft & John P. Papay & Olivia L. Chi, 2020. "Teacher Skill Development: Evidence from Performance Ratings by Principals," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 315-347, March.
    13. Naven, Matthew, 2020. "Within-School Heterogeneity in Quality: Do Schools Provide Equal Value Added to All Students?," MPRA Paper 100123, University Library of Munich, Germany.
    14. Susanna Loeb & Michael S. Christian & Heather Hough & Robert H. Meyer & Andrew B. Rice & Martin R. West, 2019. "School Differences in Social–Emotional Learning Gains: Findings From the First Large-Scale Panel Survey of Students," Journal of Educational and Behavioral Statistics, , vol. 44(5), pages 507-542, October.
    15. Hayes, Michael S. & Liu, Jing & Gershenson, Seth, 2023. "Who refers whom? The effects of teacher characteristics on disciplinary office referrals," Economics of Education Review, Elsevier, vol. 93(C).
    16. Robert Ainsworth & Rajeev Dehejia & Cristian Pop-Eleches & Miguel Urquiola, 2023. "Why Do Households Leave School Value Added on the Table? The Roles of Information and Preferences," American Economic Review, American Economic Association, vol. 113(4), pages 1049-1082, April.
    17. Hermann, Zoltán & Horváth, Hedvig, 2022. "Tanári eredményesség és tanár-diák összepárosítás az általános iskolákban. Empirikus mintázatok három magyarországi tankerület adatai alapján [Teacher effectiveness and teacher-student matching in ," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(11), pages 1377-1406.
    18. Stéphane Bonhomme & Martin Weidner, 2019. "Posterior average effects," CeMMAP working papers CWP43/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    19. Lars J. Kirkebøen, 2021. "School value-added and longterm student outcomes," Discussion Papers 970, Statistics Norway, Research Department.
    20. Bär, Marlies & Bakx, Pieter & Wouterse, Bram & van Doorslaer, Eddy, 2022. "Estimating the health value added by nursing homes," Journal of Economic Behavior & Organization, Elsevier, vol. 203(C), pages 1-23.

    More about this item

    Keywords

    ;
    ;

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • C10 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - General
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity
    • M50 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Personnel Economics - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ceswps:_9263. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Klaus Wohlrabe (email available below). General contact details of provider: https://edirc.repec.org/data/cesifde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.