IDEAS home Printed from https://ideas.repec.org/p/ces/ceswps/_9263.html
   My bibliography  Save this paper

Measuring and Summarizing the Multiple Dimensions of Teacher Effectiveness

Author

Listed:
  • Christine Mulhern
  • Isaac M. Opper

Abstract

There is an emerging consensus that teachers impact multiple student outcomes, but it remains unclear how to summarize these multiple dimensions of teacher effectiveness into simple metrics that can be used for research or personnel decisions. Here, we discuss the implications of estimating teacher effects in a multidimensional empirical Bayes framework and illustrate how to appropriately use these noisy estimates to assess the dimensionality and predictive power of the true teacher effects. Empirically, our principal components analysis indicates that the multiple dimensions can be efficiently summarized by a small number of measures; for example, one dimension explains over half the variation in the teacher effects on all the dimensions we observe. Summary measures based on the first principal component lead to similar rankings of teachers as summary measures weighting short-term effects by their prediction of long-term outcomes. We conclude by discussing the practical implications of using summary measures of effectiveness and, specifically, how to ensure that the policy implementation is fair when different sets of measures are observed for different teachers.

Suggested Citation

  • Christine Mulhern & Isaac M. Opper, 2021. "Measuring and Summarizing the Multiple Dimensions of Teacher Effectiveness," CESifo Working Paper Series 9263, CESifo.
    • Handle: RePEc:ces:ceswps:_9263
    as

    Download full text from publisher

    File URL: https://www.cesifo.org/DocDL/cesifo1_wp9263.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Hong, Han & Tamer, Elie, 2003. "A simple estimator for nonlinear error in variable models," Journal of Econometrics, Elsevier, vol. 117(1), pages 1-19, November.
    2. Raj Chetty & Nathaniel Hendren, 2018. "The Impacts of Neighborhoods on Intergenerational Mobility II: County-Level Estimates," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(3), pages 1163-1228.
    3. Michael Dinerstein & Rigissa Megalokonomou & Constantine Yannelis, 2020. "Human Capital Depreciation," Working Papers 2020-146, Becker Friedman Institute for Research In Economics.
    4. Jonah E. Rockoff & Douglas O. Staiger & Thomas J. Kane & Eric S. Taylor, 2012. "Information and Employee Evaluation: Evidence from a Randomized Intervention in Public Schools," American Economic Review, American Economic Association, vol. 102(7), pages 3184-3213, December.
    5. Seth Gershenson & Cassandra M. D. Hart & Joshua Hyman & Constance A. Lindsay & Nicholas W. Papageorge, 2022. "The Long-Run Impacts of Same-Race Teachers," American Economic Journal: Economic Policy, American Economic Association, vol. 14(4), pages 300-342, November.
    6. Joshua D. Angrist & Peter Hull & Parag Pathak & Christopher Walters, 2020. "Simple and Credible Value-Added Estimation Using Centralized School Assignment," Working Papers 2020-186, Becker Friedman Institute for Research In Economics.
    7. Thomas J. Kane & Douglas O. Staiger, 2008. "Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation," NBER Working Papers 14607, National Bureau of Economic Research, Inc.
    8. Thomas S. Dee & James Wyckoff, 2015. "Incentives, Selection, and Teacher Performance: Evidence from IMPACT," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 34(2), pages 267-297, March.
    9. Arthur Lewbel, 1998. "Semiparametric Latent Variable Model Estimation with Endogenous or Mismeasured Regressors," Econometrica, Econometric Society, vol. 66(1), pages 105-122, January.
    10. Joshua D. Angrist & Peter D. Hull & Parag A. Pathak & Christopher R. Walters, 2017. "Leveraging Lotteries for School Value-Added: Testing and Estimation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(2), pages 871-919.
    11. Amemiya, Yasuo, 1985. "Instrumental variable estimator for the nonlinear errors-in-variables model," Journal of Econometrics, Elsevier, vol. 28(3), pages 273-289, June.
    12. Seth Gershenson & Diane Whitmore Schanzenbach, 2016. "Linking Teacher Quality, Student Attendance, and Student Achievement," Education Finance and Policy, MIT Press, vol. 11(2), pages 125-149, Spring.
    13. Brian A. Jacob & Lars Lefgren, 2008. "Can Principals Identify Effective Teachers? Evidence on Subjective Performance Evaluation in Education," Journal of Labor Economics, University of Chicago Press, vol. 26(1), pages 101-136.
    14. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    15. Helen F. Ladd & Lucy C. Sorensen, 2017. "Returns to Teacher Experience: Student Achievement and Motivation in Middle School," Education Finance and Policy, MIT Press, vol. 12(2), pages 241-279, Spring.
    16. Yingyao Hu & Susanne M. Schennach, 2008. "Instrumental Variable Treatment of Nonclassical Measurement Error Models," Econometrica, Econometric Society, vol. 76(1), pages 195-216, January.
    17. C. Kirabo Jackson, 2018. "What Do Test Scores Miss? The Importance of Teacher Effects on Non–Test Score Outcomes," Journal of Political Economy, University of Chicago Press, vol. 126(5), pages 2072-2107.
    18. Matthew A. Kraft, 2019. "Teacher Effects on Complex Cognitive Skills and Social-Emotional Competencies," Journal of Human Resources, University of Wisconsin Press, vol. 54(1), pages 1-36.
    19. Hugh Macartney, 2016. "The Dynamic Effects of Educational Accountability," Journal of Labor Economics, University of Chicago Press, vol. 34(1), pages 1-28.
    20. C. B. Begg & D. H. Y. Leung, 2000. "On the use of surrogate end points in randomized trials," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 163(1), pages 15-28.
    21. Michael Gilraine & Jiaying Gu & Robert McMillan, 2020. "A New Method for Estimating Teacher Value-Added," NBER Working Papers 27094, National Bureau of Economic Research, Inc.
    22. Isaac M. Opper, 2019. "Does Helping John Help Sue? Evidence of Spillovers in Education," American Economic Review, American Economic Association, vol. 109(3), pages 1080-1115, March.
    23. John P. Papay & Eric S. Taylor & John H. Tyler & Mary E. Laski, 2020. "Learning Job Skills from Colleagues at Work: Evidence from a Field Experiment Using Teacher Performance Data," American Economic Journal: Economic Policy, American Economic Association, vol. 12(1), pages 359-388, February.
    24. Bacher-Hicks, Andrew & Chin, Mark J. & Kane, Thomas J. & Staiger, Douglas O., 2019. "An experimental evaluation of three teacher quality measures: Value-added, classroom observations, and student surveys," Economics of Education Review, Elsevier, vol. 73(C).
    25. Hausman, Jerry A. & Newey, Whitney K. & Ichimura, Hidehiko & Powell, James L., 1991. "Identification and estimation of polynomial errors-in-variables models," Journal of Econometrics, Elsevier, vol. 50(3), pages 273-295, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Michael Dinerstein & Isaac M. Opper, 2022. "Screening with Multitasking," CESifo Working Paper Series 9869, CESifo.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gershenson, Seth, 2021. "Identifying and Producing Effective Teachers," IZA Discussion Papers 14096, Institute of Labor Economics (IZA).
    2. Michael Dinerstein & Isaac M. Opper, 2022. "Screening with Multitasking," CESifo Working Paper Series 9869, CESifo.
    3. Liebowitz, David D., 2021. "Teacher evaluation for accountability and growth: Should policy treat them as complements or substitutes?," Labour Economics, Elsevier, vol. 71(C).
    4. Julie Berry Cullen & Cory Koedel & Eric Parsons, 2021. "The Compositional Effect of Rigorous Teacher Evaluation on Workforce Quality," Education Finance and Policy, MIT Press, vol. 16(1), pages 7-41, Winter.
    5. Antoine Deeb, 2021. "A Framework for Using Value-Added in Regressions," Papers 2109.01741, arXiv.org, revised Oct 2021.
    6. Matthew A. Kraft & John P. Papay & Olivia L. Chi, 2020. "Teacher Skill Development: Evidence from Performance Ratings by Principals," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 315-347, March.
    7. Timothy B. Armstrong & Michal Kolesár & Mikkel Plagborg‐Møller, 2022. "Robust Empirical Bayes Confidence Intervals," Econometrica, Econometric Society, vol. 90(6), pages 2567-2602, November.
    8. Bruhn, Jesse & Imberman, Scott & Winters, Marcus, 2022. "Regulatory arbitrage in teacher hiring and retention: Evidence from Massachusetts Charter Schools," Journal of Public Economics, Elsevier, vol. 215(C).
    9. Hayes, Michael S. & Liu, Jing & Gershenson, Seth, 2023. "Who refers whom? The effects of teacher characteristics on disciplinary office referrals," Economics of Education Review, Elsevier, vol. 93(C).
    10. Gottfried, Michael & Jacob Kirksey, J. & Hutt, Ethan, 2020. "Can teacher education programs help prepare new kindergarten and first grade teachers to address student absenteeism?," Children and Youth Services Review, Elsevier, vol. 119(C).
    11. Andrew Agopsowicz & Chris Robinson & Ralph Stinebrickner & Todd Stinebrickner, 2020. "Careers and Mismatch for College Graduates: College and Noncollege Jobs," Journal of Human Resources, University of Wisconsin Press, vol. 55(4), pages 1194-1221.
    12. Jose Maria Cabrera & Dinand Webbink, 2020. "Do Higher Salaries Yield Better Teachers and Better Student Outcomes?," Journal of Human Resources, University of Wisconsin Press, vol. 55(4), pages 1222-1257.
    13. Timothy B. Armstrong & Michal Koles'ar & Mikkel Plagborg-M{o}ller, 2020. "Robust Empirical Bayes Confidence Intervals," Papers 2004.03448, arXiv.org, revised May 2022.
    14. Nirav Mehta, 2019. "Measuring quality for use in incentive schemes: The case of “shrinkage” estimators," Quantitative Economics, Econometric Society, vol. 10(4), pages 1537-1577, November.
    15. Joan Martinez, 2022. "The Long-Term Effects of Teachers' Gender Stereotypes," Papers 2212.08220, arXiv.org, revised Jul 2023.
    16. Michael Bates & Michael Dinerstein & Andrew C. Johnston & Isaac Sorkin, 2022. "Teacher Labor Market Equilibrium and Student Achievement," CESifo Working Paper Series 9551, CESifo.
    17. Ingo E. Isphording & Ulf Zölitz, 2020. "The value of a peer," ECON - Working Papers 342, Department of Economics - University of Zurich.
    18. Stacy, Brian & Guarino, Cassandra & Wooldridge, Jeffrey, 2018. "Does the precision and stability of value-added estimates of teacher performance depend on the types of students they serve?," Economics of Education Review, Elsevier, vol. 64(C), pages 50-74.
    19. Goldhaber, Dan & Krieg, John & Theobald, Roddy, 2020. "Effective like me? Does having a more productive mentor improve the productivity of mentees?," Labour Economics, Elsevier, vol. 63(C).
    20. Koedel, Cory & Mihaly, Kata & Rockoff, Jonah E., 2015. "Value-added modeling: A review," Economics of Education Review, Elsevier, vol. 47(C), pages 180-195.

    More about this item

    Keywords

    value-added; teacher effectiveness;

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • C10 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - General
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity
    • M50 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Personnel Economics - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ceswps:_9263. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Klaus Wohlrabe (email available below). General contact details of provider: https://edirc.repec.org/data/cesifde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.