IDEAS home Printed from https://ideas.repec.org/a/bpj/statpp/v3y2012i2p24n1.html
   My bibliography  Save this article

Teacher Effect Estimates and Decision Rules for Establishing Student-Teacher Linkages: What are the Implications for High-Stakes Personnel Policies in an Urban School District?

Author

Listed:
  • Han Bing

    (RAND Corporation)

  • McCaffrey Daniel

    (RAND Corporation)

  • Springer Matthew G.

    (Vanderbilt University)

  • Gottfried Michael

    (RAND Corporation and Loyola Marymount University)

Abstract

Both research and practice of value-added models (VAM) have been growing in recent years due to the widespread effort to quantify teacher effectiveness. The existing VAM literature has not yet tested the sensitivity of value-added estimates to the rules that define which students contribute to each teacher’s value-added estimate. Student-teacher linkages are often a complex network due to various transfers, students taking multiple courses in the same subject, and students receiving special education or other “pull-out” services. Complex linkages are often considered as among the main threats to the validity of VAM. In this paper we conducted a case study to examine the sensitivity of VAM to the alternative link definitions. We examined three popular VAM approaches and applied alternative rules for linking students to teachers with each method. We found no overall sensitivity of estimated teacher effects to the linking rules. Even though more teachers had value-added estimates under more inclusive rules, for a teacher with estimates under all three rules, the correlation among pairs of estimates created using different linking rules was always above 0.95 and generally above 0.98 for each VAM approach. The value-added estimates of a small number of teachers were affected by highly different link definitions and these tended to be teachers with small numbers of students. Restricting the minimal sample size for calculating individual teachers’ value-added largely reduced the sensitivity in link definition.

Suggested Citation

  • Han Bing & McCaffrey Daniel & Springer Matthew G. & Gottfried Michael, 2012. "Teacher Effect Estimates and Decision Rules for Establishing Student-Teacher Linkages: What are the Implications for High-Stakes Personnel Policies in an Urban School District?," Statistics, Politics and Policy, De Gruyter, vol. 3(2), pages 1-24, July.
    • Handle: RePEc:bpj:statpp:v:3:y:2012:i:2:p:24:n:1
    DOI: 10.1515/2151-7509.1035
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/2151-7509.1035
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.
    File URL: https://libkey.io/10.1515/2151-7509.1035?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Steven G. Rivkin & Eric A. Hanushek & John F. Kain, 2005. "Teachers, Schools, and Academic Achievement," Econometrica, Econometric Society, vol. 73(2), pages 417-458, March.
    2. Jesse Rothstein, 2010. "Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 175-214.
    3. Daniel F. McCaffrey & Tim R. Sass & J. R. Lockwood & Kata Mihaly, 2009. "The Intertemporal Variability of Teacher Effect Estimates," Education Finance and Policy, MIT Press, vol. 4(4), pages 572-606, October.
    4. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    5. Donald Boyd & Pamela Grossman & Hamilton Lankford & Susanna Loeb & James Wyckoff, 2006. "How Changes in Entry Requirements Alter the Teacher Workforce and Affect Student Achievement," Education Finance and Policy, MIT Press, vol. 1(2), pages 176-216, April.
    6. J. R. Lockwood & Daniel F. McCaffrey & Louis T. Mariano & Claude Setodji, 2007. "Bayesian Methods for Scalable Multivariate Value-Added Assessment," Journal of Educational and Behavioral Statistics, , vol. 32(2), pages 125-150, June.
    7. Douglas O. Staiger & Jonah E. Rockoff, 2010. "Searching for Effective Teachers with Imperfect Information," Journal of Economic Perspectives, American Economic Association, vol. 24(3), pages 97-118, Summer.
    8. Brian A. Jacob & Lars Lefgren, 2005. "Principals as Agents: Subjective Performance Measurement in Education," NBER Working Papers 11463, National Bureau of Economic Research, Inc.
    9. Peter Z. Schochet & Hanley S. Chiang, "undated". "Error Rates in Measuring Teacher and School Performance Based on Student Test Score Gains," Mathematica Policy Research Reports d415285f980b4d64b7e75f40b, Mathematica Policy Research.
    10. repec:mpr:mprres:6721 is not listed on IDEAS
    11. Jacob, Brian A. & Lefgren, Lars, 2005. "Principals as Agents: Subjective Performance Measures in Education," Working Paper Series rwp05-040, Harvard University, John F. Kennedy School of Government.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sean Corcoran & Dan Goldhaber, 2013. "Value Added and Its Uses: Where You Stand Depends on Where You Sit," Education Finance and Policy, MIT Press, vol. 8(3), pages 418-434, July.
    2. Hanley Chiang & Stephen Lipscomb & Brian Gill, 2014. "Is School Value Added Indicative of Principal Quality?," Mathematica Policy Research Reports a11ab111ac3a497885e0a736f, Mathematica Policy Research.
    3. Hanley Chiang & Stephen Lipscomb & Brian Gill, 2016. "Is School Value Added Indicative of Principal Quality?," Education Finance and Policy, MIT Press, vol. 11(3), pages 283-309, Summer.
    4. Dan Goldhaber & Roddy Theobald, 2013. "Managing the Teacher Workforce in Austere Times: The Determinants and Implications of Teacher Layoffs," Education Finance and Policy, MIT Press, vol. 8(4), pages 494-527, October.
    5. Dhushyanth Raju, 2017. "Public School Teacher Management in Sri Lanka," South Asia Economic Journal, Institute of Policy Studies of Sri Lanka, vol. 18(1), pages 39-63, March.
    6. M. Caridad Araujo & Pedro Carneiro & Yyannú Cruz-Aguayo & Norbert Schady, 2016. "Teacher Quality and Learning Outcomes in Kindergarten," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(3), pages 1415-1453.
    7. Hanushek, Eric A. & Rivkin, Steven G. & Schiman, Jeffrey C., 2016. "Dynamic effects of teacher turnover on the quality of instruction," Economics of Education Review, Elsevier, vol. 55(C), pages 132-148.
    8. Hanushek, Eric A., 2011. "The economic value of higher teacher quality," Economics of Education Review, Elsevier, vol. 30(3), pages 466-479, June.
    9. Naven, Matthew, 2019. "Human-Capital Formation During Childhood and Adolescence: Evidence from School Quality and Postsecondary Success in California," MPRA Paper 97716, University Library of Munich, Germany.
    10. Dan Goldhaber & Michael Hansen, 2013. "Is it Just a Bad Class? Assessing the Long-term Stability of Estimated Teacher Performance," Economica, London School of Economics and Political Science, vol. 80(319), pages 589-612, July.
    11. David Blazar, 2018. "Validating Teacher Effects on Students’ Attitudes and Behaviors: Evidence from Random Assignment of Teachers to Students," Education Finance and Policy, MIT Press, vol. 13(3), pages 281-309, Summer.
    12. Gregory F. Branch & Eric A. Hanushek & Steven G. Rivkin, 2012. "Estimating the Effect of Leaders on Public Sector Productivity: The Case of School Principals," NBER Working Papers 17803, National Bureau of Economic Research, Inc.
    13. Eric Hanushek & Steven Rivkin, "undated". "Constrained Job Matching: Does Teacher Job Search Harm Disadvantaged Urban Schools?," Discussion Papers 09-011, Stanford Institute for Economic Policy Research.
    14. Harris, Douglas N. & Sass, Tim R., 2014. "Skills, productivity and the evaluation of teacher performance," Economics of Education Review, Elsevier, vol. 40(C), pages 183-204.
    15. Josh Kinsler, 2016. "Teacher Complementarities in Test Score Production: Evidence from Primary School," Journal of Labor Economics, University of Chicago Press, vol. 34(1), pages 29-61.
    16. Kevin C. Bastian & Gary T. Henry & Charles L. Thompson, 2013. "Incorporating Access to More Effective Teachers into Assessments of Educational Resource Equity," Education Finance and Policy, MIT Press, vol. 8(4), pages 560-580, October.
    17. Goel, Deepti & Barooah, Bidisha, 2018. "Drivers of Student Performance: Evidence from Higher Secondary Public Schools in Delhi," GLO Discussion Paper Series 231, Global Labor Organization (GLO).
    18. Cordero, Jose M. & Gil-Izquierdo, María, 2018. "The effect of teaching strategies on student achievement: An analysis using TALIS-PISA-link," Journal of Policy Modeling, Elsevier, vol. 40(6), pages 1313-1331.
    19. Allison Atteberry & Susanna Loeb & James Wyckoff, 2013. "Do First Impressions Matter? Improvement in Early Career Teacher Effectiveness," NBER Working Papers 19096, National Bureau of Economic Research, Inc.
    20. Canales, Andrea & Maldonado, Luis, 2018. "Teacher quality and student achievement in Chile: Linking teachers' contribution and observable characteristics," International Journal of Educational Development, Elsevier, vol. 60(C), pages 33-50.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:statpp:v:3:y:2012:i:2:p:24:n:1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.