IDEAS home Printed from https://ideas.repec.org/a/aea/aecrev/v107y2017i6p1685-1717.html
   My bibliography  Save this article

Measuring the Impacts of Teachers: Reply

Author

Listed:
  • Raj Chetty
  • John N. Friedman
  • Jonah E. Rockoff

Abstract

Rothstein (2017) successfully replicates Chetty, Friedman, and Rockoff's (2014a, b)—henceforth, CFR's—results using data from North Carolina, but raises concerns about CFR's methods. We show that Rothstein's methodological critiques are invalid by presenting simulations and supplementary empirical evidence which show that (i) his preferred imputation of missing data generates bias; (ii) his "placebo test" rejects valid research designs; and (iii) his method of controlling for covariates yields inconsistent estimates of teachers' long-term effects. Consistent with the conclusions of Bacher-Hicks, Kane, and Staiger (2016) using data from Los Angeles, we conclude that Rothstein's replication study ultimately reinforces CFR's methods and results.

Suggested Citation

  • Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2017. "Measuring the Impacts of Teachers: Reply," American Economic Review, American Economic Association, vol. 107(6), pages 1685-1717, June.
  • Handle: RePEc:aea:aecrev:v:107:y:2017:i:6:p:1685-1717
    Note: DOI: 10.1257/aer.20170108
    as

    Download full text from publisher

    File URL: https://www.aeaweb.org/articles?id=10.1257/aer.20170108
    Download Restriction: no

    File URL: https://www.aeaweb.org/articles/attachments?retrieve=NwPWlwbghg_xYQuE7ojkH4r-J9CBCimn
    Download Restriction: no

    File URL: https://www.aeaweb.org/articles/attachments?retrieve=RFQ-u3AMocDGfU7q50LOSmosvUuIWiJN
    Download Restriction: Access to full text is restricted to AEA members and institutional subscribers.
    ---><---

    References listed on IDEAS

    as
    1. Raj Chetty & John N. Friedman & Jonah Rockoff, 2016. "Using Lagged Outcomes to Evaluate Bias in Value-Added Models," American Economic Review, American Economic Association, vol. 106(5), pages 393-399, May.
    2. Jesse Rothstein, 2010. "Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement," The Quarterly Journal of Economics, Oxford University Press, vol. 125(1), pages 175-214.
    3. David Deming, 2009. "Early Childhood Intervention and Life-Cycle Skill Development: Evidence from Head Start," American Economic Journal: Applied Economics, American Economic Association, vol. 1(3), pages 111-134, July.
    4. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    5. Thomas J. Kane & Douglas O. Staiger, 2008. "Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation," NBER Working Papers 14607, National Bureau of Economic Research, Inc.
    6. Raj Chetty & John N. Friedman & Nathaniel Hilger & Emmanuel Saez & Diane Whitmore Schanzenbach & Danny Yagan, 2011. "How Does Your Kindergarten Classroom Affect Your Earnings? Evidence from Project Star," The Quarterly Journal of Economics, Oxford University Press, vol. 126(4), pages 1593-1660.
    7. Jesse Rothstein, 2017. "Measuring the Impacts of Teachers: Comment," American Economic Review, American Economic Association, vol. 107(6), pages 1656-1684, June.
    8. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2011. "The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood," NBER Working Papers 17699, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dan Goldhaber, 2018. "Impact and Your Death Bed: Playing the Long Game," Education Finance and Policy, MIT Press, vol. 13(1), pages 1-18, Winter.
    2. Caterina Calsamiglia & Francisco Martínez-Mora & Antonio Miralles, 2021. "School Choice Design, Risk Aversion and Cardinal Segregation," The Economic Journal, Royal Economic Society, vol. 131(635), pages 1081-1104.
    3. Annika B. Bergbauer & Eric A. Hanushek & Ludger Woessmann, 2018. "Testing," NBER Working Papers 24836, National Bureau of Economic Research, Inc.
    4. Naven, Matthew, 2019. "Human-Capital Formation During Childhood and Adolescence: Evidence from School Quality and Postsecondary Success in California," MPRA Paper 97716, University Library of Munich, Germany.
    5. Bacher-Hicks, Andrew & Chin, Mark J. & Kane, Thomas J. & Staiger, Douglas O., 2019. "An experimental evaluation of three teacher quality measures: Value-added, classroom observations, and student surveys," Economics of Education Review, Elsevier, vol. 73(C).
    6. Godstime Osekhebhen Eigbiremolen, 2020. "Estimating Private School Premium for Primary School Children in Ethiopia: Evidence from Individual-level Panel Data," Progress in Development Studies, , vol. 20(1), pages 26-44, January.
    7. Brian Gill & Christine Ross & Erin Dillon & Ann Li & Patricia Troppe & Eric Isenberg & Anthony Milanowski & Roberta Garrison-Mogren & Louis Rizzo, "undated". "The Transition to ESSA: State and District Approaches to Implementing Title I and Title II-A in 2017–18," Mathematica Policy Research Reports b081b481f0ae4dcca46c338f9, Mathematica Policy Research.
    8. Wangyang Li & Minyi Li & Yongai Jin & Shiqi Wang & Yi Zhang, 2020. "Double Jeopardy in Contemporary China: Intersecting the Socioeconomic Gradient and Geographic Context on Early Childhood Development," IJERPH, MDPI, vol. 17(14), pages 1-24, July.
    9. Eric Parsons & Cory Koedel & Li Tan, 2019. "Accounting for Student Disadvantage in Value-Added Models," Journal of Educational and Behavioral Statistics, , vol. 44(2), pages 144-179, April.
    10. Lundborg, Petter & James, Stefan & Lagerqvist, Bo & Vikström, Johan, 2021. "Learning-by-Doing and Productivity Growth among High-Skilled Workers: Evidence from the Treatment of Heart Attacks," IZA Discussion Papers 14744, Institute of Labor Economics (IZA).
    11. Brown, Annette N. & Wood, Benjamin Douglas Kuflick, 2018. "Which tests not witch hunts: A diagnostic approach for conducting replication research," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-26.
    12. Jesse Rothstein, 2017. "Measuring the Impacts of Teachers: Comment," American Economic Review, American Economic Association, vol. 107(6), pages 1656-1684, June.
    13. W. Bentley MacLeod & Miguel Urquiola, 2018. "Is Education Consumption or Investment? Implications for the Effect of School Competition," NBER Working Papers 25117, National Bureau of Economic Research, Inc.
    14. Zheng, Lei & Qi, Xiang & Zhang, Chongjiu, 2023. "Can improvements in teacher quality reduce the cognitive gap between urban and rural students in China?," International Journal of Educational Development, Elsevier, vol. 100(C).
    15. Yeycol Leiva & Gabriel Pino, 2020. "Analysis of the impact of school performance on income inequality in the long run: An application to Chilean municipalities," Growth and Change, Wiley Blackwell, vol. 51(3), pages 1045-1080, September.
    16. Michael J. Andrews & Aaron K. Chatterji & Scott Stern, 2021. "Introduction: Beyond 140 Characters," NBER Chapters, in: The Role of Innovation and Entrepreneurship in Economic Growth, pages 1-28, National Bureau of Economic Research, Inc.
    17. Backes, Ben & Cowan, James & Goldhaber, Dan & Koedel, Cory & Miller, Luke C. & Xu, Zeyu, 2018. "The common core conundrum: To what extent should we worry that changes to assessments will affect test-based measures of teacher performance?," Economics of Education Review, Elsevier, vol. 62(C), pages 48-65.
    18. Camelo, Rafael & Ponczek, Vladimir, 2021. "Teacher Turnover and Financial Incentives in Underprivileged Schools: Evidence from a Compensation Policy in a Developing Country," Economics of Education Review, Elsevier, vol. 80(C).
    19. Tom Ahn & Esteban Aucejo & Jonathan James, 2021. "The Importance of Matching Effects for Labor Productivity: Evidence from Teacher-Student Interactions," Working Papers 2106, California Polytechnic State University, Department of Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. C. Kirabo Jackson, 2012. "Non-Cognitive Ability, Test Scores, and Teacher Quality: Evidence from 9th Grade Teachers in North Carolina," NBER Working Papers 18624, National Bureau of Economic Research, Inc.
    2. Joshua D. Angrist & Sarah R. Cohodes & Susan M. Dynarski & Parag A. Pathak & Christopher R. Walters, 2016. "Stand and Deliver: Effects of Boston's Charter High Schools on College Preparation, Entry, and Choice," Journal of Labor Economics, University of Chicago Press, vol. 34(2), pages 275-318.
    3. Patrick Kline & Christopher R. Walters, 2016. "Evaluating Public Programs with Close Substitutes: The Case of HeadStart," The Quarterly Journal of Economics, Oxford University Press, vol. 131(4), pages 1795-1848.
    4. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood," American Economic Review, American Economic Association, vol. 104(9), pages 2633-2679, September.
    5. Koedel, Cory & Mihaly, Kata & Rockoff, Jonah E., 2015. "Value-added modeling: A review," Economics of Education Review, Elsevier, vol. 47(C), pages 180-195.
    6. Elizabeth U. Cascio & Douglas O. Staiger, 2012. "Knowledge, Tests, and Fadeout in Educational Interventions," NBER Working Papers 18038, National Bureau of Economic Research, Inc.
    7. Mookerjee, Sulagna & Slichter, David, 2018. "Test Scores, Schools, and the Geography of Economic Opportunity," MPRA Paper 89101, University Library of Munich, Germany.
    8. Naven, Matthew, 2019. "Human-Capital Formation During Childhood and Adolescence: Evidence from School Quality and Postsecondary Success in California," MPRA Paper 97716, University Library of Munich, Germany.
    9. Seth Gershenson & Diane Whitmore Schanzenbach, 2016. "Linking Teacher Quality, Student Attendance, and Student Achievement," Education Finance and Policy, MIT Press, vol. 11(2), pages 125-149, Spring.
    10. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2011. "The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood," NBER Working Papers 17699, National Bureau of Economic Research, Inc.
    11. Tahir Andrabi & Jishnu Das & Asim Ijaz Khwaja & Tristan Zajonc, 2011. "Do Value-Added Estimates Add Value? Accounting for Learning Dynamics," American Economic Journal: Applied Economics, American Economic Association, vol. 3(3), pages 29-54, July.
    12. Hanushek, Eric A. & Rivkin, Steven G. & Schiman, Jeffrey C., 2016. "Dynamic effects of teacher turnover on the quality of instruction," Economics of Education Review, Elsevier, vol. 55(C), pages 132-148.
    13. David J. Deming & Sarah Cohodes & Jennifer Jennings & Christopher Jencks, 2016. "School Accountability, Postsecondary Attainment, and Earnings," The Review of Economics and Statistics, MIT Press, vol. 98(5), pages 848-862, December.
    14. Koedel Cory & Leatherman Rebecca & Parsons Eric, 2012. "Test Measurement Error and Inference from Value-Added Models," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 12(1), pages 1-37, November.
    15. Zheng, Lei & Qi, Xiang & Zhang, Chongjiu, 2023. "Can improvements in teacher quality reduce the cognitive gap between urban and rural students in China?," International Journal of Educational Development, Elsevier, vol. 100(C).
    16. Braga, Michela & Paccagnella, Marco & Pellizzari, Michele, 2014. "Evaluating students’ evaluations of professors," Economics of Education Review, Elsevier, vol. 41(C), pages 71-88.
    17. Joshua Angrist & Peter Hull & Parag Pathak & Christopher Walters, 2016. "Interpreting Tests of School VAM Validity," American Economic Review, American Economic Association, vol. 106(5), pages 388-392, May.
    18. Raj Chetty & John N. Friedman & Nathaniel Hilger & Emmanuel Saez & Diane Whitmore Schanzenbach & Danny Yagan, 2011. "How Does Your Kindergarten Classroom Affect Your Earnings? Evidence from Project Star," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(4), pages 1593-1660.
    19. Allison Atteberry & Susanna Loeb & James Wyckoff, 2013. "Do First Impressions Matter? Improvement in Early Career Teacher Effectiveness," NBER Working Papers 19096, National Bureau of Economic Research, Inc.
    20. Canales, Andrea & Maldonado, Luis, 2018. "Teacher quality and student achievement in Chile: Linking teachers' contribution and observable characteristics," International Journal of Educational Development, Elsevier, vol. 60(C), pages 33-50.

    More about this item

    JEL classification:

    • H75 - Public Economics - - State and Local Government; Intergovernmental Relations - - - State and Local Government: Health, Education, and Welfare
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • J45 - Labor and Demographic Economics - - Particular Labor Markets - - - Public Sector Labor Markets

    Lists

    This item is featured on the following reading lists, Wikipedia, or ReplicationWiki pages:
    1. Measuring the Impacts of Teachers: Comment (AER 2017) in ReplicationWiki

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:aecrev:v:107:y:2017:i:6:p:1685-1717. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Michael P. Albert (email available below). General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.