IDEAS home Printed from https://ideas.repec.org/a/tpr/edfpol/v12y2017i2p197-223.html
   My bibliography  Save this article

Screen Twice, Cut Once: Assessing the Predictive Validity of Applicant Selection Tools

Author

Listed:
  • Dan Goldhaber

    () (Center for Education Data & Research University of Washington– Bothell Seattle, WA 98103)

  • Cyrus Grout

    () (Center for Education Data & Research University of Washington-- Bothell Seattle, WA 98103)

  • Nick Huntington-Klein

    () (Department of Economics California State University, Fullerton Fullerton, CA 92831)

Abstract

Despite their widespread use, there is little academic evidence on whether applicant selection instruments can improve teacher hiring. We examine the relationship between two screening instruments used by Spokane Public Schools to select classroom teachers and three teacher outcomes: value added, absences, and attrition. We observe all applicants to the district (not only those who are hired), allowing us to estimate sample selection-corrected models using random tally errors and variation in the level of competition across job postings as instruments. Ratings on the screening instruments significantly predict value added in math and teacher attrition, but not absences—an increase of one standard deviation in screening scores is associated with an increase of about 0.06 standard deviations of student math achievement, and a decrease in teacher attrition of 3 percentage points. Hence the use of selection instruments appears to be a key means of improving the quality of the teacher workforce.

Suggested Citation

  • Dan Goldhaber & Cyrus Grout & Nick Huntington-Klein, 2017. "Screen Twice, Cut Once: Assessing the Predictive Validity of Applicant Selection Tools," Education Finance and Policy, MIT Press, vol. 12(2), pages 197-223, Spring.
  • Handle: RePEc:tpr:edfpol:v:12:y:2017:i:2:p:197-223
    as

    Download full text from publisher

    File URL: http://www.mitpressjournals.org/doi/pdf/10.1162/EDFP_a_00200
    Download Restriction: Access to PDF is restricted to subscribers.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Steven G. Rivkin & Eric A. Hanushek & John F. Kain, 2005. "Teachers, Schools, and Academic Achievement," Econometrica, Econometric Society, vol. 73(2), pages 417-458, March.
    2. Steven Glazerman & Daniel Mayer & Paul Decker, 2006. "Alternative routes to teaching: The impacts of Teach for America on student achievement and other outcomes," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 25(1), pages 75-96.
    3. Thomas J. Kane & Douglas O. Staiger, 2008. "Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation," NBER Working Papers 14607, National Bureau of Economic Research, Inc.
    4. repec:mpr:mprres:4761 is not listed on IDEAS
    5. Charles T. Clotfelter & Helen F. Ladd & Jacob L. Vigdor, 2011. "Teacher Mobility, School Segregation, and Pay-Based Policies to Level the Playing Field," Education Finance and Policy, MIT Press, vol. 6(3), pages 399-438, July.
    6. Don Boyd & Hamp Lankford & Susanna Loeb & Matthew Ronfeldt & Jim Wyckoff, 2011. "The role of teacher quality in retention and hiring: Using applications to transfer to uncover preferences of teachers and schools," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 30(1), pages 88-110, December.
    7. Olsen, Randall J, 1980. "A Least Squares Correction for Selectivity Bias," Econometrica, Econometric Society, vol. 48(7), pages 1815-1820, November.
    8. Edward P. Lazear & Kathryn L. Shaw, 2007. "Personnel Economics: The Economist's View of Human Resources," Journal of Economic Perspectives, American Economic Association, vol. 21(4), pages 91-114, Fall.
    9. Eric A. Hanushek & John F. Kain & Steven G. Rivkin & Daniel M. O'Brien, 2005. "The Market for Teacher Quality," Discussion Papers 04-025, Stanford Institute for Economic Policy Research.
    10. Daniel Aaronson & Lisa Barrow & William Sander, 2007. "Teachers and Student Achievement in the Chicago Public High Schools," Journal of Labor Economics, University of Chicago Press, vol. 25, pages 95-135.
    11. Peter Hinrichs, 2014. "What Kind of Teachers Are Schools Looking For? Evidence from a Randomized Field Experiment," Working Papers (Old Series) 1436, Federal Reserve Bank of Cleveland, revised 10 Dec 2014.
    12. repec:hrv:faseco:30749073 is not listed on IDEAS
    13. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    14. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood," American Economic Review, American Economic Association, vol. 104(9), pages 2633-2679, September.
    15. Dan Goldhaber & Michael Hansen, 2013. "Is it Just a Bad Class? Assessing the Long-term Stability of Estimated Teacher Performance," Economica, London School of Economics and Political Science, vol. 80(319), pages 589-612, July.
    16. Mariesa A. Herrmann & Jonah E. Rockoff, 2012. "Worker Absence and Productivity: Evidence from Teaching," Journal of Labor Economics, University of Chicago Press, vol. 30(4), pages 749-782.
    17. Donald Boyd & Hamilton Lankford & Susanna Loeb & James Wyckoff, 2013. "Analyzing the Determinants of the Matching of Public School Teachers to Jobs: Disentangling the Preferences of Teachers and Employers," Journal of Labor Economics, University of Chicago Press, vol. 31(1), pages 83-117.
    18. repec:hrv:faseco:30749606 is not listed on IDEAS
    19. Jonah E. Rockoff & Brian A. Jacob & Thomas J. Kane & Douglas O. Staiger, 2011. "Can You Recognize an Effective Teacher When You Recruit One?," Education Finance and Policy, MIT Press, vol. 6(1), pages 43-74, January.
    20. Andrew Bacher-Hicks & Thomas J. Kane & Douglas O. Staiger, 2014. "Validating Teacher Effect Estimates Using Changes in Teacher Assignments in Los Angeles," NBER Working Papers 20657, National Bureau of Economic Research, Inc.
    21. Jonah E. Rockoff, 2004. "The Impact of Individual Teachers on Student Achievement: Evidence from Panel Data," American Economic Review, American Economic Association, vol. 94(2), pages 247-252, May.
    22. Richard Blundell & Monica Costa Dias, 2000. "Evaluation methods for non-experimental data," Fiscal Studies, Institute for Fiscal Studies, vol. 21(4), pages 427-468, January.
    23. Thomas S. Dee & James Wyckoff, 2015. "Incentives, Selection, and Teacher Performance: Evidence from IMPACT," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 34(2), pages 267-297, March.
    24. Douglas O. Staiger & Jonah E. Rockoff, 2010. "Searching for Effective Teachers with Imperfect Information," Journal of Economic Perspectives, American Economic Association, vol. 24(3), pages 97-118, Summer.
    25. Eric A. Hanushek & Steven G. Rivkin, 2010. "Generalizations about Using Value-Added Measures of Teacher Quality," American Economic Review, American Economic Association, vol. 100(2), pages 267-271, May.
    26. Dan Goldhaber & Roddy Theobald, 2013. "Managing the Teacher Workforce in Austere Times: The Determinants and Implications of Teacher Layoffs," Education Finance and Policy, MIT Press, vol. 8(4), pages 494-527, October.
    27. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Publishing House "SINERGIA PRESS", vol. 31(3), pages 129-137.
    28. Koedel, Cory & Mihaly, Kata & Rockoff, Jonah E., 2015. "Value-added modeling: A review," Economics of Education Review, Elsevier, vol. 47(C), pages 180-195.
    29. Joshua D. Angrist, 1998. "Estimating the Labor Market Impact of Voluntary Military Service Using Social Security Data on Military Applicants," Econometrica, Econometric Society, vol. 66(2), pages 249-288, March.
    30. Charles T. Clotfelter & Helen F. Ladd & Jacob L. Vigdor, 2006. "Teacher-Student Matching and the Assessment of Teacher Effectiveness," Journal of Human Resources, University of Wisconsin Press, vol. 41(4).
    31. Strauss, Robert P. & Bowes, Lori R. & Marks, Mindy S. & Plesko, Mark R., 2000. "Improving teacher preparation and selection: lessons from the Pennsylvania experience," Economics of Education Review, Elsevier, vol. 19(4), pages 387-415, October.
    32. Douglas N. Harris & Stacey A. Rutledge & William K. Ingle & Cynthia C. Thompson, 2010. "Mix and Match: What Principals Really Look for When Hiring Teachers," Education Finance and Policy, MIT Press, vol. 5(2), pages 228-246, April.
    33. Dale Ballou, 1996. "Do Public Schools Hire the Best Applicants?," The Quarterly Journal of Economics, Oxford University Press, vol. 111(1), pages 97-133.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dan Goldhaber, 2018. "Impact and Your Death Bed: Playing the Long Game," Education Finance and Policy, MIT Press, vol. 13(1), pages 1-18, Winter.
    2. Matthew A. Kraft & John P. Papay & Olivia L. Chi, 2020. "Teacher Skill Development: Evidence from Performance Ratings by Principals," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 315-347, March.
    3. Jacob, Brian A. & Rockoff, Jonah E. & Taylor, Eric S. & Lindy, Benjamin & Rosen, Rachel, 2018. "Teacher applicant hiring and teacher performance: Evidence from DC public schools," Journal of Public Economics, Elsevier, vol. 166(C), pages 81-97.
    4. Brian Jacob & Jonah E. Rockoff & Eric S. Taylor & Benjamin Lindy & Rachel Rosen, 2016. "Teacher Applicant Hiring and Teacher Performance: Evidence from DC Public Schools," NBER Working Papers 22054, National Bureau of Economic Research, Inc.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tpr:edfpol:v:12:y:2017:i:2:p:197-223. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Ann Olson). General contact details of provider: https://www.mitpressjournals.org/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.