IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/8968.html
   My bibliography  Save this paper

Accountability, Incentives and Behavior: The Impact of High-Stakes Testing in the Chicago Public Schools

Author

Listed:
  • Brian A. Jacob

Abstract

The recent federal education bill, No Child Left Behind, requires states to test students in grades three to eight each year, and to judge school performance on the basis of these test scores. While intended to maximize student learning, there is little empirical evidence about the effectiveness of such policies. This study examines the impact of an accountability policy implemented in the Chicago Public Schools in 1996-97. Using a panel of student-level, administrative data, I find that math and reading achievement increased sharply following the introduction of the accountability policy, in comparison to both prior achievement trends in the district and to changes experienced by other large, urban districts in the mid-west. I demonstrate that these gains were driven largely by increases in test-specific skills and student effort, and did not lead to comparable gains on a state-administered, low-stakes exam. I also find that teachers responded strategically to the incentives along a variety of dimensions -- by increasing special education placements, preemptively retaining students and substituting away from low-stakes subjects like science and social studies.

Suggested Citation

  • Brian A. Jacob, 2002. "Accountability, Incentives and Behavior: The Impact of High-Stakes Testing in the Chicago Public Schools," NBER Working Papers 8968, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:8968
    Note: CH ED
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w8968.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. repec:pri:crcwel:447.pdf is not listed on IDEAS
    2. Alan Krueger, 2000. "Economic Considerations and Class Size," Working Papers 826, Princeton University, Department of Economics, Industrial Relations Section..
    3. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
    4. repec:pri:crcwel:447 is not listed on IDEAS
    5. Brian A. Jacob & Steven D. Levitt, 2003. "Rotten Apples: An Investigation of the Prevalence and Predictors of Teacher Cheating," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(3), pages 843-877.
    6. Brian A. Jacob & Lars Lefgren, 2004. "Remedial Education and Student Achievement: A Regression-Discontinuity Analysis," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 226-244, February.
    7. Brian A. Jacob & Lars Lefgren, 2004. "The Impact of Teacher Training on Student Achievement: Quasi-Experimental Evidence from School Reform Efforts in Chicago," Journal of Human Resources, University of Wisconsin Press, vol. 39(1).
    8. Richards, Craig E. & Sheu, Tian Ming, 1992. "The South Carolina school incentive reward program: A policy analysis," Economics of Education Review, Elsevier, vol. 11(1), pages 71-86, March.
    9. Hanushek, Eric A., 2006. "School Resources," Handbook of the Economics of Education, in: Erik Hanushek & F. Welch (ed.), Handbook of the Economics of Education, edition 1, volume 2, chapter 14, pages 865-908, Elsevier.
    10. Julie Berry Cullen & Randall Reback, 2006. "Tinkering Toward Accolades: School Gaming Under a Performance Accountability System," NBER Working Papers 12286, National Bureau of Economic Research, Inc.
    11. Caroline M. Hoxby, 2002. "The Cost of Accountability," NBER Working Papers 8855, National Bureau of Economic Research, Inc.
    12. Alan B. Krueger, 2003. "Economic Considerations and Class Size," Economic Journal, Royal Economic Society, vol. 113(485), pages 34-63, February.
    13. Figlio, David N. & Winicki, Joshua, 2005. "Food for thought: the effects of school accountability plans on school nutrition," Journal of Public Economics, Elsevier, vol. 89(2-3), pages 381-394, February.
    14. Ladd, Helen F., 1999. "The Dallas school accountability and incentive program: an evaluation of its impacts on student outcomes," Economics of Education Review, Elsevier, vol. 18(1), pages 1-16, February.
    15. Eric A. Hanushek & John F. Kain & Steven G. Rivkin, 1998. "Does Special Education Raise Academic Achievement for Students with Disabilities?," NBER Working Papers 6690, National Bureau of Economic Research, Inc.
    16. David N. Figlio & Lawrence S. Getzler, 2002. "Accountability , Ability and Disability: Gaming the System," NBER Working Papers 9307, National Bureau of Economic Research, Inc.
    17. John H. Bishop, 1998. "The Effect of Curriculum-Based External Exit Exam Systems on Student Achievement," The Journal of Economic Education, Taylor & Francis Journals, vol. 29(2), pages 171-182, June.
    18. repec:fth:prinin:447 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jaag, Christian, 2006. "Teacher Incentives," MPRA Paper 340, University Library of Munich, Germany.
    2. Stoddard, Christiana & Kuhn, Peter J., 2004. "Incentives and Effort in the Public Sector: Have U.S. Education Reforms Increased Teachers’ Work Hours?," IZA Discussion Papers 1412, Institute of Labor Economics (IZA).
    3. Brian A. Jacob, 2007. "Test-Based Accountability and Student Achievement: An Investigation of Differential Performance on NAEP and State Assessments," NBER Working Papers 12817, National Bureau of Economic Research, Inc.
    4. Deborah Wilson & Bronwyn Croxson & Adele Atkinson, 2004. "“What Gets Measured Gets Done”: Headteachers’ Responses to the English Secondary School," The Centre for Market and Public Organisation 04/107, The Centre for Market and Public Organisation, University of Bristol, UK.
    5. Paul Glewwe & Nauman Ilias & Michael Kremer, 2010. "Teacher Incentives," American Economic Journal: Applied Economics, American Economic Association, vol. 2(3), pages 205-227, July.
    6. Christian Jaag, 2005. "Hidden Teacher Effort in Educational Production: Monitoring vs. Merit Pay," HEW 0503003, University Library of Munich, Germany.
    7. Oriana Bandiera & Iwan Barankay & Imran Rasul, 2007. "Incentives for Managers and Inequality among Workers: Evidence from a Firm-Level Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(2), pages 729-773.
    8. Kaoru Nabeshima, 2003. "Raising the quality of secondary education in East Asia," Policy Research Working Paper Series 3140, The World Bank.
    9. Justin Ross, 2008. "A theoretical model of the distribution of teacher attention under benchmark testing," Economics Bulletin, AccessEcon, vol. 9(29), pages 1-8.
    10. Atif Mian, 2008. "Incentives in Markets, Firms, and Governments," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 24(2), pages 273-306, October.
    11. repec:wvu:wpaper:07-03 is not listed on IDEAS
    12. Figlio, David N. & Rouse, Cecilia Elena, 2006. "Do accountability and voucher threats improve low-performing schools?," Journal of Public Economics, Elsevier, vol. 90(1-2), pages 239-255, January.
    13. repec:ebl:ecbull:v:9:y:2008:i:29:p:1-8 is not listed on IDEAS
    14. Elizabeth Webster & Mark Wooden & Gary Marks, 2004. "Reforming the Labour Market for Australian Teachers," Melbourne Institute Working Paper Series wp2004n28, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jacob, Brian A., 2005. "Accountability, incentives and behavior: the impact of high-stakes testing in the Chicago Public Schools," Journal of Public Economics, Elsevier, vol. 89(5-6), pages 761-796, June.
    2. Chiang, Hanley, 2009. "How accountability pressure on failing schools affects student achievement," Journal of Public Economics, Elsevier, vol. 93(9-10), pages 1045-1057, October.
    3. Figlio, David N. & Rouse, Cecilia Elena, 2006. "Do accountability and voucher threats improve low-performing schools?," Journal of Public Economics, Elsevier, vol. 90(1-2), pages 239-255, January.
    4. Brian A. Jacob, 2007. "Test-Based Accountability and Student Achievement: An Investigation of Differential Performance on NAEP and State Assessments," NBER Working Papers 12817, National Bureau of Economic Research, Inc.
    5. Emiliana Vegas & Ilana Umansky, 2005. "Improving Teaching and Learning through Effective Incentives : What Can We Learn from Education Reforms in Latin America?," World Bank Publications - Reports 8694, The World Bank Group.
    6. Thomas J. Nechyba, 2006. "Alternative education finance strategies," Regional Economic Development, Federal Reserve Bank of St. Louis, issue Mar, pages 7-27.
    7. Derek Neal, 2011. "The Design of Performance Pay in Education," NBER Working Papers 16710, National Bureau of Economic Research, Inc.
    8. Rajashri Chakrabarti, 2013. "Vouchers, Public School Response, And The Role Of Incentives: Evidence From Florida," Economic Inquiry, Western Economic Association International, vol. 51(1), pages 500-526, January.
    9. Brian A. Jacob & Steven D. Levitt, 2003. "Rotten Apples: An Investigation of the Prevalence and Predictors of Teacher Cheating," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(3), pages 843-877.
    10. Cuesta, José Ignacio & González, Felipe & Larroulet Philippi, Cristian, 2020. "Distorted quality signals in school markets," Journal of Development Economics, Elsevier, vol. 147(C).
    11. Rezende, Marcelo, 2010. "The effects of accountability on higher education," Economics of Education Review, Elsevier, vol. 29(5), pages 842-856, October.
    12. Brian A. Jacob & Lars Lefgren, 2004. "The Impact of Teacher Training on Student Achievement: Quasi-Experimental Evidence from School Reform Efforts in Chicago," Journal of Human Resources, University of Wisconsin Press, vol. 39(1).
    13. Mansfield, Jonathan & Slichter, David, 2021. "The Long-Run Effects of Consequential School Accountability," IZA Discussion Papers 14503, Institute of Labor Economics (IZA).
    14. Eric A. Hanushek & Margaret E. Raymond, 2002. "Improving educational quality: how best to evaluate our schools," Conference Series ; [Proceedings], Federal Reserve Bank of Boston, vol. 47(Jun), pages 193-247.
    15. Wo[ss]mann, Ludger & West, Martin, 2006. "Class-size effects in school systems around the world: Evidence from between-grade variation in TIMSS," European Economic Review, Elsevier, vol. 50(3), pages 695-736, April.
    16. Liang Choon Wang, 2016. "The effect of high-stakes testing on suicidal ideation of teenagers with reference-dependent preferences," Journal of Population Economics, Springer;European Society for Population Economics, vol. 29(2), pages 345-364, April.
    17. Coelli, Michael & Foster, Gigi, 2024. "Unintended consequences of school accountability reforms: Public versus private schools," Economics of Education Review, Elsevier, vol. 99(C).
    18. Vegas, E & Ganimian, A. J., 2013. "Theory and Evidence on Teacher Policies in Developed and Developing Countries," Working Paper 104291, Harvard University OpenScholar.
    19. Ludger Wößmann, 2006. "Bildungspolitische Lehren aus den internationalen Schülertests: Wettbewerb, Autonomie und externe Leistungsüberprüfung," Perspektiven der Wirtschaftspolitik, Verein für Socialpolitik, vol. 7(3), pages 417-444, August.
    20. Lombardi, María, 2019. "Is the remedy worse than the disease? The impact of teacher remediation on teacher and student performance in Chile," Economics of Education Review, Elsevier, vol. 73(C).

    More about this item

    JEL classification:

    • I20 - Health, Education, and Welfare - - Education - - - General
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:8968. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.