IDEAS home Printed from https://ideas.repec.org/a/aea/aejapp/v11y2019i3p382-423.html
   My bibliography  Save this article

The Causes and Consequences of Test Score Manipulation: Evidence from the New York Regents Examinations

Author

Listed:
  • Thomas S. Dee
  • Will Dobbie
  • Brian A. Jacob
  • Jonah Rockoff

Abstract

We show that the design and decentralized scoring of New York's high school exit exams—the Regents Examinations—led to systematic manipulation of test scores just below important proficiency cutoffs. Exploiting a series of reforms that eliminated score manipulation, we find heterogeneous effects of test score manipulation on academic outcomes. While inflating a score increases the probability of a student graduating from high school by about 17 percentage points, the probability of taking advanced coursework declines by roughly 10 percentage points. We argue that these results are consistent with test score manipulation helping less advanced students on the margin of dropping out but hurting more advanced students that are not pushed to gain a solid foundation in the introductory material.

Suggested Citation

  • Thomas S. Dee & Will Dobbie & Brian A. Jacob & Jonah Rockoff, 2019. "The Causes and Consequences of Test Score Manipulation: Evidence from the New York Regents Examinations," American Economic Journal: Applied Economics, American Economic Association, vol. 11(3), pages 382-423, July.
  • Handle: RePEc:aea:aejapp:v:11:y:2019:i:3:p:382-423
    Note: DOI: 10.1257/app.20170520
    as

    Download full text from publisher

    File URL: https://www.aeaweb.org/doi/10.1257/app.20170520
    Download Restriction: no

    File URL: https://www.aeaweb.org/doi/10.1257/app.20170520.data
    Download Restriction: no

    File URL: https://www.aeaweb.org/doi/10.1257/app.20170520.appx
    Download Restriction: no

    File URL: https://www.aeaweb.org/doi/10.1257/app.20170520.ds
    Download Restriction: Access to full text is restricted to AEA members and institutional subscribers.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Joshua D. Angrist & Alan B. Keueger, 1991. "Does Compulsory School Attendance Affect Schooling and Earnings?," The Quarterly Journal of Economics, Oxford University Press, vol. 106(4), pages 979-1014.
    2. Jörn-Steffen Pischke & Till von Wachter, 2008. "Zero Returns to Compulsory Schooling in Germany: Evidence and Interpretation," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 592-598, August.
    3. Jaeger, David A & Page, Marianne E, 1996. "Degrees Matter: New Evidence on Sheepskin Effects in the Returns to Education," The Review of Economics and Statistics, MIT Press, vol. 78(4), pages 733-740, November.
    4. Ou, Dongshu, 2010. "To leave or not to leave? A regression discontinuity analysis of the impact of failing the high school exit exam," Economics of Education Review, Elsevier, vol. 29(2), pages 171-186, April.
    5. Victor Lavy, 2009. "Performance Pay and Teachers' Effort, Productivity, and Grading Ethics," American Economic Review, American Economic Association, vol. 99(5), pages 1979-2011, December.
    6. Derek Neal, 2013. "The Consequences of Using one Assessment System to Pursue two Objectives," The Journal of Economic Education, Taylor & Francis Journals, vol. 44(4), pages 339-352, October.
    7. Giorgio Brunello & Margherita Fort & Guglielmo Weber, 2009. "Changes in Compulsory Schooling, Education and the Distribution of Wages in Europe," Economic Journal, Royal Economic Society, vol. 119(536), pages 516-539, March.
    8. Raj Chetty & John N. Friedman & Tore Olsen & Luigi Pistaferri, 2011. "Adjustment Costs, Firm Responses, and Micro vs. Macro Labor Supply Elasticities: Evidence from Danish Tax Records," The Quarterly Journal of Economics, Oxford University Press, vol. 126(2), pages 749-804.
    9. Hinnerich, Björn Tyrefors & Höglin, Erik & Johannesson, Magnus, 2011. "Are boys discriminated in Swedish high schools?," Economics of Education Review, Elsevier, vol. 30(4), pages 682-690, August.
    10. Julie Berry Cullen & Randall Reback, 2006. "Tinkering Toward Accolades: School Gaming Under a Performance Accountability System," NBER Working Papers 12286, National Bureau of Economic Research, Inc.
    11. Brian A. Jacob & Steven D. Levitt, 2003. "Rotten Apples: An Investigation of the Prevalence and Predictors of Teacher Cheating," The Quarterly Journal of Economics, Oxford University Press, vol. 118(3), pages 843-877.
    12. Lavy, Victor & Sand, Edith, 2015. "On The Origins of Gender Human Capital Gaps: Short and Long Term Consequences of Teachers’ Stereotypical Biases," Economic Research Papers 270009, University of Warwick - Department of Economics.
    13. Derek Neal, 2013. "The Consequences of Using One Assessment System To Pursue Two Objectives," NBER Working Papers 19214, National Bureau of Economic Research, Inc.
    14. Lavy, Victor, 2008. "Do gender stereotypes reduce girls' or boys' human capital outcomes? Evidence from a natural experiment," Journal of Public Economics, Elsevier, vol. 92(10-11), pages 2083-2105, October.
    15. Roland G. Fryer, 2013. "Teacher Incentives and Student Achievement: Evidence from New York City Public Schools," Journal of Labor Economics, University of Chicago Press, vol. 31(2), pages 373-407.
    16. Papay, John P. & Willett, John B. & Murnane, Richard J., 2011. "Extending the regression-discontinuity approach to multiple assignment variables," Journal of Econometrics, Elsevier, vol. 161(2), pages 203-207, April.
    17. Emmanuel Saez, 2010. "Do Taxpayers Bunch at Kink Points?," American Economic Journal: Economic Policy, American Economic Association, vol. 2(3), pages 180-212, August.
    18. Derek Neal & Diane Whitmore Schanzenbach, 2010. "Left Behind by Design: Proficiency Counts and Test-Based Accountability," The Review of Economics and Statistics, MIT Press, vol. 92(2), pages 263-283, May.
    19. Henrik J. Kleven & Mazhar Waseem, 2013. "Using Notches to Uncover Optimization Frictions and Structural Elasticities: Theory and Evidence from Pakistan," The Quarterly Journal of Economics, Oxford University Press, vol. 128(2), pages 669-723.
    20. Oreopoulos, Philip, 2007. "Do dropouts drop out too soon? Wealth, health and happiness from compulsory schooling," Journal of Public Economics, Elsevier, vol. 91(11-12), pages 2213-2229, December.
    21. Persson, Petra, 2015. "Social insurance and the marriage market," Working Paper Series 2015:6, IFAU - Institute for Evaluation of Labour Market and Education Policy.
    22. Simon Burgess & Ellen Greaves, 2013. "Test Scores, Subjective Assessment, and Stereotyping of Ethnic Minorities," Journal of Labor Economics, University of Chicago Press, vol. 31(3), pages 535-576.
    23. Damon Clark & Paco Martorell, 2014. "The Signaling Value of a High School Diploma," Journal of Political Economy, University of Chicago Press, vol. 122(2), pages 282-318.
    24. David N. Figlio & Lawrence S. Getzler, 2002. "Accountability , Ability and Disability: Gaming the System," NBER Working Papers 9307, National Bureau of Economic Research, Inc.
    25. repec:ucp:jlabec:doi:10.1086/694468 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hinnerich, Björn Tyrefors & Vlachos, Jonas, 2017. "The impact of upper-secondary voucher school attendance on student achievement. Swedish evidence using external and internal evaluations," Labour Economics, Elsevier, vol. 47(C), pages 1-14.
    2. repec:eee:econom:v:200:y:2017:i:2:p:344-362 is not listed on IDEAS
    3. Erich Battistin & Lorenzo Neri, 2017. "School Performance, Score Inflation and Economic Geography," Working Papers 837, Queen Mary University of London, School of Economics and Finance.
    4. repec:eee:pubeco:v:159:y:2018:i:c:p:203-224 is not listed on IDEAS
    5. Reynaert, Mathias & Sallee, James M., 2016. "Who Benefits When Firms Game Corrective Policies?," TSE Working Papers 16-739, Toulouse School of Economics (TSE), revised Feb 2018.
    6. Battistin, Erich & De Nadai, Michele & Vuri, Daniela, 2017. "Counting rotten apples: Student achievement and score manipulation in Italian elementary Schools," Journal of Econometrics, Elsevier, vol. 200(2), pages 344-362.
    7. repec:eee:indorg:v:62:y:2019:i:c:p:228-250 is not listed on IDEAS
    8. Laura Blattner & Luisa Farinha & Francisca Rebelo, 2017. "When Losses Turn Into Loans: The Cost of Undercapitalized Banks," 2017 Papers pbl215, Job Market Papers.
    9. repec:spr:stmapp:v:27:y:2018:i:3:d:10.1007_s10260-018-0426-2 is not listed on IDEAS
    10. Claudio Lucifora & Marco Tonello, 2016. "Monitoring and sanctioning cheating at school: What works? Evidence from a national evaluation program," DISCE - Working Papers del Dipartimento di Economia e Finanza def051, Università Cattolica del Sacro Cuore, Dipartimenti e Istituti di Scienze Economiche (DISCE).
    11. Lusher, Lester & Campbell, Doug & Carrell, Scott, 2018. "TAs like me: Racial interactions between graduate teaching assistants and undergraduates," Journal of Public Economics, Elsevier, vol. 159(C), pages 203-224.
    12. repec:aea:aejapp:v:9:y:2017:i:4:p:216-49 is not listed on IDEAS
    13. Joshua D. Angrist & Erich Battistin & Daniela Vuri, 2017. "In a Small Moment: Class Size and Moral Hazard in the Italian Mezzogiorno," American Economic Journal: Applied Economics, American Economic Association, vol. 9(4), pages 216-249, October.

    More about this item

    JEL classification:

    • H75 - Public Economics - - State and Local Government; Intergovernmental Relations - - - State and Local Government: Health, Education, and Welfare
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:aejapp:v:11:y:2019:i:3:p:382-423. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Michael P. Albert). General contact details of provider: http://edirc.repec.org/data/aeaaaea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.