IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/22165.html
   My bibliography  Save this paper

The Causes and Consequences of Test Score Manipulation: Evidence from the New York Regents Examinations

Author

Listed:
  • Thomas S. Dee
  • Will Dobbie
  • Brian A. Jacob
  • Jonah Rockoff

Abstract

In this paper, we show that the design and decentralized, school-based scoring of New York’s high school exit exams – the Regents Examinations – led to the systematic manipulation of test sores just below important proficiency cutoffs. Our estimates suggest that teachers inflate approximately 40 percent of test scores near the proficiency cutoffs. Teachers are more likely to inflate the scores of high-achieving students on the margin, but low-achieving students benefit more from manipulation in aggregate due to the greater density of these students near the proficiency cutoffs. Exploiting a series of reforms that eliminated score manipulation, we find that inflating a student’s score to fall just above a cutoff increases his or her probability of graduating from high school by 27 percent. These results have important implications for educational attainment of marginal high school graduates. For example, we estimate that the black-white graduation gap is about 5 percent larger in the absence of test score manipulation.

Suggested Citation

  • Thomas S. Dee & Will Dobbie & Brian A. Jacob & Jonah Rockoff, 2016. "The Causes and Consequences of Test Score Manipulation: Evidence from the New York Regents Examinations," NBER Working Papers 22165, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:22165
    Note: ED
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w22165.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Jörn-Steffen Pischke & Till von Wachter, 2008. "Zero Returns to Compulsory Schooling in Germany: Evidence and Interpretation," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 592-598, August.
    2. Giorgio Brunello & Margherita Fort & Guglielmo Weber, 2009. "Changes in Compulsory Schooling, Education and the Distribution of Wages in Europe," Economic Journal, Royal Economic Society, vol. 119(536), pages 516-539, March.
    3. Victor Lavy, 2009. "Performance Pay and Teachers' Effort, Productivity, and Grading Ethics," American Economic Review, American Economic Association, vol. 99(5), pages 1979-2011, December.
    4. Raj Chetty & John N. Friedman & Tore Olsen & Luigi Pistaferri, 2011. "Adjustment Costs, Firm Responses, and Micro vs. Macro Labor Supply Elasticities: Evidence from Danish Tax Records," The Quarterly Journal of Economics, Oxford University Press, vol. 126(2), pages 749-804.
    5. Hinnerich, Björn Tyrefors & Höglin, Erik & Johannesson, Magnus, 2011. "Are boys discriminated in Swedish high schools?," Economics of Education Review, Elsevier, vol. 30(4), pages 682-690, August.
    6. Derek Neal, 2013. "The Consequences of Using One Assessment System To Pursue Two Objectives," NBER Working Papers 19214, National Bureau of Economic Research, Inc.
    7. Brian A. Jacob & Steven D. Levitt, 2003. "Rotten Apples: An Investigation of the Prevalence and Predictors of Teacher Cheating," The Quarterly Journal of Economics, Oxford University Press, vol. 118(3), pages 843-877.
    8. Emmanuel Saez, 2010. "Do Taxpayers Bunch at Kink Points?," American Economic Journal: Economic Policy, American Economic Association, vol. 2(3), pages 180-212, August.
    9. Ou, Dongshu, 2010. "To leave or not to leave? A regression discontinuity analysis of the impact of failing the high school exit exam," Economics of Education Review, Elsevier, vol. 29(2), pages 171-186, April.
    10. Simon Burgess & Ellen Greaves, 2013. "Test Scores, Subjective Assessment, and Stereotyping of Ethnic Minorities," Journal of Labor Economics, University of Chicago Press, vol. 31(3), pages 535-576.
    11. Lavy, Victor & Sand, Edith, 2015. "On The Origins of Gender Human Capital Gaps: Short and Long Term Consequences of Teachers’ Stereotypical Biases," The Warwick Economics Research Paper Series (TWERPS) 1085, University of Warwick, Department of Economics.
    12. Julie Berry Cullen & Randall Reback, 2006. "Tinkering Toward Accolades: School Gaming Under a Performance Accountability System," NBER Working Papers 12286, National Bureau of Economic Research, Inc.
    13. Papay, John P. & Willett, John B. & Murnane, Richard J., 2011. "Extending the regression-discontinuity approach to multiple assignment variables," Journal of Econometrics, Elsevier, vol. 161(2), pages 203-207, April.
    14. Derek Neal & Diane Whitmore Schanzenbach, 2010. "Left Behind by Design: Proficiency Counts and Test-Based Accountability," The Review of Economics and Statistics, MIT Press, vol. 92(2), pages 263-283, May.
    15. Henrik J. Kleven & Mazhar Waseem, 2013. "Using Notches to Uncover Optimization Frictions and Structural Elasticities: Theory and Evidence from Pakistan," The Quarterly Journal of Economics, Oxford University Press, vol. 128(2), pages 669-723.
    16. Oreopoulos, Philip, 2007. "Do dropouts drop out too soon? Wealth, health and happiness from compulsory schooling," Journal of Public Economics, Elsevier, vol. 91(11-12), pages 2213-2229, December.
    17. Persson, Petra, 2015. "Social insurance and the marriage market," Working Paper Series 2015:6, IFAU - Institute for Evaluation of Labour Market and Education Policy.
    18. Joshua D. Angrist & Alan B. Keueger, 1991. "Does Compulsory School Attendance Affect Schooling and Earnings?," The Quarterly Journal of Economics, Oxford University Press, vol. 106(4), pages 979-1014.
    19. David N. Figlio & Lawrence S. Getzler, 2002. "Accountability , Ability and Disability: Gaming the System," NBER Working Papers 9307, National Bureau of Economic Research, Inc.
    20. Jaeger, David A & Page, Marianne E, 1996. "Degrees Matter: New Evidence on Sheepskin Effects in the Returns to Education," The Review of Economics and Statistics, MIT Press, vol. 78(4), pages 733-740, November.
    21. Derek Neal, 2013. "The Consequences of Using one Assessment System to Pursue two Objectives," The Journal of Economic Education, Taylor & Francis Journals, vol. 44(4), pages 339-352, October.
    22. Lavy, Victor, 2008. "Do gender stereotypes reduce girls' or boys' human capital outcomes? Evidence from a natural experiment," Journal of Public Economics, Elsevier, vol. 92(10-11), pages 2083-2105, October.
    23. Roland G. Fryer, 2013. "Teacher Incentives and Student Achievement: Evidence from New York City Public Schools," Journal of Labor Economics, University of Chicago Press, vol. 31(2), pages 373-407.
    24. Damon Clark & Paco Martorell, 2014. "The Signaling Value of a High School Diploma," Journal of Political Economy, University of Chicago Press, vol. 122(2), pages 282-318.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hinnerich, Björn Tyrefors & Vlachos, Jonas, 2017. "The impact of upper-secondary voucher school attendance on student achievement. Swedish evidence using external and internal evaluations," Labour Economics, Elsevier, vol. 47(C), pages 1-14.
    2. repec:eee:econom:v:200:y:2017:i:2:p:344-362 is not listed on IDEAS
    3. Erich Battistin & Lorenzo Neri, 2017. "School Performance, Score Inflation and Economic Geography," Working Papers 837, Queen Mary University of London, School of Economics and Finance.
    4. repec:eee:pubeco:v:159:y:2018:i:c:p:203-224 is not listed on IDEAS
    5. Battistin, Erich & De Nadai, Michele & Vuri, Daniela, 2017. "Counting rotten apples: Student achievement and score manipulation in Italian elementary Schools," Journal of Econometrics, Elsevier, vol. 200(2), pages 344-362.
    6. Claudio Lucifora & Marco Tonello, 2016. "Monitoring and sanctioning cheating at school: What works? Evidence from a national evaluation program," DISCE - Working Papers del Dipartimento di Economia e Finanza def051, Università Cattolica del Sacro Cuore, Dipartimenti e Istituti di Scienze Economiche (DISCE).
    7. repec:aea:aejapp:v:9:y:2017:i:4:p:216-49 is not listed on IDEAS
    8. Joshua D. Angrist & Erich Battistin & Daniela Vuri, 2017. "In a Small Moment: Class Size and Moral Hazard in the Italian Mezzogiorno," American Economic Journal: Applied Economics, American Economic Association, vol. 9(4), pages 216-249, October.

    More about this item

    JEL classification:

    • I20 - Health, Education, and Welfare - - Education - - - General
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I24 - Health, Education, and Welfare - - Education - - - Education and Inequality

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:22165. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: () or (Joanne Lustig). General contact details of provider: http://edirc.repec.org/data/nberrus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.