IDEAS home Printed from https://ideas.repec.org/p/bdi/wptemi/td_1047_16.html
   My bibliography  Save this paper

A new method for the correction of test scores manipulation

Author

Listed:
  • Santiago Pereda Fernández

    (Banca d’Italia)

Abstract

I propose a method to correct for test scores manipulation and apply it to a natural experiment in the Italian education system consisting in the random assignment of external monitors to classrooms. The empirical strategy is based on a likelihood approach, using nonlinear panel data methods to obtain clean estimates of cheating controlling for unobserved heterogeneity. The likelihood of each classroom's scores is later used to correct them for cheating. Cheating is not associated with an increase in the correlation of the answers after we control for mean test scores. The method produces estimates of manipulation more frequent in the South and Islands and among female students and immigrants in Italian tests. A simulation shows how the manipulation reduces the accuracy of an exam in reflecting students' knowledge, and the correction proposed in this paper makes up for about a half of this loss.

Suggested Citation

  • Santiago Pereda Fernández, 2016. "A new method for the correction of test scores manipulation," Temi di discussione (Economic working papers) 1047, Bank of Italy, Economic Research and International Relations Area.
    • Handle: RePEc:bdi:wptemi:td_1047_16
    as

    Download full text from publisher

    File URL: http://www.bancaditalia.it/pubblicazioni/temi-discussione/2016/2016-1047/en_tema_1047.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Battistin, Erich & De Nadai, Michele & Vuri, Daniela, 2017. "Counting rotten apples: Student achievement and score manipulation in Italian elementary Schools," Journal of Econometrics, Elsevier, vol. 200(2), pages 344-362.
    2. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    3. Victor Chernozhukov & Iv·n Fern·ndez-Val & Alfred Galichon, 2010. "Quantile and Probability Curves Without Crossing," Econometrica, Econometric Society, vol. 78(3), pages 1093-1125, May.
    4. Andrew Bacher-Hicks & Thomas J. Kane & Douglas O. Staiger, 2014. "Validating Teacher Effect Estimates Using Changes in Teacher Assignments in Los Angeles," NBER Working Papers 20657, National Bureau of Economic Research, Inc.
    5. repec:hal:wpspec:info:hdl:2441/5rkqqmvrn4tl22s9mc4b6ga2g is not listed on IDEAS
    6. Bertoni, Marco & Brunello, Giorgio & Rocco, Lorenzo, 2013. "When the cat is near, the mice won't play: The effect of external examiners in Italian schools," Journal of Public Economics, Elsevier, vol. 104(C), pages 65-77.
    7. Gary Chamberlain, 1980. "Analysis of Covariance with Qualitative Data," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 47(1), pages 225-238.
    8. repec:spo:wpmain:info:hdl:2441/5rkqqmvrn4tl22s9mc4b6ga2g is not listed on IDEAS
    9. repec:hal:spmain:info:hdl:2441/5rkqqmvrn4tl22s9mc4b6ga2g is not listed on IDEAS
    10. Randall Reback & Julie Berry Cullen, 2006. "Tinkering toward accolades: School gaming under a performance accountability system," Working Papers 0601, Barnard College, Department of Economics.
    11. repec:spo:wpecon:info:hdl:2441/5rkqqmvrn4tl22s9mc4b6ga2g is not listed on IDEAS
    12. Stéphane Bonhomme, 2012. "Functional Differencing," Econometrica, Econometric Society, vol. 80(4), pages 1337-1385, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Brandolini Andrea & Carta Francesca, 2016. "Some Reflections on the Social Welfare Bases of the Measurement of Global Income Inequality," Journal of Globalization and Development, De Gruyter, vol. 7(1), pages 1-15, June.
    2. Palazzo, Francesco, 2017. "Search costs and the severity of adverse selection," Research in Economics, Elsevier, vol. 71(1), pages 171-197.
    3. Sergio Longobardi & Patrizia Falzetti & Margherita Maria Pagliuca, 2018. "Quis custiodet ipsos custodes? How to detect and correct teacher cheating in Italian student data," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 27(3), pages 515-543, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Seth Gershenson, 2016. "Performance Standards and Employee Effort: Evidence From Teacher Absences," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 35(3), pages 615-638, June.
    2. Seth Gershenson & Stephen B. Holt & Nicholas Papageorge, 2015. "Who Believes in Me? The Effect of Student-Teacher Demographic Match on Teacher Expectations," Upjohn Working Papers 15-231, W.E. Upjohn Institute for Employment Research.
    3. Aguirregabiria, Victor & Gu, Jiaying & Luo, Yao, 2021. "Sufficient statistics for unobserved heterogeneity in structural dynamic logit models," Journal of Econometrics, Elsevier, vol. 223(2), pages 280-311.
    4. Canales, Andrea & Maldonado, Luis, 2018. "Teacher quality and student achievement in Chile: Linking teachers' contribution and observable characteristics," International Journal of Educational Development, Elsevier, vol. 60(C), pages 33-50.
    5. Hinnerich, Björn Tyrefors & Vlachos, Jonas, 2017. "The impact of upper-secondary voucher school attendance on student achievement. Swedish evidence using external and internal evaluations," Labour Economics, Elsevier, vol. 47(C), pages 1-14.
    6. Santiago Pereda-Fernández, 2021. "Copula-Based Random Effects Models for Clustered Data," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(2), pages 575-588, March.
    7. Filmer,Deon P. & Nahata,Vatsal & Sabarwal,Shwetlena, 2021. "Preparation, Practice, and Beliefs : A Machine Learning Approach to Understanding Teacher Effectiveness," Policy Research Working Paper Series 9847, The World Bank.
    8. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Jun 2024.
    9. Chetty, Raj & Friedman, John & Rockoff, Jonah, 2015. "Measuring the Impacts of Teachers: Response to Rothstein (2014)," CEPR Discussion Papers 10768, C.E.P.R. Discussion Papers.
    10. Markus Nagler & Marc Piopiunik & Martin R. West, 2020. "Weak Markets, Strong Teachers: Recession at Career Start and Teacher Effectiveness," Journal of Labor Economics, University of Chicago Press, vol. 38(2), pages 453-500.
    11. Naven, Matthew, 2019. "Human-Capital Formation During Childhood and Adolescence: Evidence from School Quality and Postsecondary Success in California," MPRA Paper 97716, University Library of Munich, Germany.
    12. Julie Berry Cullen & Eric A. Hanushek & Gregory Phelan & Steven G. Rivkin, 2024. "Performance Information and Personnel Decisions in the Public Sector: The Case of School Principals," Journal of Human Resources, University of Wisconsin Press, vol. 59(1), pages 109-140.
    13. Bertoni, Marco & Brunello, Giorgio & De Benedetto, Marco Alberto & De Paola, Maria, 2019. "External Monitors and Score Manipulation in Italian Schools: Symptomatic Treatment or Cure?," IZA Discussion Papers 12591, Institute of Labor Economics (IZA).
    14. Julie Berry Cullen & Cory Koedel & Eric Parsons, 2021. "The Compositional Effect of Rigorous Teacher Evaluation on Workforce Quality," Education Finance and Policy, MIT Press, vol. 16(1), pages 7-41, Winter.
    15. Hugh Macartney & Robert McMillan & Uros Petronijevic, 2021. "A Quantitative Framework for Analyzing the Distributional Effects of Incentive Schemes," Working Papers tecipa-696, University of Toronto, Department of Economics.
    16. Renata Lemos & Karthik Muralidharan & Daniela Scur, 2024. "Personnel Management and School Productivity: Evidence from India," The Economic Journal, Royal Economic Society, vol. 134(661), pages 2071-2100.
    17. Peter Hull, 2018. "Estimating Treatment Effects in Mover Designs," Papers 1804.06721, arXiv.org.
    18. Hermann, Zoltán & Horváth, Hedvig, 2022. "Tanári eredményesség és tanár-diák összepárosítás az általános iskolákban. Empirikus mintázatok három magyarországi tankerület adatai alapján [Teacher effectiveness and teacher-student matching in ," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(11), pages 1377-1406.
    19. Dan Goldhaber & Cyrus Grout & Nick Huntington-Klein, 2017. "Screen Twice, Cut Once: Assessing the Predictive Validity of Applicant Selection Tools," Education Finance and Policy, MIT Press, vol. 12(2), pages 197-223, Spring.
    20. Godstime Osekhebhen Eigbiremolen, 2020. "Estimating Private School Premium for Primary School Children in Ethiopia: Evidence from Individual-level Panel Data," Progress in Development Studies, , vol. 20(1), pages 26-44, January.

    More about this item

    Keywords

    cheating correction; copula; nonlinear panel data; test scores manipulation;
    All these keywords.

    JEL classification:

    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models
    • C25 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Discrete Regression and Qualitative Choice Models; Discrete Regressors; Proportions; Probabilities
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bdi:wptemi:td_1047_16. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/bdigvit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.