IDEAS home Printed from https://ideas.repec.org/p/bdi/wptemi/td_1047_16.html
   My bibliography  Save this paper

A new method for the correction of test scores manipulation

Author

Listed:
  • Santiago Pereda Fernández

    (Banca d’Italia)

Abstract

I propose a method to correct for test scores manipulation and apply it to a natural experiment in the Italian education system consisting in the random assignment of external monitors to classrooms. The empirical strategy is based on a likelihood approach, using nonlinear panel data methods to obtain clean estimates of cheating controlling for unobserved heterogeneity. The likelihood of each classroom's scores is later used to correct them for cheating. Cheating is not associated with an increase in the correlation of the answers after we control for mean test scores. The method produces estimates of manipulation more frequent in the South and Islands and among female students and immigrants in Italian tests. A simulation shows how the manipulation reduces the accuracy of an exam in reflecting students' knowledge, and the correction proposed in this paper makes up for about a half of this loss.

Suggested Citation

  • Santiago Pereda Fernández, 2016. "A new method for the correction of test scores manipulation," Temi di discussione (Economic working papers) 1047, Bank of Italy, Economic Research and International Relations Area.
    • Handle: RePEc:bdi:wptemi:td_1047_16
    as

    Download full text from publisher

    File URL: http://www.bancaditalia.it/pubblicazioni/temi-discussione/2016/2016-1047/en_tema_1047.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Victor Chernozhukov & Iv·n Fern·ndez-Val & Alfred Galichon, 2010. "Quantile and Probability Curves Without Crossing," Econometrica, Econometric Society, vol. 78(3), pages 1093-1125, May.
    2. repec:hal:wpspec:info:hdl:2441/5rkqqmvrn4tl22s9mc4b6ga2g is not listed on IDEAS
    3. Bertoni, Marco & Brunello, Giorgio & Rocco, Lorenzo, 2013. "When the cat is near, the mice won't play: The effect of external examiners in Italian schools," Journal of Public Economics, Elsevier, vol. 104(C), pages 65-77.
    4. Gary Chamberlain, 1980. "Analysis of Covariance with Qualitative Data," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 47(1), pages 225-238.
    5. Battistin, Erich & De Nadai, Michele & Vuri, Daniela, 2017. "Counting rotten apples: Student achievement and score manipulation in Italian elementary Schools," Journal of Econometrics, Elsevier, vol. 200(2), pages 344-362.
    6. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    7. Stéphane Bonhomme, 2012. "Functional Differencing," Econometrica, Econometric Society, vol. 80(4), pages 1337-1385, July.
    8. Julie Berry Cullen & Randall Reback, 2006. "Tinkering Toward Accolades: School Gaming Under a Performance Accountability System," NBER Working Papers 12286, National Bureau of Economic Research, Inc.
    9. repec:hal:spmain:info:hdl:2441/5rkqqmvrn4tl22s9mc4b6ga2g is not listed on IDEAS
    10. Andrew Bacher-Hicks & Thomas J. Kane & Douglas O. Staiger, 2014. "Validating Teacher Effect Estimates Using Changes in Teacher Assignments in Los Angeles," NBER Working Papers 20657, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Brandolini Andrea & Carta Francesca, 2016. "Some Reflections on the Social Welfare Bases of the Measurement of Global Income Inequality," Journal of Globalization and Development, De Gruyter, vol. 7(1), pages 1-15, June.
    2. Palazzo, Francesco, 2017. "Search costs and the severity of adverse selection," Research in Economics, Elsevier, vol. 71(1), pages 171-197.
    3. Sergio Longobardi & Patrizia Falzetti & Margherita Maria Pagliuca, 2018. "Quis custiodet ipsos custodes? How to detect and correct teacher cheating in Italian student data," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 27(3), pages 515-543, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Seth Gershenson, 2016. "Performance Standards and Employee Effort: Evidence From Teacher Absences," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 35(3), pages 615-638, June.
    2. Gershenson, Seth & Holt, Stephen B. & Papageorge, Nicholas W., 2015. "Who Believes in Me? The Effect of Student-Teacher Demographic Match on Teacher Expectations," IZA Discussion Papers 9202, Institute of Labor Economics (IZA).
    3. Markus Nagler & Marc Piopiunik & Martin R. West, 2020. "Weak Markets, Strong Teachers: Recession at Career Start and Teacher Effectiveness," Journal of Labor Economics, University of Chicago Press, vol. 38(2), pages 453-500.
    4. Aguirregabiria, Victor & Gu, Jiaying & Luo, Yao, 2021. "Sufficient statistics for unobserved heterogeneity in structural dynamic logit models," Journal of Econometrics, Elsevier, vol. 223(2), pages 280-311.
    5. Canales, Andrea & Maldonado, Luis, 2018. "Teacher quality and student achievement in Chile: Linking teachers' contribution and observable characteristics," International Journal of Educational Development, Elsevier, vol. 60(C), pages 33-50.
    6. Hinnerich, Björn Tyrefors & Vlachos, Jonas, 2017. "The impact of upper-secondary voucher school attendance on student achievement. Swedish evidence using external and internal evaluations," Labour Economics, Elsevier, vol. 47(C), pages 1-14.
    7. Santiago Pereda-Fernández, 2021. "Copula-Based Random Effects Models for Clustered Data," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(2), pages 575-588, March.
    8. Marco Bertoni & Giorgio Brunello & Marco Alberto De Benedetto & Maria De Paola, 2019. "External Monitors And Score Manipulation In Italian Schools: Symptomatic Treatment Or Cure?," Working Papers 201907, Università della Calabria, Dipartimento di Economia, Statistica e Finanza "Giovanni Anania" - DESF.
    9. Cavalieri, Marina & Finocchiaro Castro, Massimo & Guccio, Calogero, 2023. "Organised crime and educational outcomes in Southern Italy: An empirical investigation," Socio-Economic Planning Sciences, Elsevier, vol. 89(C).
    10. Joshua D. Angrist & Erich Battistin & Daniela Vuri, 2017. "In a Small Moment: Class Size and Moral Hazard in the Italian Mezzogiorno," American Economic Journal: Applied Economics, American Economic Association, vol. 9(4), pages 216-249, October.
    11. Marina Cavalieri & Massimo Finocchiaro Castro & Calogero Guccio, 2020. "Does the Fish Rot from the Head? Organised Crime and Educational Outcomes in Southern Italy," Working papers 97, Società Italiana di Economia Pubblica.
    12. Erich Battistin, 2016. "How manipulating test scores affects school accountability and student achievement," IZA World of Labor, Institute of Labor Economics (IZA), pages 295-295, September.
    13. Filmer,Deon P. & Nahata,Vatsal & Sabarwal,Shwetlena, 2021. "Preparation, Practice, and Beliefs : A Machine Learning Approach to Understanding Teacher Effectiveness," Policy Research Working Paper Series 9847, The World Bank.
    14. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Mar 2024.
    15. Bruhn, Jesse & Imberman, Scott & Winters, Marcus, 2022. "Regulatory arbitrage in teacher hiring and retention: Evidence from Massachusetts Charter Schools," Journal of Public Economics, Elsevier, vol. 215(C).
    16. Chetty, Raj & Friedman, John & Rockoff, Jonah, 2015. "Measuring the Impacts of Teachers: Response to Rothstein (2014)," CEPR Discussion Papers 10768, C.E.P.R. Discussion Papers.
    17. Naven, Matthew, 2019. "Human-Capital Formation During Childhood and Adolescence: Evidence from School Quality and Postsecondary Success in California," MPRA Paper 97716, University Library of Munich, Germany.
    18. Julie Berry Cullen & Eric A. Hanushek & Gregory Phelan & Steven G. Rivkin, 2024. "Performance Information and Personnel Decisions in the Public Sector: The Case of School Principals," Journal of Human Resources, University of Wisconsin Press, vol. 59(1), pages 109-140.
    19. Julie Berry Cullen & Cory Koedel & Eric Parsons, 2021. "The Compositional Effect of Rigorous Teacher Evaluation on Workforce Quality," Education Finance and Policy, MIT Press, vol. 16(1), pages 7-41, Winter.
    20. Milla, J. & San Martin , E. & Van Bellegem, S., 2015. "Higher education value added using multiple outcomes," LIDAM Discussion Papers CORE 2015045, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).

    More about this item

    Keywords

    cheating correction; copula; nonlinear panel data; test scores manipulation;
    All these keywords.

    JEL classification:

    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models
    • C25 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Discrete Regression and Qualitative Choice Models; Discrete Regressors; Proportions; Probabilities
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bdi:wptemi:td_1047_16. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/bdigvit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.