IDEAS home Printed from https://ideas.repec.org/a/spr/stmapp/v27y2018i3d10.1007_s10260-018-0426-2.html
   My bibliography  Save this article

Quis custiodet ipsos custodes? How to detect and correct teacher cheating in Italian student data

Author

Listed:
  • Sergio Longobardi

    (University of Naples “Parthenope”)

  • Patrizia Falzetti

    (Italian Institute for the Educational Evaluation of Instruction and Training (INVALSI))

  • Margherita Maria Pagliuca

    (University of Naples “Parthenope”)

Abstract

The increasing diffusion of standardized assessments of students’ competences has been accompanied by an increasing need to make reliable data available to all stakeholders of the educational system (policy makers, teachers, researchers, families and students). In this light, we propose a multistep approach to detect and correct teacher cheating, which decreases the quality of student data offered by the Italian Institute for the Educational Evaluation of Instruction and Training. Our method integrates the “mechanistic” logic of the fuzzy clustering technique with a statistical model-based approach, and it aims to improve the detection of cheating and to correct test scores at both the class and student level. The results show a normalization of the scores and a stronger correction on data for Southern regions, where the propensity to cheat appears to be highest.

Suggested Citation

  • Sergio Longobardi & Patrizia Falzetti & Margherita Maria Pagliuca, 2018. "Quis custiodet ipsos custodes? How to detect and correct teacher cheating in Italian student data," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 27(3), pages 515-543, August.
  • Handle: RePEc:spr:stmapp:v:27:y:2018:i:3:d:10.1007_s10260-018-0426-2
    DOI: 10.1007/s10260-018-0426-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10260-018-0426-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10260-018-0426-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Santiago Pereda Fernández, 2016. "A new method for the correction of test scores manipulation," Temi di discussione (Economic working papers) 1047, Bank of Italy, Economic Research and International Relations Area.
    2. Victor Lavy, 2009. "Performance Pay and Teachers' Effort, Productivity, and Grading Ethics," American Economic Review, American Economic Association, vol. 99(5), pages 1979-2011, December.
    3. Thomas Fuchs & Ludger Wößmann, 2008. "What accounts for international differences in student prformance? A re-examination using PISA data," Studies in Empirical Economics, in: Christian Dustmann & Bernd Fitzenberger & Stephen Machin (ed.), The Economics of Education and Training, pages 209-240, Springer.
    4. Rebecca Diamond & Petra Persson, 2016. "The Long-term Consequences of Teacher Discretion in Grading of High-stakes Tests," NBER Working Papers 22207, National Bureau of Economic Research, Inc.
    5. Marco Paccagnella & Paolo Sestito, 2014. "School cheating and social capital," Education Economics, Taylor & Francis Journals, vol. 22(4), pages 367-388, August.
    6. Brian A. Jacob & Steven D. Levitt, 2003. "Rotten Apples: An Investigation of the Prevalence and Predictors of Teacher Cheating," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(3), pages 843-877.
    7. Figlio, David N., 2006. "Testing, crime and punishment," Journal of Public Economics, Elsevier, vol. 90(4-5), pages 837-851, May.
    8. Roland Fryer & Steven Levitt & John List & Sally Sadoff, 2012. "Enhancing the Efficacy of Teacher Incentives through Loss Aversion: A Field Experiment," Framed Field Experiments 00591, The Field Experiments Website.
    9. Massimiliano Bratti & Daniele Checchi & Antonio Filippin, 2007. "Geographical Differences in Italian Students' Mathematical Competencies: Evidence from Pisa 2003," Giornale degli Economisti, GDE (Giornale degli Economisti e Annali di Economia), Bocconi University, vol. 66(3), pages 299-333, November.
    10. Agasisti, Tommaso & Longobardi, Sergio, 2014. "Inequality in education: Can Italian disadvantaged students close the gap?," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 52(C), pages 8-20.
    11. Sarah R. Cohodes, 2016. "Teaching to the Student: Charter School Effectiveness in Spite of Perverse Incentives," Education Finance and Policy, MIT Press, vol. 11(1), pages 1-42, Winter.
    12. César Martinelli & Susan W. Parker & Ana Cristina Pérez-Gea & Rodimiro Rodrigo, 2018. "Cheating and Incentives: Learning from a Policy Experiment," American Economic Journal: Economic Policy, American Economic Association, vol. 10(1), pages 298-325, February.
    13. Giorgio Brunello & Daniele Checchi, 2007. "Does school tracking affect equality of opportunity? New international evidence [‘Educational opportunities and the role of institutions’]," Economic Policy, CEPR, CESifo, Sciences Po;CES;MSH, vol. 22(52), pages 782-861.
    14. Thomas Ahn & Jacob Vigdor, 2014. "The Impact of No Child Left Behind's Accountability Sanctions on School Performance: Regression Discontinuity Evidence from North Carolina," NBER Working Papers 20511, National Bureau of Economic Research, Inc.
    15. Claudio Lucifora & Marco Tonello, 2020. "Monitoring and Sanctioning Cheating at School: What Works? Evidence from a National Evaluation Program," Journal of Human Capital, University of Chicago Press, vol. 14(4), pages 584-616.
    16. Jacob, Brian A., 2005. "Accountability, incentives and behavior: the impact of high-stakes testing in the Chicago Public Schools," Journal of Public Economics, Elsevier, vol. 89(5-6), pages 761-796, June.
    17. J. Gimenez-Nadal & Jose Molina, 2013. "Parents’ education as a determinant of educational childcare time," Journal of Population Economics, Springer;European Society for Population Economics, vol. 26(2), pages 719-749, April.
    18. Thomas S. Dee & Will Dobbie & Brian A. Jacob & Jonah Rockoff, 2019. "The Causes and Consequences of Test Score Manipulation: Evidence from the New York Regents Examinations," American Economic Journal: Applied Economics, American Economic Association, vol. 11(3), pages 382-423, July.
    19. Cristina Borra & Almudena Sevilla & Jonathan Gershuny, 2013. "Calibrating Time-Use Estimates for the British Household Panel Survey," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 114(3), pages 1211-1224, December.
    20. Bertoni, Marco & Brunello, Giorgio & Rocco, Lorenzo, 2013. "When the cat is near, the mice won't play: The effect of external examiners in Italian schools," Journal of Public Economics, Elsevier, vol. 104(C), pages 65-77.
    21. George Wesolowsky, 2000. "Detecting excessive similarity in answers on multiple choice exams," Journal of Applied Statistics, Taylor & Francis Journals, vol. 27(7), pages 909-921.
    22. Derek Neal & Diane Whitmore Schanzenbach, 2010. "Left Behind by Design: Proficiency Counts and Test-Based Accountability," The Review of Economics and Statistics, MIT Press, vol. 92(2), pages 263-283, May.
    23. Iftikhar Hussain, 2015. "Subjective Performance Evaluation in the Public Sector: Evidence from School Inspections," Journal of Human Resources, University of Wisconsin Press, vol. 50(1), pages 189-221.
    24. Zhao, Zhong, 2008. "Sensitivity of propensity score methods to the specifications," Economics Letters, Elsevier, vol. 98(3), pages 309-319, March.
    25. Lydia Mechtenberg, 2009. "Cheap Talk in the Classroom: How Biased Grading at School Explains Gender Differences in Achievements, Career Choices and Wages," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(4), pages 1431-1459.
    26. Iftikhar Hussain, 2012. "Subjective Performance Evaluation in the Public Sector: Evidence from School Inspections," CEE Discussion Papers 0135, Centre for the Economics of Education, LSE.
    27. Lucifora, Claudio & Tonello, Marco, 2015. "Cheating and social interactions. Evidence from a randomized experiment in a national evaluation program," Journal of Economic Behavior & Organization, Elsevier, vol. 115(C), pages 45-66.
    28. Samuel Kotz, 2006. "Handbook of Beta Distribution and Its Applications edited by A.K. Gupta and S. Nadarajah," Biometrics, The International Biometric Society, vol. 62(1), pages 309-310, March.
    29. Edward P. Lazear, 2006. "Speeding, Terrorism, and Teaching to the Test," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 121(3), pages 1029-1061.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Marco Bertoni & Giorgio Brunello & Marco Alberto De Benedetto & Maria De Paola, 2019. "External Monitors And Score Manipulation In Italian Schools: Symptomatic Treatment Or Cure?," Working Papers 201907, Università della Calabria, Dipartimento di Economia, Statistica e Finanza "Giovanni Anania" - DESF.
    2. Claudio Lucifora & Marco Tonello, 2020. "Monitoring and Sanctioning Cheating at School: What Works? Evidence from a National Evaluation Program," Journal of Human Capital, University of Chicago Press, vol. 14(4), pages 584-616.
    3. Marina Cavalieri & Massimo Finocchiaro Castro & Calogero Guccio, 2020. "Does the Fish Rot from the Head? Organised Crime and Educational Outcomes in Southern Italy," Working papers 97, Società Italiana di Economia Pubblica.
    4. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.
    5. Cuesta, José Ignacio & González, Felipe & Larroulet Philippi, Cristian, 2020. "Distorted quality signals in school markets," Journal of Development Economics, Elsevier, vol. 147(C).
    6. Lucifora, Claudio & Tonello, Marco, 2015. "Cheating and social interactions. Evidence from a randomized experiment in a national evaluation program," Journal of Economic Behavior & Organization, Elsevier, vol. 115(C), pages 45-66.
    7. Seth Gershenson, 2016. "Performance Standards and Employee Effort: Evidence From Teacher Absences," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 35(3), pages 615-638, June.
    8. Yue-Yi Hwa & Clare Leaver, 2021. "Management in education systems," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 37(2), pages 367-391.
    9. Ludger Wößmann, 2006. "Bildungspolitische Lehren aus den internationalen Schülertests: Wettbewerb, Autonomie und externe Leistungsüberprüfung," Perspektiven der Wirtschaftspolitik, Verein für Socialpolitik, vol. 7(3), pages 417-444, August.
    10. Battistin, Erich & De Nadai, Michele & Vuri, Daniela, 2017. "Counting rotten apples: Student achievement and score manipulation in Italian elementary Schools," Journal of Econometrics, Elsevier, vol. 200(2), pages 344-362.
    11. Buerger, Christian & Lincove, Jane Arnold & Mata, Catherine, 2023. "How context shapes the relationship between school autonomy and test-scores. An explanatory analysis using PISA 2015," International Journal of Educational Development, Elsevier, vol. 99(C).
    12. Bertoni, Marco & Brunello, Giorgio & Rocco, Lorenzo, 2013. "When the cat is near, the mice won't play: The effect of external examiners in Italian schools," Journal of Public Economics, Elsevier, vol. 104(C), pages 65-77.
    13. Cavalieri, Marina & Finocchiaro Castro, Massimo & Guccio, Calogero, 2023. "Organised crime and educational outcomes in Southern Italy: An empirical investigation," Socio-Economic Planning Sciences, Elsevier, vol. 89(C).
    14. Joshua D. Angrist & Erich Battistin & Daniela Vuri, 2017. "In a Small Moment: Class Size and Moral Hazard in the Italian Mezzogiorno," American Economic Journal: Applied Economics, American Economic Association, vol. 9(4), pages 216-249, October.
    15. Derek Neal, 2011. "The Design of Performance Pay in Education," NBER Working Papers 16710, National Bureau of Economic Research, Inc.
    16. Jones, Michael D., 2013. "Teacher behavior under performance pay incentives," Economics of Education Review, Elsevier, vol. 37(C), pages 148-164.
    17. Ulrik Hvidman & Hans Henrik Sievertsen, 2021. "High-Stakes Grades and Student Behavior," Journal of Human Resources, University of Wisconsin Press, vol. 56(3), pages 821-849.
    18. Gershenson, Seth & Holt, Stephen B. & Papageorge, Nicholas W., 2015. "Who Believes in Me? The Effect of Student-Teacher Demographic Match on Teacher Expectations," IZA Discussion Papers 9202, Institute of Labor Economics (IZA).
    19. Ferman, Bruno & Fontes, Luiz Felipe, 2022. "Assessing knowledge or classroom behavior? Evidence of teachers’ grading bias," Journal of Public Economics, Elsevier, vol. 216(C).
    20. Gian Paolo Barbetta & Giuseppe Sorrenti & Gilberto Turati, 2021. "Multigrading and Child Achievement," Journal of Human Resources, University of Wisconsin Press, vol. 56(3), pages 940-968.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stmapp:v:27:y:2018:i:3:d:10.1007_s10260-018-0426-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.