IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp8405.html
   My bibliography  Save this paper

Counting Rotten Apples: Student Achievement and Score Manipulation in Italian Elementary Schools

Author

Listed:
  • Battistin, Erich

    () (University of Maryland)

  • De Nadai, Michele

    () (University of Padova)

  • Vuri, Daniela

    () (University of Rome Tor Vergata)

Abstract

We derive bounds for the average of math and language scores of elementary school students in Italy correcting for pervasive score manipulation. Information on the fraction of manipulated data is retrieved from a natural experiment that randomly assigns external monitors to schools. We show how bounds can be tightened imposing restrictions on the measurement properties of the manipulation indicator developed by the government agency charged with test administration and data collection. We additionally assume that manipulation is more likely in those classes at the lower end of the distribution of true scores. Our results show that regional rankings by academic performance are reversed once manipulation is properly taken into account.

Suggested Citation

  • Battistin, Erich & De Nadai, Michele & Vuri, Daniela, 2014. "Counting Rotten Apples: Student Achievement and Score Manipulation in Italian Elementary Schools," IZA Discussion Papers 8405, Institute of Labor Economics (IZA).
  • Handle: RePEc:iza:izadps:dp8405
    as

    Download full text from publisher

    File URL: http://ftp.iza.org/dp8405.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Anders Böhlmark & Mikael Lindahl, 2015. "Independent Schools and Long-run Educational Outcomes: Evidence from Sweden's Large-scale Voucher Reform," Economica, London School of Economics and Political Science, vol. 82(327), pages 508-551, July.
    2. Andrea Ichino & Giovanni Maggi, 2000. "Work Environment and Individual Background: Explaining Regional Shirking Differentials in a Large Italian Firm," The Quarterly Journal of Economics, Oxford University Press, vol. 115(3), pages 1057-1090.
    3. Joshua D. Angrist & Parag A. Pathak & Christopher R. Walters, 2013. "Explaining Charter School Effectiveness," American Economic Journal: Applied Economics, American Economic Association, vol. 5(4), pages 1-27, October.
    4. Aigner, Dennis J., 1973. "Regression with a binary independent variable subject to errors of observation," Journal of Econometrics, Elsevier, vol. 1(1), pages 49-59, March.
    5. Joshua D. Angrist & Erich Battistin & Daniela Vuri, 2014. "In a Small Moment: Class Size and Moral Hazard in the Mezzogiorno," FBK-IRVAPP Working Papers 2014-04, Research Institute for the Evaluation of Public Policies (IRVAPP), Bruno Kessler Foundation.
    6. Rebecca Diamond & Petra Persson, 2016. "The Long-term Consequences of Teacher Discretion in Grading of High-stakes Tests," NBER Working Papers 22207, National Bureau of Economic Research, Inc.
    7. Aviv Nevo & Adam M. Rosen, 2012. "Identification With Imperfect Instruments," The Review of Economics and Statistics, MIT Press, vol. 94(3), pages 659-671, August.
    8. Brian A. Jacob & Steven D. Levitt, 2003. "Rotten Apples: An Investigation of the Prevalence and Predictors of Teacher Cheating," The Quarterly Journal of Economics, Oxford University Press, vol. 118(3), pages 843-877.
    9. Brent Kreider & John V. Pepper, 2011. "Identification of Expected Outcomes in a Data Error Mixing Model With Multiplicative Mean Independence," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(1), pages 49-60, January.
    10. Nicoletti, Cheti & Peracchi, Franco & Foliano, Francesca, 2011. "Estimating Income Poverty in the Presence of Missing Data and Measurement Error," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(1), pages 61-72.
    11. Battistin, Erich & De Nadai, Michele & Sianesi, Barbara, 2014. "Misreported schooling, multiple measures and returns to educational qualifications," Journal of Econometrics, Elsevier, vol. 181(2), pages 136-150.
    12. Battistin, Erich & Meroni, Elena Claudia, 2016. "Should we increase instruction time in low achieving schools? Evidence from Southern Italy," Economics of Education Review, Elsevier, vol. 55(C), pages 39-56.
    13. Arthur Lewbel, 2007. "Estimation of Average Treatment Effects with Misclassification," Econometrica, Econometric Society, vol. 75(2), pages 537-551, March.
    14. Horowitz, Joel L & Manski, Charles F, 1995. "Identification and Robustness with Contaminated and Corrupted Data," Econometrica, Econometric Society, vol. 63(2), pages 281-302, March.
    15. Thomas J. Kane & Cecilia Elena Rouse & Douglas Staiger, 1999. "Estimating Returns to Schooling When Schooling is Misreported," NBER Working Papers 7235, National Bureau of Economic Research, Inc.
    16. Tommaso Nannicini & Andrea Stella & Guido Tabellini & Ugo Troiano, 2013. "Social Capital and Political Accountability," American Economic Journal: Economic Policy, American Economic Association, vol. 5(2), pages 222-250, May.
    17. Bratti, Massimiliano & Checchi, Daniele & Filippin, Antonio, 2007. "Territorial Differences in Italian Students’ Mathematical Competencies: Evidence from PISA 2003," IZA Discussion Papers 2603, Institute of Labor Economics (IZA).
    18. Hahn, Jinyong, 1996. "A Note on Bootstrapping Generalized Method of Moments Estimators," Econometric Theory, Cambridge University Press, vol. 12(1), pages 187-197, March.
    19. Kreider, Brent & Pepper, John V., 2007. "Disability and Employment: Reevaluating the Evidence in Light of Reporting Errors," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 432-441, June.
    20. Luigi Guiso & Paola Sapienza & Luigi Zingales, 2004. "The Role of Social Capital in Financial Development," American Economic Review, American Economic Association, vol. 94(3), pages 526-556, June.
    21. Guido W. Imbens & Charles F. Manski, 2004. "Confidence Intervals for Partially Identified Parameters," Econometrica, Econometric Society, vol. 72(6), pages 1845-1857, November.
    22. Brent Kreider & John V. Pepper & Craig Gundersen & Dean Jolliffe, 2012. "Identifying the Effects of SNAP (Food Stamps) on Child Health Outcomes When Participation Is Endogenous and Misreported," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(499), pages 958-975, September.
    23. Charles F. Manski & John V. Pepper, 2000. "Monotone Instrumental Variables, with an Application to the Returns to Schooling," Econometrica, Econometric Society, vol. 68(4), pages 997-1012, July.
    24. Hu, Yingyao, 2008. "Identification and estimation of nonlinear models with misclassification error using instrumental variables: A general solution," Journal of Econometrics, Elsevier, vol. 144(1), pages 27-61, May.
    25. Xiaohong Chen & Han Hong & Denis Nekipelov, 2011. "Nonlinear Models of Measurement Errors," Journal of Economic Literature, American Economic Association, vol. 49(4), pages 901-937, December.
    26. Thomas S. Dee & Will Dobbie & Brian A. Jacob & Jonah Rockoff, 2019. "The Causes and Consequences of Test Score Manipulation: Evidence from the New York Regents Examinations," American Economic Journal: Applied Economics, American Economic Association, vol. 11(3), pages 382-423, July.
    27. Bertoni, Marco & Brunello, Giorgio & Rocco, Lorenzo, 2013. "When the cat is near, the mice won't play: The effect of external examiners in Italian schools," Journal of Public Economics, Elsevier, vol. 104(C), pages 65-77.
    28. Molinari, Francesca, 2008. "Partial identification of probability distributions with misclassified data," Journal of Econometrics, Elsevier, vol. 144(1), pages 81-117, May.
    29. Luigi Cannari & Francesco Nucci & Paolo Sestito, 2000. "Geographic labour mobility and the cost of housing: evidence from Italy," Applied Economics, Taylor & Francis Journals, vol. 32(14), pages 1899-1906.
    30. Aprajit Mahajan, 2006. "Identification and Estimation of Regression Models with Misclassification," Econometrica, Econometric Society, vol. 74(3), pages 631-665, May.
    31. Thomas J. Kane & Cecilia Rouse & Douglas Staiger, 1999. "Estimating Returns to Schooling When Schooling is Misreported," Working Papers 798, Princeton University, Department of Economics, Industrial Relations Section..
    32. Abadie A., 2002. "Bootstrap Tests for Distributional Treatment Effects in Instrumental Variable Models," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 284-292, March.
    33. Robert P. Sherman & Jeff Dominitz, 2006. "Identification and estimation of bounds on school performance measures: a nonparametric analysis of a mixture model with verification," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(8), pages 1295-1326.
    34. Di Addario, Sabrina & Patacchini, Eleonora, 2008. "Wages and the City. Evidence from Italy," Labour Economics, Elsevier, vol. 15(5), pages 1040-1061, October.
    35. Edward Vytlacil, 2002. "Independence, Monotonicity, and Latent Index Models: An Equivalence Result," Econometrica, Econometric Society, vol. 70(1), pages 331-341, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Claudio Lucifora & Marco Tonello, 2016. "Monitoring and sanctioning cheating at school: What works? Evidence from a national evaluation program," DISCE - Working Papers del Dipartimento di Economia e Finanza def051, Università Cattolica del Sacro Cuore, Dipartimenti e Istituti di Scienze Economiche (DISCE).
    2. Joshua D. Angrist & Erich Battistin & Daniela Vuri, 2017. "In a Small Moment: Class Size and Moral Hazard in the Italian Mezzogiorno," American Economic Journal: Applied Economics, American Economic Association, vol. 9(4), pages 216-249, October.
    3. Santiago Pereda Fernández, 2016. "A new method for the correction of test scores manipulation," Temi di discussione (Economic working papers) 1047, Bank of Italy, Economic Research and International Relations Area.
    4. Martin Gustafsson & Carol Nuga Deliwe, 2017. "Rotten apples or just apples and pears? Understanding patterns consistent with cheating in international test data," Working Papers 17/2017, Stellenbosch University, Department of Economics.
    5. Erich Battistin, 2016. "How manipulating test scores affects school accountability and student achievement," IZA World of Labor, Institute of Labor Economics (IZA), pages 295-295, September.
    6. Bertoni, Marco & Brunello, Giorgio & De Benedetto, Marco Alberto & De Paola, Maria, 2019. "External Monitors and Score Manipulation in Italian Schools: Symptomatic Treatment or Cure?," IZA Discussion Papers 12591, Institute of Labor Economics (IZA).

    More about this item

    Keywords

    corrupt sampling; measurement error; nonparametric bounds; partial identification;

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C31 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models; Quantile Regressions; Social Interaction Models
    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp8405. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Holger Hinte). General contact details of provider: http://www.iza.org .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.