IDEAS home Printed from https://ideas.repec.org/p/ucl/cepeow/25-12.html

Measuring systematic gaps in teacher judgement: A new approach

Author

Listed:
  • Oliver Cassagneau-Francis

    (UCL Centre for Education Policy and Equalising Opportunities)

  • Lindsey Macmillan

    (UCL Centre for Education Policy and Equalising Opportunities)

  • Richard Murphy

    (Department of Economics, University of Texas at Austin)

  • Gill Wyness

    (UCL Centre for Education Policy and Equalising Opportunities)

Abstract

We propose a new approach to test for systematic biases in teacher evaluations. We exploit a setting where teachers were required to assign students both grades and rankings within each grade. Comparing students immediately adjacent to grade boundaries, we apply a local randomization approach to estimate imbalance in student characteristics. Our findings reveal systematic bias favoring higher income and female students. These grading decisions carry real consequences: students just above the grade threshold are significantly more likely to attend university. Our approach can be applied whenever there is a system with many thresholds and subjective rankings.

Suggested Citation

  • Oliver Cassagneau-Francis & Lindsey Macmillan & Richard Murphy & Gill Wyness, 2025. "Measuring systematic gaps in teacher judgement: A new approach," CEPEO Working Paper Series 25-12, UCL Centre for Education Policy and Equalising Opportunities, revised Oct 2025.
  • Handle: RePEc:ucl:cepeow:25-12
    as

    Download full text from publisher

    File URL: https://repec-cepeo.ucl.ac.uk/cepeow/cepeowp25-12.pdf
    File Function: Initial version, 2025
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Richard Murphy & Gill Wyness, 2020. "Minority report: the impact of predicted grades on university admissions of disadvantaged groups," Education Economics, Taylor & Francis Journals, vol. 28(4), pages 333-350, July.
    2. Holbein, John B. & Ladd, Helen F., 2017. "Accountability pressure: Regression discontinuity estimates of how No Child Left Behind influenced student behavior," Economics of Education Review, Elsevier, vol. 58(C), pages 55-67.
    3. Hinnerich, Björn Tyrefors & Höglin, Erik & Johannesson, Magnus, 2011. "Are boys discriminated in Swedish high schools?," Economics of Education Review, Elsevier, vol. 30(4), pages 682-690, August.
    4. Eric Chyn & Brigham Frandsen & Emily Leslie, 2025. "Examiner and Judge Designs in Economics: A Practitioner's Guide," Journal of Economic Literature, American Economic Association, vol. 63(2), pages 401-439, June.
    5. Tim Kautz & James J. Heckman & Ron Diris & Bas ter Weel & Lex Borghans, 2014. "Fostering and Measuring Skills: Improving Cognitive and Non-cognitive Skills to Promote Lifetime Success," OECD Education Working Papers 110, OECD Publishing.
    6. Fernando Botelho & Ricardo Madeira, Marcos A. Rangel, 2015. "Racial Discrimination in Grading: Evidence from Brazil," Working Papers, Department of Economics 2015_04, University of São Paulo (FEA-USP).
    7. Graetz, Georg & Karimi, Arizo, 2022. "Gender gap variation across assessment types: Explanations and implications," Economics of Education Review, Elsevier, vol. 91(C).
    8. Curtis R. Taylor & Huseyin Yildirim, 2011. "Subjective Performance and the Value of Blind Evaluation," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 78(2), pages 762-794.
    9. Lavy, Victor & Sand, Edith, "undated". "On The Origins of Gender Human Capital Gaps: Short and Long Term Consequences of Teachers’ Stereotypical Biases," Economic Research Papers 270009, University of Warwick - Department of Economics.
    10. Sarena Goodman, 2016. "Learning from the Test: Raising Selective College Enrollment by Providing Information," The Review of Economics and Statistics, MIT Press, vol. 98(4), pages 671-684, October.
    11. Simon Burgess & Ellen Greaves, 2013. "Test Scores, Subjective Assessment, and Stereotyping of Ethnic Minorities," Journal of Labor Economics, University of Chicago Press, vol. 31(3), pages 535-576.
    12. Vincenzo Galasso & Paola Profeta, 2024. "Gender Differences in Math Tests: the Role of Time Pressure," The Economic Journal, Royal Economic Society, vol. 134(664), pages 3461-3475.
    13. Victor Lavy & Rigissa Megalokonomou, 2024. "The Short- and the Long-Run Impact of Gender-Biased Teachers," American Economic Journal: Applied Economics, American Economic Association, vol. 16(2), pages 176-218, April.
    14. Zhu, Maria, 2024. "New Findings on Racial Bias in Teachers' Evaluations of Student Achievement," IZA Discussion Papers 16815, IZA Network @ LISER.
    15. Ichino, Andrea & Polo, Michele & Rettore, Enrico, 2003. "Are judges biased by labor market conditions?," European Economic Review, Elsevier, vol. 47(5), pages 913-944, October.
    16. Terrier, Camille, 2020. "Boys lag behind: How teachers’ gender biases affect student achievement," Economics of Education Review, Elsevier, vol. 77(C).
    17. Lavy, Victor, 2008. "Do gender stereotypes reduce girls' or boys' human capital outcomes? Evidence from a natural experiment," Journal of Public Economics, Elsevier, vol. 92(10-11), pages 2083-2105, October.
    18. repec:hal:pseose:halshs-01379340 is not listed on IDEAS
    19. Burgess, Simon & Hauberg, Daniel Sloth & Rangvid, Beatrice Schindler & Sievertsen, Hans Henrik, 2022. "The importance of external assessments: High school math and gender gaps in STEM degrees," Economics of Education Review, Elsevier, vol. 88(C).
    20. John N. Friedman & Bruce Sacerdote & Douglas O. Staiger & Michele Tine, 2025. "Standardized Test Scores and Academic Performance at Ivy Plus Colleges," AEA Papers and Proceedings, American Economic Association, vol. 115, pages 676-681, May.
    21. Fernando Botelho & Ricardo A. Madeira & Marcos A. Rangel, 2015. "Racial Discrimination in Grading: Evidence from Brazil," American Economic Journal: Applied Economics, American Economic Association, vol. 7(4), pages 37-52, October.
    22. Michela Carlana, 2019. "Implicit Stereotypes: Evidence from Teachers’ Gender Bias," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(3), pages 1163-1224.
    23. Oliver Cassagneau-Francis & Gillian Wyness, 2025. "The merits of teacher assessment versus external exams to measure student achievement," World of Labour, LISER, pages 517-517, June.
    24. Alberto Alesina & Michela Carlana & Eliana La Ferrara & Paolo Pinotti, 2024. "Revealing Stereotypes: Evidence from Immigrants in Schools," American Economic Review, American Economic Association, vol. 114(7), pages 1916-1948, July.
    25. Eliana Avitzour & Adi Choen & Daphna Joel & Victor Lavy, 2020. "On the Origins of Gender-Biased Behavior: The Role of Explicit and Implicit Stereotypes," NBER Working Papers 27818, National Bureau of Economic Research, Inc.
    26. Gill Wyness & Lindsey Macmillan & Jake Anders & Catherine Dilnot, 2023. "Grade expectations: how well can past performance predict future grades?," Education Economics, Taylor & Francis Journals, vol. 31(4), pages 397-418, July.
    27. Thomas Breda & Mélina Hillion, 2016. "Teaching accreditation exams reveal grading biases favor women in male-dominated disciplines in France," PSE-Ecole d'économie de Paris (Postprint) halshs-01379340, HAL.
    28. John N. Friedman & Bruce Sacerdote & Douglas O. Staiger & Michele Tine, 2025. "Standardized Test Scores and Academic Performance at Ivy-Plus Colleges," NBER Working Papers 33570, National Bureau of Economic Research, Inc.
    29. Andreu Arenas & Caterina Calsamiglia, 2025. "Gender Differences in High-Stakes Performance and College Admission Policies," Management Science, INFORMS, vol. 71(10), pages 8413-8429, October.
    30. Phelps, Edmund S, 1972. "The Statistical Theory of Racism and Sexism," American Economic Review, American Economic Association, vol. 62(4), pages 659-661, September.
    31. Alma Cohen & Crystal S. Yang, 2019. "Judicial Politics and Sentencing Decisions," American Economic Journal: Economic Policy, American Economic Association, vol. 11(1), pages 160-191, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Burn, Hester & Fumagalli, Laura & Rabe, Birgitta, 2024. "Stereotyping and ethnicity gaps in teacher assigned grades," Labour Economics, Elsevier, vol. 89(C).
    2. Ferman, Bruno & Fontes, Luiz Felipe, 2020. "Discriminating Behavior: Evidence from teachers’ grading bias," MPRA Paper 100400, University Library of Munich, Germany.
    3. Leckie, G. & Maragkou, K., 2024. "Tracing the Origins of Gender Bias in Teacher Grades," Cambridge Working Papers in Economics 2457, Faculty of Economics, University of Cambridge.
    4. Delaney, Judith & Devereux, Paul J., 2023. "Gender Differences in Teacher Judgement of Comparative Advantage," CEPR Discussion Papers 18647, C.E.P.R. Discussion Papers.
    5. Huong Thu Le & Ha Trong Nguyen, 2018. "The evolution of the gender test score gap through seventh grade: new insights from Australia using unconditional quantile regression and decomposition," IZA Journal of Labor Economics, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 7(1), pages 1-42, December.
    6. Victor Lavy & Rigissa Megalokonomou, 2019. "Persistency in Teachers’ Grading Bias and Effects on Longer-Term Outcomes: University Admissions Exams and Choice of Field of Study," NBER Working Papers 26021, National Bureau of Economic Research, Inc.
    7. Nicoletti, Cheti & Sevilla, Almudena & Tonei, Valentina, 2022. "Gender Stereotypes in the Family," IZA Discussion Papers 15773, IZA Network @ LISER.
    8. Ferman, Bruno & Fontes, Luiz Felipe, 2022. "Assessing knowledge or classroom behavior? Evidence of teachers’ grading bias," Journal of Public Economics, Elsevier, vol. 216(C).
    9. Thomas van Huizen, 2021. "Teacher bias or measurement error bias? Evidence from track recommendations," Working Papers 2113, Utrecht School of Economics.
    10. Nicole Black & Sonja C. de New, 2020. "Short, Heavy and Underrated? Teacher Assessment Biases by Children's Body Size," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 82(5), pages 961-987, October.
    11. Bart H. H. Golsteyn & Cécile Magnée & Inge Wolf, 2024. "Did Migrant Children Benefit from a Delay in the Dutch Primary School Exit Test?," De Economist, Springer, vol. 172(3), pages 141-166, September.
    12. Lavy, Victor & Sand, Edith, 2018. "On the origins of gender gaps in human capital: Short- and long-term consequences of teachers' biases," Journal of Public Economics, Elsevier, vol. 167(C), pages 263-279.
    13. Bredtmann, Julia & Otten, Sebastian & Vonnahme, Christina, 2024. "Discrimination in grading? Evidence on teachers' evaluation bias towards minority students," Ruhr Economic Papers 1122, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    14. Thomas van Huizen & Madelon Jacobs & Matthijs Oosterveen, 2024. "Teacher bias or measurement error?," Papers 2401.04200, arXiv.org, revised Sep 2025.
    15. Lavy, Victor & Sand, Edith, "undated". "On The Origins of Gender Human Capital Gaps: Short and Long Term Consequences of Teachers’ Stereotypical Biases," Economic Research Papers 270009, University of Warwick - Department of Economics.
    16. Elisa Baldazzi & Pietro Biroli & Marina Della Giusta & Florent Dubois, 2025. "Seeing Stereotypes," Papers 2503.02146, arXiv.org.
    17. Michela Carlana & Eliana La Ferrara & Paolo Pinotti, 2022. "Goals and Gaps: Educational Careers of Immigrant Children," Econometrica, Econometric Society, vol. 90(1), pages 1-29, January.
    18. Marisa Bucheli & Florencia Amábile & Carmen Estrades, 2024. "Gender differences in teachers' assessments and blind test results – evidence from Uruguay," Documentos de Trabajo (working papers) 0324, Department of Economics - dECON.
    19. Alberto Alesina & Michela Carlana & Eliana La Ferrara & Paolo Pinotti, 2024. "Revealing Stereotypes: Evidence from Immigrants in Schools," American Economic Review, American Economic Association, vol. 114(7), pages 1916-1948, July.
    20. Carlos J. Gil-Hernández & Mar C. Espadafor, 2024. "An Elephant in the Classroom: Teacher Bias by Student SES or Ability Measurement Bias?," Econometrics Working Papers Archive 2024_05, Universita' degli Studi di Firenze, Dipartimento di Statistica, Informatica, Applicazioni "G. Parenti".

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C10 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - General
    • C25 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Discrete Regression and Qualitative Choice Models; Discrete Regressors; Proportions; Probabilities
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • I24 - Health, Education, and Welfare - - Education - - - Education and Inequality
    • J15 - Labor and Demographic Economics - - Demographic Economics - - - Economics of Minorities, Races, Indigenous Peoples, and Immigrants; Non-labor Discrimination

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ucl:cepeow:25-12. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Jake Anders (email available below). General contact details of provider: https://edirc.repec.org/data/epucluk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.