Watching the watchmen: A statistical analysis of mark consistency across taught modules

Verifying that taught modules are marked and taught to a common standard is important but doing so by comparing mean module marks is inadequate when students’ ability is not uniform across these modules. For example, a module taken by a group of students of above average ability may justifiably result in a high mean mark, without implying that inconsistent standards have been applied. We propose a modified version of the fixed effects regression that provides direct estimates of module mark biases while conditioning for student composition and requiring no additional, potentially confidential, information on students or staff. We describe how this modified fixed effects regression can be implemented on a set of student marks and how the results can be interpreted. Increases in student numbers and tuition fees have increased the preoccupation with, and monitoring of, marks. We show how one can generate statistics that are more informative of the biases in marking, while being explicit about their limitations.