Cultural Context in Standardized Tests

We report results from a ï¬ eld experiment on cultural context in standardized tests among 6th- and 8th-grade school students in Australia. The National Assessment Program Literacy and Numeracy (NAPLAN) is a series of basicskills tests given to Australian students. In our experiment, 1135 students in Dubbo – a regional area in the North-Western part of the state of New South Wales – were randomly assigned to either a regular NAPLAN test or a contextualized test designed speciï¬ cally for this experiment by the NSW Aboriginal Education Consultative Group — a not-for-proï¬ t Aboriginal organization. The contextualized test was speciï¬ cally designed to mimic the regular test, but adapted to the local context of Dubbo. We evaluate effects on tests scores in numeracy for grades 6 and 8, and reading for grade 6. In numeracy, we do not ï¬ nd robust evidence of an impact on test scores. In reading, we ï¬ nd qualitatively large effects. The average treatment effect for reading is 0.27 s.d., with higher effects for Indigenous students (0.30 s.d.) than non-Indigenous students (0.24 s.d.) Together these results imply that cultural context may be important for performance on certain types of basic-skills tests.