Using "Cheat Sheets" to Distinguish Ability from Knowledge: Evidence from a Randomized Control Trial in Chile

According to the existing evidence some higher education admission tests may be screening out students who, despite a relative lack of specific knowledge, possess as much intellectual ability as their peers. If this is the case, students from disadvantaged socioeconomic backgrounds are likely to be disproportionately affected, since they generally receive a primary and secondary education of worse quality than their better-off peers, often resulting in significant knowledge gaps. Also, although in some cases these formative shortcomings might be too large to be feasibly addressed at the time of enrollment in higher education, it is plausible to think that in some cases they may perhaps be relatively easy to remedy. In view of all this, in this paper I present a diagnostics experiment, aimed at helping to better understand this issue. In particular, I custom-designed a multiple-choice test, intended to measure an individual's mathematical ability, while minimizing the reliance on previously acquired knowledge. Also, I put together a two page "cheat sheet", which outlined all the necessary concepts to successfully complete the exam, without providing any explicit answers. This test was subsequently used to evaluate the candidates applying for admission into a special access program at one of the leading Chilean universities. A staged randomized control trial was used to measure the difference in academic performance (i.e. number of correctly answered questions) across the three parts of the exam between students who received a "cheat sheet" after the first or second parts of the test, respectively. As expected, "cheat sheets" improved the average performance of candidates on the exam, but their impact varied considerably across individuals. Most importantly, "cheat sheets" proved significantly more beneficial (in terms of improved test performance) to those students who were more likely to have had a secondary education of lower quality. This result has important implications for educational policies in Chile and elsewhere, suggesting that a transition to ability-focused admission tests would facilitate the access to higher education for talented students from disadvantaged backgrounds.