Balanced versus Randomized Field Experiments in Economics: Why W. S. Gosset aka "Student" Matters

Over the past decade randomized field experiments have gained prominence in the toolkit of empirical economics and policy making. In an article titled "Field Experiments in Economics: The Past, the Present, and the Future," Levitt and List (2009) make three main claims about the history, philosophy, and future of field experiments in economics. (1) They claim that field experiments in economics began in the 1920s and 1930s in agricultural work by Neyman and Fisher. (2) They claim that artificial randomization is essential for good experimental design because, they claim, randomization is the only valid justification for Student's test of significance. (3) They claim that decision-making in private sector firms will be advanced by partnering with economists doing randomized experiments. Several areas of research have been influenced by the article despite the absence of historical and methodological review. This paper seeks to fill that gap in the literature. The power and efficiency of balanced over random designs — discovered by William S. Gosset aka Student, and confirmed by Pearson, Neyman, Jeffreys, and others adopting a balanced, decision-theoretic and/or Bayesian approach to experiments — is not mentioned in the Levitt and List article. Neglect of Student is regrettable. A body of evidence descending from Student (1911) and extending to Heckman and Vytlacil (2007) suggests that artificial randomization is neither necessary nor sufficient for improving efficiency, identifying causal relationships, and discovering economically significant differences. Three easy ways to improve field experiments are proposed and briefly illustrated.