Large-Scale Social Experimentation in the United States

Randomized controlled experimentation has a long history. But the transition from statistical theory to statistical practice remains a difficult one, especially when the experiments are large in scale. In this paper, we review several major randomized social experiments carried out in the United States beginning in the early 1970s. We describe briefly the policy settings out of which these experiments grew, and we indicate how this type of experimentation fits into the historical development of randomized experimentation more generally. We give detailed design features for five different sets of social experiments, and we briefly summarize some of the analyses of the experimental data. We end the paper with a discussion of analytical strategies for social experiments, and possible alternatives to large-scale experimentation, including combining the results of several experiments, and evolutionary designs. Throughout, we place special emphasis on the possible roles of models and of social and economic theory in both the design and the analysis of experiments, and we describe several open research problems requiring the attention of mathematical statisticians and economists.