Partial identification, distributional preferences, and the welfare ranking of policies

Many methodological debates in microeconometrics are driven by the tension between ``what we can get'' (identification) and ``what we want'' (parameters of interest). This paper proposes to consider models of policy choice which allow for a joint formal discussion of both issues. We consider a non-standard empirical object of interest, the ranking of counterfactual policies. This paper connects the literatures on partial identification and on ambiguity, where partially identified policy rankings are formally analogous to choice under Knightian uncertainty. Partial identification of conditional average treatment effects maps into a partial ordering of treatment assignment policies in terms of social welfare. This paper gives geometric characterizations of the identified partial ordering of policies, and derives conditions for restricted policy sets to be completely ordered or completely unordered. Such conditions map sets of feasible policies into requirements on data that allow to rank these policies. Generalizing to non-linear objective functions, it is then shown that policy effects are partially identified if and only if the policy objective is a robust statistic in the sense of having a bounded influence function. Furthermore, rankings derived from a linearized version of the objective function give correct rankings in a neighborhood of a status quo policy, and are easy to calculate in practice. The theoretical results of this paper are applied to data from the ``project STAR'' experiment, in which children were randomly assigned to classes of different sizes. This application illustrates the dependence of identifiability of the policy ranking on identifying assumptions, the feasible policy set, and distributional preferences.