Evaluating Deliberative Competence: A Simple Method with an Application to Financial Choice

We examine methods for evaluating opportunity-neutral interventions designed to improve the quality of decision making in settings where people imperfectly comprehend consequences. In an experiment involving financial education, conventional outcome metrics (financial literacy and directional changes in behavior) imply that two interventions, one with practice and feedback, one without, are equally beneficial even though only the first reduces average bias. We trace these evaluative failures to violations of implicit assumptions. We propose a simple intuitive outcome metric that properly differentiates between the interventions, and that is robustly interpretable as a measure of welfare loss even when consumers suffer from other biases.