The Cognitive Illusion Controversy: A Methodological Debate in Disguise That Matters to Economists

In the early 1970s, Daniel Kahneman and Amos Tversky launched a research program that showed that heuristic short-cuts can result in probability judgments that deviate from statistical principles. Because these cognitive illusions have important implications for economic behavior, the heuristics-and-biases program has attracted the attention of economists as well as numerous social scientists. Even as the heuristics-and-biases program gained acceptance outside psychology, it drew criticism within the field. In this chapter, we mine the debate among psychologists about the reality of cognitive illusions for methodological lessons of relevance to experimental economists. Our concern here is neither the controversy about cognitive illusions nor its implications for rationality. Instead, it is what we see as the important methodological insights that have emerged from the controversy, which can inform the choices that all behavioral experimenters wittingly or unwittingly make when they sample and represent stimuli for use in their experiments.