Who likes it more? Using response times to elicit group preferences in surveys

Surveys and opinion polls are essential instruments to elicit societal preferences and uncover differences between socioeconomic or demographic groups. However, survey data is noisy, and survey bias is ubiquitous, limiting the reliability and usefulness of standard analyses. We provide a new method that uncovers group preferences and unambiguously ranks the relative strength of preference between groups of agents, leveraging the information contained in response times. The method delivers a nonparametric criterion to determine whether a group (defined, e.g., by gender, age cohort, socioeconomic status, political orientation, etc.) prefers an option over its alternative, and whether it does so more strongly than another group, without any assumptions on the underlying noise. We demonstrate the practical value of this method by studying preferences over important socioeconomic topics in a representative sample of the U.K. population. We find that the new method often provides results when tests based on choice frequencies are inconclusive, and also identifies cases where tests are significant but inferences on preferences are unwarranted.