How should we measure Americans’ perceptions of socio-economic mobility?

Several scholars have suggested that Americans’ (distorted) beliefs about the rate of upward social mobility in the United States may affect political judgment and decision-making outcomes. In this article, we consider the psychometric properties of two different questionnaire items that researchers have used to measure these subjective perceptions. Namely, we report the results of a new set of experiments (N = 2,167 U.S. MTurkers) in which we compared the question wording employed by Chambers, Swan and Heesacker (2015) with the question wording employed by Davidai and Gilovich (2015). Each (independent) research team had prompted similar groups of respondents to estimate the percentage of Americans born into the bottom of the income distribution who improved their socio-economic standing by adulthood, yet the two teams reached ostensibly irreconcilable conclusions: that Americans tend to underestimate (Chambers et al.) and overestimate (Davidai & Gilovich) the true rate of upward social mobility in the U.S. First, we successfully reproduced both contradictory results. Next, we isolated and experimentally manipulated one salient difference between the two questions’ response-option formats: asking participants to divide the population into either (a) “thirds” (tertiles) or (b) “20%” segments (quintiles). Inverting this tertile-quintile factor significantly altered both teams’ findings, suggesting that these measures are inappropriate (too vulnerable to question-wording and item-formulation artifacts) for use in studies of perceptual (in)accuracy. Finally, we piloted a new question for measuring subjective perceptions of social mobility. We conclude with tentative recommendations for researchers who wish to model the causes and consequences of Americans’ mobility-related beliefs.