Credible Comparisons Using Interpersonally Incomparable Data: Nonparametric Scales with Anchoring Vignettes

Comparisons of individuals based on their selections from an ordinal scale traditionally assume that all respondents interpret subjective scale categories in exactly the same way. Anchoring vignettes have been proposed as a method to replace this homogeneity assumption with individual‐specific data about how each respondent uses the ordinal scale. However, improving interpersonal comparisons with anchoring vignettes also requires a new set of assumptions. In this article, I derive the assumptions needed to make credible nonparametric comparisons using anchoring vignettes, and propose a new nonparametric scale that does not assume homogeneity among respondents. I also provide methods for evaluating empirically whether a set of anchoring objects can produce credible nonparametric interpersonal comparisons. Two empirical studies illustrate the importance of accounting for differences in the use of ordinal scales by showing how our inferences about interpersonal comparisons may change as a function of the assumptions we accept.