Split-Half Experiment on Data Quality of Factorial Surveys: Reducing Social Desirability Bias and Priming

Factorial surveys are well-established and advantageous but not immune to issues of data quality that result from social desirability bias or unintended priming. Systematic methodical research needs to examine the possible antecedents – and what exacerbates them – to attain more truthful responses. The aim of the current study is to examine under what conditions the 2nd person point of view – versus the 3rd person point of view – leads to higher data quality in factorial surveys. Specifically, the assumptions are tested that while the 2nd person point of view may generally deliver the better data, the opposite is true when sensitive topics are involved. Additionally, selecting a point of view can influence unintended priming effects on respondent variables on similar topics. This issue needs to be examined to avoid increasing the vignette data quality at the expense of relevant respondent items in the questionnaire. In order to test these assumptions, this paper applies a systematic methodical approach used in previous studies examining factorial surveys and social desirability: a split-half experiment of a factorial survey. Two scenarios, one regular and one sensitive, are presented to respondents who are randomly allocated to one of two conditions: The 2nd person point of view or the 3rd person point of view (of the protagonist in the scenario). The results of this experiment are analyzed in-depth, including OLS and logistic regressions as well as multi-level regression models. The results provide support for the assumption that the more immersive 2nd person point of view generally performs better. When sensitive topics are involved, the 3rd person point of view may be preferable, although the results are not as clear in this regard. Moreover, the 3rd person point of view is less prone to unintended priming effects of the vignettes on respondent items covering similar topics.