Identifying and Managing Fraudulent Respondents in Online Stated Preferences Surveys: A Case Example from Best–Worst Scaling in Health Preferences Research

Background There is limited evidence and guidance in health preferences research to prevent, identify, and manage fraudulent respondents and data fraud, especially for best–worst scaling (BWS) and discrete choice experiments with nonordered attributes. Using an example from a BWS survey in which we experienced data fraud, we aimed to: (1) develop an approach to identify, verify, and categorize fraudulent respondents; (2) assess the impact of fraudulent respondents on data and results; and (3) identify variables associated with fraudulent respondents. Methods An online BWS survey on healthcare services for inflammatory bowel disease (IBD) was administered to Canadian IBD patients. We used a three-step approach to identify, verify, and categorize respondents as likely fraudulent (LF), likely real (LR), and unsure. First, responses to 12 “red flag” variables (variables identified as indicators of fraud) were coded 0 (pass) or 1 (fail) then summed to generate a “fraudulent response score” (FRS; range: 0–12 (most likely fraudulent)) used to categorize respondents. Second, respondents categorized LR or unsure underwent age verification. Third, categorization was updated on the basis of age verification results. BWS data were analyzed using conditional logit and latent class analysis. Subgroup analysis was done by final categorization, FRS, and red flag variables. Results Overall, n = 4334 respondents underwent initial categorization resulting in 24% (n = 1019) LF and 76% (n = 3315) needing further review. After review, 75% (n = 3258) were categorized as LF and n = 484 underwent age verification. Respondent categorization was updated on the basis of age verification, with final categorization of 76% (n = 3297) LF, 14% (n = 592) unsure, 10% (n = 442) LR, and