Improvements to Survey Design from Pilot Testing a Discrete-Choice Experiment of the Preferences of Persons Living with HIV for Long-Acting Antiretroviral Therapies

Background and Objective Development of clear and effective discrete-choice experiment surveys is an important step toward ensuring accurate and usable preference results. Pretest interviews and pilot testing are common in the development of discrete-choice experiments, and it is important for researchers to report details of survey changes resulting from patient feedback elicited in pilot work. This paper details pilot testing of an online discrete-choice experiment to elicit preferences for long-acting antiretroviral therapies among patients with HIV. Methods The survey included an introduction to hypothetical treatment options, descriptions of attributes, comprehension questions, instructions for completing a discrete-choice experiment, a discrete-choice experiment with 17 choice tasks, and questions about personal characteristics. We piloted the survey with 50 respondents over ten waves. Each wave incorporated design improvements based on observations made during the previous wave. Respondents completed the online survey while screen sharing with a researcher, allowing interactive discussion. We developed a scheme for assessing and categorizing the survey changes. Results Changes to the pilot were categorized by ways they impacted aspects of the discrete-choice experiment or the likely quality of resulting data. The four categories of impact are: understanding of attributes, underlying discrete-choice experiment and understanding of the choice question, collection of individual characteristics hypothesized to affect preference, and changes that improved clarity and usability of the survey without directly affecting the other categories (e.g., survey navigation and instructional clarity, formatting changes). Conclusions Detailed attention to the respondent experience in this large pilot allowed survey improvements that will likely reduce ambiguity, ensure more accurate capture of patient preferences and, ultimately, improve product development for long-acting antiretroviral therapies.