A national study of substance use: Demonstrated use of recommendations for best practice online data collection

Objectives: Since the COVID-19 pandemic, more researchers have used to online data collection to recruit participants to research studies. However, one perceived limitation of online data collection is a belief that it results in lower quality data due to the introduction of bots or misrepresentation by participants to qualify for study compensation. The current study demonstrates that following recommendations for online data collection results in quick collection of a high-quality, diverse, multi-state sample. Methods: The current study followed recommendations for best practice, advertising on social media sites combined with investigator-implemented (e.g., splash page, attention checks, use of physical payment) and built-in Qualtrics tools (e.g., IP tracking, CAPTCHA) to collect data from participants who use substances from 15 states within the United States examining cannabis use and perceptions of harm reduction interventions (HRIs). Results: Before cleaning, 3,642 participants completed the screener across 172 days of survey up-time. After cleaning, the final sample included 639 responses in the final cannabis survey, and 1,137 responses in the final HRI survey including 264 participants completing both surveys. The study yielded approximately 8.8 cleaned surveys per day and a usable data rate of 60.3% for participants who completed the cannabis survey only, 72.4% for participants who completed the HRI survey only, and 72.6% for participants who completed both. Conclusions: While every method of data collection has strengths and weaknesses, when implemented using appropriate tools to prevent completion of surveys by non-valid participants, internet-based data collection methods can provide researchers with relatively low-cost, high-quality samples.