Project Nonprobability Sample Inference: Fieldwork Report

"Probability-based surveys (PS) face increasing pressure due to high costs and low responserates. At the same time, nonprobability-based surveys (NPS) which are presumably cheaper and allow for faster data collection become more and more popular. The Project “Nonprobability Sample Inference” (NOSI) evaluates to which extent NPS may serve as aviable alternative to PS in labor market research by comparing one PS to four NPS collected by different providers. This report provides a brief overview of data collection processes, data quality, and costs of these overall five surveys. The PS involved a 2 × 2 experiment on using incentives and reminders for recruitment. The incentive consisted of a 5 € voucher, while the reminder was a follow-up invitation sent after a two-week delay. Using both an incentive and a reminder increased the response rate by almost 10 percentage points compared to using neither (17.1 vs 7.5 percent). Data collection lasted for 69 days, while the NPS took a maximum of 20 days, with some only taking seven days. For the PS sample, we find that higher-educated, employed, and German individuals are more likely to participate in the survey. Comparing the socio-demographic composition of all samples, we find that respondents without Germany citizenship are widely underrepresented in NPS, while more respondents in the probability-based survey are highly educated. Regarding the employment status, we observe large differences across all samples, also concerning changes in their status. The expected duration of the questionnaire was around ten minutes. The respondents inthe PS had a median duration of approximately ten minutes, while three of four NPS had median durations below six minutes. Attention check failure was higher in the NPS with proportions up to 25 percent. Among three NPS samples, more than 20 percent of the respondents participate in five or more surveys per week. The average costs per completed interview for the PS depend on reminder and incentive usage. Only two of the NPS are cheaper than the PS without incentives and reminders. One NPS is even more expensive than the PS with incentives and reminders." (Author's abstract, IAB-Doku) ((en))