IDEAS home Printed from https://ideas.repec.org/a/sae/somere/v52y2023i2p879-908.html
   My bibliography  Save this article

Response Quality in Nonprobability and Probability-based Online Panels

Author

Listed:
  • Carina Cornesse
  • Annelies G. Blom

Abstract

Recent years have seen a growing number of studies investigating the accuracy of nonprobability online panels; however, response quality in nonprobability online panels has not yet received much attention. To fill this gap, we investigate response quality in a comprehensive study of seven nonprobability online panels and three probability-based online panels with identical fieldwork periods and questionnaires in Germany. Three response quality indicators typically associated with survey satisficing are assessed: straight-lining in grid questions, item nonresponse, and midpoint selection in visual design experiments. Our results show that there is significantly more straight-lining in the nonprobability online panels than in the probability-based online panels. However, contrary to our expectations, there is no generalizable difference between nonprobability online panels and probability-based online panels with respect to item nonresponse. Finally, neither respondents in nonprobability online panels nor respondents in probability-based online panels are significantly affected by the visual design of the midpoint of the answer scale.

Suggested Citation

  • Carina Cornesse & Annelies G. Blom, 2023. "Response Quality in Nonprobability and Probability-based Online Panels," Sociological Methods & Research, , vol. 52(2), pages 879-908, May.
    • Handle: RePEc:sae:somere:v:52:y:2023:i:2:p:879-908
    DOI: 10.1177/0049124120914940
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0049124120914940
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0049124120914940?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Malhotra, Neil & Krosnick, Jon A., 2007. "The Effect of Survey Mode and Sampling on Inferences about Political Attitudes and Behavior: Comparing the 2000 and 2004 ANES to Internet Surveys with Nonprobability Samples," Political Analysis, Cambridge University Press, vol. 15(3), pages 286-323, July.
    2. Converse, Philip E., 1974. "Comment: The Status of Nonattitudes," American Political Science Review, Cambridge University Press, vol. 68(2), pages 650-660, June.
    3. Vera Toepoel & Corrie Vis & Marcel Das & Arthur van Soest, 2009. "Design of Web Questionnaires," Sociological Methods & Research, , vol. 37(3), pages 371-392, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lindhjem, Henrik & Navrud, Ståle, 2011. "Using Internet in Stated Preference Surveys: A Review and Comparison of Survey Modes," International Review of Environmental and Resource Economics, now publishers, vol. 5(4), pages 309-351, September.
    2. Catherine Chen & Bo MacInnis & Matthew Waltman & Jon A. Krosnick, 2021. "Public opinion on climate change in the USA: to what extent can it be nudged by questionnaire design features?," Climatic Change, Springer, vol. 167(3), pages 1-18, August.
    3. Kevin J. Boyle & Mark Morrison & Darla Hatton MacDonald & Roderick Duncan & John Rose, 2016. "Investigating Internet and Mail Implementation of Stated-Preference Surveys While Controlling for Differences in Sample Frames," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 64(3), pages 401-419, July.
    4. Lasse J. Jessen & Sebastian Koehne & Patrick Nüß & Jens Ruhose, 2024. "Socioeconomic Inequality in Life Expectancy: Perception and Policy Demand," CESifo Working Paper Series 10940, CESifo.
    5. Karytsas, Spyridon & Theodoropoulou, Helen, 2014. "Socioeconomic and demographic factors that influence publics' awareness on the different forms of renewable energy sources," Renewable Energy, Elsevier, vol. 71(C), pages 480-485.
    6. Aaron C. Sparks & Heather Hodges & Sarah Oliver & Eric R. A. N. Smith, 2020. "Confidence in Local, National, and International Scientists on Climate Change," Sustainability, MDPI, vol. 13(1), pages 1-13, December.
    7. Sakshaug Joseph W. & Wiśniowski Arkadiusz & Ruiz Diego Andres Perez & Blom Annelies G., 2019. "Supplementing Small Probability Samples with Nonprobability Samples: A Bayesian Approach," Journal of Official Statistics, Sciendo, vol. 35(3), pages 653-681, September.
    8. de Bruijne, M.A., 2015. "Designing web surveys for the multi-device internet," Other publications TiSEM 19e4d446-a62b-4a95-8691-8, Tilburg University, School of Economics and Management.
    9. Richard Carson & Nicholas Flores & Norman Meade, 2001. "Contingent Valuation: Controversies and Evidence," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 19(2), pages 173-210, June.
    10. Richard T. Carson, 2011. "Contingent Valuation," Books, Edward Elgar Publishing, number 2489.
    11. Magdalena Smyk & Joanna Tyrowicz & Lucas van der Velde, 2021. "A Cautionary Note on the Reliability of the Online Survey Data: The Case of Wage Indicator," Sociological Methods & Research, , vol. 50(1), pages 429-464, February.
    12. Anders Westholm, 1987. "Measurement error in causal analysis of panel data: Attenuated versus inflated relationships," Quality & Quantity: International Journal of Methodology, Springer, vol. 21(1), pages 3-20, March.
    13. Bart Buelens & Jan A. van den Brakel, 2015. "Measurement Error Calibration in Mixed-mode Sample Surveys," Sociological Methods & Research, , vol. 44(3), pages 391-426, August.
    14. Guy Grossman & Devorah Manekin & Dan Miodownik, 2013. "The Political Legacies of Combat: Attitudes towards war and peace amongst Israeli ex-combatants," HiCN Working Papers 161, Households in Conflict Network.
    15. Amihai Glazer & Bernard Grofman, 1989. "Why representatives are ideologists though voters are not," Public Choice, Springer, vol. 61(1), pages 29-39, April.
    16. Chatpong Tangmanee & Phattharaphong Niruttinanon, 2015. "Effects of Forced Responses and Question Display Styles on Web Survey Response Rates," International Journal of Research in Business and Social Science (2147-4478), Center for the Strategic Studies in Business and Finance, vol. 4(2), pages 54-62, April.
    17. Karytsas, Spyridon & Polyzou, Olympia & Karytsas, Constantine, 2019. "Factors affecting willingness to adopt and willingness to pay for a residential hybrid system that provides heating/cooling and domestic hot water," Renewable Energy, Elsevier, vol. 142(C), pages 591-603.
    18. Chatpong Tangmanee & Phattharaphong Niruttinanon, 2019. "Web Survey’s Completion Rates: Effects of Forced Responses, Question Display Styles, and Subjects’ Attitude," International Journal of Research in Business and Social Science (2147-4478), Center for the Strategic Studies in Business and Finance, vol. 8(1), pages 20-29, January.
    19. Vera Toepoel & Hannah Emerson, 2017. "Using experts’ consensus (the Delphi method) to evaluate weighting techniques in web surveys not based on probability schemes," Mathematical Population Studies, Taylor & Francis Journals, vol. 24(3), pages 161-171, July.
    20. Carina Cornesse & Ulrich Krieger & Marie‐Lou Sohnius & Marina Fikel & Sabine Friedel & Tobias Rettig & Alexander Wenz & Sebastian Juhl & Roni Lehrer & Katja Möhring & Elias Naumann & Maximiliane Reife, 2022. "From German Internet Panel to Mannheim Corona Study: Adaptable probability‐based online panel infrastructures during the pandemic," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(3), pages 773-797, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:somere:v:52:y:2023:i:2:p:879-908. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.