IDEAS home Printed from https://ideas.repec.org/a/bpj/statpp/v8y2017i1p105-120n2.html
   My bibliography  Save this article

What do we Lose with Online-Only Surveys? Estimating the Bias in Selected Political Variables Due to Online Mode Restriction

Author

Listed:
  • Pforr Klaus

    (GESIS – Leibniz-Institute for the Social Sciences, Monitoring Society and Social Change, B2, 1, Mannheim 68159, Germany)

  • Dannwolf Tanja

    (TU Kaiserslautern, Faculty of Social Science – Political Sciences I, Erwin-Schroedinger-Str., D-67653 Kaiserslautern, Germany)

Abstract

This paper adds to the discussion on the value of online surveys for political science research. Mainly because of the lower costs, collecting survey data over the web has become increasingly popular in recent years, despite the higher sampling and coverage error in web-only surveys, especially online access polls. Recruiting respondents for the actual panel surveys based on a representative sample using a different mode is regarded as a solution to the sampling problem. Two approaches have been used to tackle the problem of coverage error: Providing respondents with computers (e.g. LISS, ELIPPS, GIP) and offering a different mode than online to respondents, thereby adopting a mixed mode design (e.g. paper as in GALLUP, GESIS Panel). The literature suggests that offering participation in the respondent’s preferred mode affects response rates positively but not much is known about respondents’ reasoning to choose a specific mode. We argue that it is important to understand this decision to evaluate the selection into online surveys and the consequences this has for data quality. We investigate this question by drawing on data from the GESIS Panel face-to-face recruitment interview for building a mixed mode access panel (paper and web) in Germany that gave a mode choice to internet users. Our results suggest that web literacy, age and education alone do not explain the mode choice but that affinity towards the technology related to the online mode has an independent effect. In a second step, we analyse the effect of this selection mechanism on answers to questions on typical variables used in political participation research such as media competence, political interest and civic duty in the subsequent mixed mode survey. We assess the added value of adopting a mixed-mode strategy. The results inform the evaluation of biases in unimode online surveys.

Suggested Citation

  • Pforr Klaus & Dannwolf Tanja, 2017. "What do we Lose with Online-Only Surveys? Estimating the Bias in Selected Political Variables Due to Online Mode Restriction," Statistics, Politics and Policy, De Gruyter, vol. 8(1), pages 105-120, October.
    • Handle: RePEc:bpj:statpp:v:8:y:2017:i:1:p:105-120:n:2
    DOI: 10.1515/spp-2016-0004
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/spp-2016-0004
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.
    File URL: https://libkey.io/10.1515/spp-2016-0004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ansolabehere, Stephen & Schaffner, Brian F., 2014. "Does Survey Mode Still Matter? Findings from a 2010 Multi-Mode Comparison," Political Analysis, Cambridge University Press, vol. 22(3), pages 285-303, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Julius Grund & Antje Brock, 2020. "Education for Sustainable Development in Germany: Not Just Desired but Also Effective for Transformative Action," Sustainability, MDPI, vol. 12(7), pages 1-20, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kolcava, Dennis, 2020. "Do citizens hold business accountable for greenwashing by demanding more government intervention?," OSF Preprints sj4dk, Center for Open Science.
    2. Ulla A. Saari & Rupert J. Baumgartner & Saku J. Mäkinen, 2017. "Eco-Friendly Brands to Drive Sustainable Development: Replication and Extension of the Brand Experience Scale in a Cross-National Context," Sustainability, MDPI, vol. 9(7), pages 1-26, July.
    3. Helbling, Marc & Jungkunz, Sebastian, 2020. "Social divides in the age of globalization," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 43(6), pages 1187-1210.
    4. Sakshaug Joseph W. & Wiśniowski Arkadiusz & Ruiz Diego Andres Perez & Blom Annelies G., 2019. "Supplementing Small Probability Samples with Nonprobability Samples: A Bayesian Approach," Journal of Official Statistics, Sciendo, vol. 35(3), pages 653-681, September.
    5. Kolcava, Dennis & Bernauer, Thomas, 2020. "Greening the Economy Through Voluntary Private Sector Initiatives or Government Regulation? A Public Opinion Perspective," OSF Preprints zsk43, Center for Open Science.
    6. Piatak Jaclyn, 2023. "Do Sociocultural Factors Drive Civic Engagement? An Examination of Political Interest and Religious Attendance," Nonprofit Policy Forum, De Gruyter, vol. 14(2), pages 185-204, April.
    7. David M. Konisky & Llewelyn Hughes & Charles H. Kaylor, 2016. "Extreme weather events and climate change concern," Climatic Change, Springer, vol. 134(4), pages 533-547, February.
    8. Michael A. Cacciatore & Glen J. Nowak & Nathaniel J. Evans, 2018. "It's Complicated: The 2014–2015 U.S. Measles Outbreak and Parents’ Vaccination Beliefs, Confidence, and Intentions," Risk Analysis, John Wiley & Sons, vol. 38(10), pages 2178-2192, October.
    9. McFadden, Brandon R. & Malone, Trey, 2018. "How will mandatory labeling of genetically modified food nudge consumer decision-making?," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 77(C), pages 186-194.
    10. Lisa Maria Dellmuth & Jonas Tallberg, 2020. "Why national and international legitimacy beliefs are linked: Social trust as an antecedent factor," The Review of International Organizations, Springer, vol. 15(2), pages 311-337, April.
    11. Craig F. Berning & Brian E. Roe, 2017. "Assessing the National Bioengineered Food Disclosure Standard of 2016: Can Americans Access Electronic Disclosure Information?," Sustainability, MDPI, vol. 9(6), pages 1-9, May.
    12. Anderson, Brilé & Bernauer, Thomas, 2016. "How much carbon offsetting and where? Implications of efficiency, effectiveness, and ethicality considerations for public opinion formation," Energy Policy, Elsevier, vol. 94(C), pages 387-395.
    13. Aaron Martin & Timothy B Gravelle & Erik Baekkeskov & Jenny Lewis & Yoshi Kashima, 2019. "Enlisting the support of trusted sources to tackle policy problems: The case of antimicrobial resistance," PLOS ONE, Public Library of Science, vol. 14(3), pages 1-9, March.
    14. Timmons, Shane & Barjaková, Martina & Robertson, Deirdre & Belton, Cameron & Lunn, Pete, 2020. "Public understanding and perceptions of the COVID-19 Test-and-Trace system," Research Series, Economic and Social Research Institute (ESRI), number SUSTAT96, June.
    15. Rudolph, Lukas & Freitag, Markus & Thurner, Paul, 2021. "The Comparative Legitimacy of Arms Exports - A Conjoint Experiment in Germany and France," SocArXiv r73pv, Center for Open Science.
    16. Radford, Jason & Green, Jon & Quintana, Alexi & Safarpour, Alauna & Simonson, Matthew D & Baum, Matthew & Lazer, David & Ognyanova, Katherine & Druckman, James & Perlis, Roy, 2022. "Evaluating the generalizability of the COVID States survey — a large-scale, non-probability survey," OSF Preprints cwkg7, Center for Open Science.
    17. Justin T. Pickett & Thomas A. Loughran & Shawn Bushway, 2015. "On the Measurement and Properties of Ambiguity in Probabilistic Expectations," Sociological Methods & Research, , vol. 44(4), pages 636-676, November.
    18. Matthew B. Arbuckle & David M. Konisky, 2015. "The Role of Religion in Environmental Attitudes," Social Science Quarterly, Southwestern Social Science Association, vol. 96(5), pages 1244-1263, November.
    19. Smiley, Kevin T. & Emerson, Michael Oluf, 2017. "A Spirit of Urban Capitalism: Market Cities, People Cities, and Cultural Justifications," SocArXiv uexh9, Center for Open Science.
    20. Hannan, Kellie & Cullen, Francis T. & Butler, Leah C. & Graham, Amanda & Burton, Alexander L. & Burton, Velmer S. Jr., 2020. "Racial Sympathy and Support for Capital Punishment: A Case Study in Concept Transfer," SocArXiv xybj9, Center for Open Science.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:statpp:v:8:y:2017:i:1:p:105-120:n:2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.