IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v185y2022i3p872-890.html
   My bibliography  Save this article

Innovating the collection of open‐ended answers: The linguistic and content characteristics of written and oral answers to political attitude questions

Author

Listed:
  • Konstantin Gavras
  • Jan Karem Höhne
  • Annelies G. Blom
  • Harald Schoen

Abstract

The rapid increase in smartphone surveys and technological developments open novel opportunities for collecting survey answers. One of these opportunities is the use of open‐ended questions with requests for oral instead of written answers, which may facilitate the answer process and result in more in‐depth and unfiltered information. Whereas it is now possible to collect oral answers on smartphones, we still lack studies on the impact of this novel answer format on the characteristics of respondents' answers. In this study, we compare the linguistic and content characteristics of written versus oral answers to political attitude questions. For this purpose, we conducted an experiment in a smartphone survey (N = 2402) and randomly assigned respondents to an answer format (written or oral). Oral answers were collected via the open source ‘SurveyVoice (SVoice)’ tool, whereas written answers were typed in via the smartphone keypad. Applying length analysis, lexical structure analysis, sentiment analysis and structural topic models, our results reveal that written and oral answers differ substantially from each other in terms of lengths, structures, sentiments and topics. We find evidence that written answers are characterized by an intentional and conscious answering, whereas oral answers are characterized by an intuitive and spontaneous answering.

Suggested Citation

  • Konstantin Gavras & Jan Karem Höhne & Annelies G. Blom & Harald Schoen, 2022. "Innovating the collection of open‐ended answers: The linguistic and content characteristics of written and oral answers to political attitude questions," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(3), pages 872-890, July.
    • Handle: RePEc:bla:jorssa:v:185:y:2022:i:3:p:872-890
    DOI: 10.1111/rssa.12807
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssa.12807
    Download Restriction: no
    File URL: https://libkey.io/10.1111/rssa.12807?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Laura K. Nelson & Derek Burk & Marcel Knudsen & Leslie McCall, 2021. "The Future of Coding: A Comparison of Hand-Coding and Three Types of Computer-Assisted Text Analysis Methods," Sociological Methods & Research, , vol. 50(1), pages 202-237, February.
    2. Christopher Claassen, 2020. "Does Public Support Help Democracy Survive?," American Journal of Political Science, John Wiley & Sons, vol. 64(1), pages 118-134, January.
    3. Lodge, Milton & McGraw, Kathleen M. & Stroh, Patrick, 1989. "An Impression-Driven Model of Candidate Evaluation," American Political Science Review, Cambridge University Press, vol. 83(2), pages 399-419, June.
    4. Achen, Christopher H., 1975. "Mass Political Attitudes and the Survey Response," American Political Science Review, Cambridge University Press, vol. 69(4), pages 1218-1231, December.
    5. Melanie Revilla & Carlos Ochoa, 2016. "Open narrative questions in PC and smartphones: is the device playing a role?," Quality & Quantity: International Journal of Methodology, Springer, vol. 50(6), pages 2495-2513, November.
    6. Proksch, Sven-Oliver & Wratil, Christopher & Wäckerle, Jens, 2019. "Testing the Validity of Automatic Speech Recognition for Political Text Analysis," Political Analysis, Cambridge University Press, vol. 27(3), pages 339-359, July.
    7. Anna DeCastellarnau, 2018. "A classification of response scale characteristics that affect data quality: a literature review," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(4), pages 1523-1559, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jan Karem Höhne & Christoph Kern & Konstantin Gavras & Stephan Schlosser, 2024. "The sound of respondents: predicting respondents’ level of interest in questions with voice data in smartphone surveys," Quality & Quantity: International Journal of Methodology, Springer, vol. 58(3), pages 2907-2927, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Galasso, Vincenzo & Nannicini, Tommaso & Nozza, Debora, 2024. "We Need to Talk: Audio Surveys and Information Extraction," IZA Discussion Papers 17488, Institute of Labor Economics (IZA).
    2. Bernhardt, Lea & Dewenter, Ralf & Thomas, Tobias, 2023. "Measuring partisan media bias in US newscasts from 2001 to 2012," European Journal of Political Economy, Elsevier, vol. 78(C).
    3. Joël Cariolle & Yasmine Elkhateeb & Mathilde Maurel, 2022. "(Mis-)information technology: Internet use and perception of democracy in Africa," Documents de travail du Centre d'Economie de la Sorbonne 22010, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
    4. Enriqueta Aragones, 1997. "Negativity Effect and the Emergence of Ideologies," Journal of Theoretical Politics, , vol. 9(2), pages 189-210, April.
    5. Felderer, Barbara & Repke, Lydia & Weber, Wiebke & Schweisthal, jonas & Bothmann, Ludwig, 2024. "Predicting the Validity and Reliability of Survey Questions," OSF Preprints hkngd, Center for Open Science.
    6. Cavaillé, Charlotte & Chen, Daniel L. & Van Der Straeten, Karine, 2022. "Who Cares? Measuring Preference Intensity in a Polarized Environment," IAST Working Papers 22-130, Institute for Advanced Study in Toulouse (IAST).
    7. John Patty & Roberto Weber, 2007. "Letting the good times roll: A theory of voter inference and experimental evidence," Public Choice, Springer, vol. 130(3), pages 293-310, March.
    8. Jennifer Oser & Marc Hooghe & Zsuzsa Bakk & Roberto Mari, 2023. "Changing citizenship norms among adolescents, 1999-2009-2016: A two-step latent class approach with measurement equivalence testing," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(5), pages 4915-4933, October.
    9. repec:osf:osfxxx:jvpbw_v1 is not listed on IDEAS
    10. Daron Acemoglu & Nicolás Ajzenman & Cevat Giray Aksoy & Martin Fiszbein & Carlos Molina, 2025. "(Successful) Democracies Breed Their Own Support," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 92(2), pages 621-655.
    11. Alex Luscombe & Kevin Dick & Kevin Walby, 2022. "Algorithmic thinking in the public interest: navigating technical, legal, and ethical hurdles to web scraping in the social sciences," Quality & Quantity: International Journal of Methodology, Springer, vol. 56(3), pages 1023-1044, June.
    12. Lupia, Arthur & Prior, Markus, 2005. "What Citizens Know Depends on How You Ask Them: Political Knowledge and Political Learning Skills," MPRA Paper 103, University Library of Munich, Germany, revised 25 Sep 2006.
    13. Amihai Glazer, 2013. "Performance when misinformation increases with experience," Journal of Theoretical Politics, , vol. 25(1), pages 63-74, January.
    14. Lockwood, Ben & Le, Minh & Rockey, James, 2024. "Dynamic electoral competition with voter loss-aversion and imperfect recall," Journal of Public Economics, Elsevier, vol. 232(C).
    15. Nehemia Geva & James Mayhar & J. Mark Skorick, 2000. "The Cognitive Calculus of Foreign Policy Decision Making," Journal of Conflict Resolution, Peace Science Society (International), vol. 44(4), pages 447-471, August.
    16. Viola Merhof & Thorsten Meiser, 2023. "Dynamic Response Strategies: Accounting for Response Process Heterogeneity in IRTree Decision Nodes," Psychometrika, Springer;The Psychometric Society, vol. 88(4), pages 1354-1380, December.
    17. Duane F. Alwin & Jon A. Krosnick, 1991. "The Reliability of Survey Attitude Measurement," Sociological Methods & Research, , vol. 20(1), pages 139-181, August.
    18. Giovana Vitória Nunes Leite Duarte & Susana Pereira Antunes Procópio & Angélica Cotta Lobo Leite Carneiro & Leandro de Morais Cardoso, 2022. "Development and Validation of a Tool for Assessing Sustainable Social Practices in Food Services," Sustainability, MDPI, vol. 14(24), pages 1-18, December.
    19. Arne Weigold & Ingrid K. Weigold & Migyeong Jang & Emily M. Thornton, 2022. "College students’ and Mechanical Turk workers’ environmental factors while completing online surveys," Quality & Quantity: International Journal of Methodology, Springer, vol. 56(4), pages 2589-2612, August.
    20. Thomas Isbell, 2024. "Where You Sit Is Where You Stand: Perceived (In)Equality and Demand for Democracy in Africa," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 174(3), pages 817-836, September.
    21. Oriol J. Bosch & Melanie Revilla, 2021. "Using emojis in mobile web surveys for Millennials? A study in Spain and Mexico," Quality & Quantity: International Journal of Methodology, Springer, vol. 55(1), pages 39-61, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:185:y:2022:i:3:p:872-890. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.