IDEAS home Printed from https://ideas.repec.org/p/tiu/tiutis/bff59032-44ab-4be3-8c96-196a4e8d2444.html
   My bibliography  Save this paper

Design of web questionnaires : The effect of layout in rating scales

Author

Listed:
  • Toepoel, V.

    (Tilburg University, School of Economics and Management)

  • Das, J.W.M.

    (Tilburg University, School of Economics and Management)

  • van Soest, A.H.O.

    (Tilburg University, School of Economics and Management)

Abstract

This article shows that respondents gain meaning from verbal cues (words) as well as nonverbal cues (layout; numbers) in a web survey. We manipulated the layout of a five-point rating scale in two experiments. In the first experiment, we compared answers for different presentations of the responses: in one column with separate rows for each answer (“linear”), in three columns and two rows (“nonlinear”) in various orders, and after adding numerical labels to each response option. Our results show significant differences between a linear and nonlinear layout of response options. In the second experiment we looked at effects of verbal, graphical, and numerical language. We compared two linear vertical layouts with reverse orderings (from positive to negative and from negative to positive), a horizontal layout, and layouts with various numerical labels (1 to 5, 5 to 1, and 2 to 22). We found effects of verbal and graphical language. The effect of numerical language was only apparent when the numbers 2 to 22 were added to the verbal labels. We also examined whether the effects of design vary with personal characteristics. Elderly respondents appeared to be more sensitive to verbal, graphical, and numerical language.
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was bo
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Toepoel, V. & Das, J.W.M. & van Soest, A.H.O., 2009. "Design of web questionnaires : The effect of layout in rating scales," Other publications TiSEM bff59032-44ab-4be3-8c96-1, Tilburg University, School of Economics and Management.
  • Handle: RePEc:tiu:tiutis:bff59032-44ab-4be3-8c96-196a4e8d2444
    as

    Download full text from publisher

    File URL: https://pure.uvt.nl/ws/portalfiles/portal/1170497/VTW_Toepoel_Design_JoOS_2009_pdf.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Toepoel, V. & Dillman, D.A., 2008. "Words, Numbers and Visual Heuristics in Web Surveys : Is there a Hierarchy of Importance?," Discussion Paper 2008-92, Tilburg University, Center for Economic Research.
    2. Elisabeth Deutskens & Ko de Ruyter & Martin Wetzels & Paul Oosterveld, 2004. "Response Rate and Response Quality of Internet-Based Surveys: An Experimental Study," Marketing Letters, Springer, vol. 15(1), pages 21-36, February.
    3. Toepoel, V. & Dillman, D.A., 2008. "Words, Numbers and Visual Heuristics in Web Surveys : Is there a Hierarchy of Importance?," Other publications TiSEM 21fbcc01-3c5d-4942-b41c-7, Tilburg University, School of Economics and Management.
    4. Natacha Borgers & Dirk Sikkel & Joop Hox, 2004. "Response Effects in Surveys on Children and Adolescents: The Effect of Number of Response Options, Negative Wording, and Neutral Mid-Point," Quality & Quantity: International Journal of Methodology, Springer, vol. 38(1), pages 17-33, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Toepoel, V. & Das, J.W.M. & van Soest, A.H.O., 2008. "Design Effects in Web Surveys : Comparing Trained and Fresh Respondents," Other publications TiSEM 4c8b40aa-7557-4844-9474-b, Tilburg University, School of Economics and Management.
    2. Brauner, Jacob, 2020. "Are Smileys Valid Answers? Survey Data Quality with Innovative Item Formats," SocArXiv dk9bc, Center for Open Science.
    3. Bart Buelens & Jan A. van den Brakel, 2015. "Measurement Error Calibration in Mixed-mode Sample Surveys," Sociological Methods & Research, , vol. 44(3), pages 391-426, August.
    4. Dana Garbarski & Nora Cate Schaeffer & Jennifer Dykema, 2019. "The Effects of Features of Survey Measurement on Self-Rated Health: Response Option Order and Scale Orientation," Applied Research in Quality of Life, Springer;International Society for Quality-of-Life Studies, vol. 14(2), pages 545-560, April.
    5. Toepoel, V. & Dillman, D.A., 2008. "Words, Numbers and Visual Heuristics in Web Surveys : Is there a Hierarchy of Importance?," Discussion Paper 2008-92, Tilburg University, Center for Economic Research.
    6. Toepoel, V. & Dillman, D.A., 2008. "Words, Numbers and Visual Heuristics in Web Surveys : Is there a Hierarchy of Importance?," Other publications TiSEM 21fbcc01-3c5d-4942-b41c-7, Tilburg University, School of Economics and Management.
    7. Anna DeCastellarnau, 2018. "A classification of response scale characteristics that affect data quality: a literature review," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(4), pages 1523-1559, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Toepoel, V. & Das, J.W.M. & van Soest, A.H.O., 2005. "Design of Web Questionnaires : A Test for Number of Items per Screen," Discussion Paper 2005-114, Tilburg University, Center for Economic Research.
    2. Toepoel, V. & Dillman, D.A., 2008. "Words, Numbers and Visual Heuristics in Web Surveys : Is there a Hierarchy of Importance?," Discussion Paper 2008-92, Tilburg University, Center for Economic Research.
    3. Toepoel, V. & Dillman, D.A., 2008. "Words, Numbers and Visual Heuristics in Web Surveys : Is there a Hierarchy of Importance?," Other publications TiSEM 21fbcc01-3c5d-4942-b41c-7, Tilburg University, School of Economics and Management.
    4. Srinivasan, V. Seenu & Netzer, Oded, 2007. "Adaptive Self-Explication of Multi-attribute Preferences," Research Papers 1979, Stanford University, Graduate School of Business.
    5. Tatsushi Fukaya & Masayuki Suzuki & Ikumi Ozawa & Takumi Nakagoshi, 2022. "An Examination of Related Factors of Mathematical Pedagogical Content Knowledge in Elementary School Teachers: Focusing on Conceptions of Teaching and Learning and Test Utilization Strategy," SAGE Open, , vol. 12(4), pages 21582440221, October.
    6. Lusk, Jayson L., 2012. "The political ideology of food," Food Policy, Elsevier, vol. 37(5), pages 530-542.
    7. Carney, JoLynn V. & Liu, Yanhong & Hazler, Richard J., 2018. "A path analysis on school bullying and critical school environment variables: A social capital perspective," Children and Youth Services Review, Elsevier, vol. 93(C), pages 231-239.
    8. Rolf Becker, 2023. "Short- and long-term effects of reminders on panellists’ survey participation in a probability-based panel study with a sequential mixed-mode design," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(5), pages 4095-4119, October.
    9. Vahid Sobhani & Mohammadjavad Rostamizadeh & Seyed Morteza Hosseini & Seyed Ebrahim Hashemi & Ignacio Refoyo Román & Daniel Mon-López, 2022. "Anthropometric, Physiological, and Psychological Variables That Determine the Elite Pistol Performance of Women," IJERPH, MDPI, vol. 19(3), pages 1-10, January.
    10. Heleen van der Meulen & Rinaldo Kühne & Suzanna J. Opree, 2018. "Validating the Material Values Scale for Children (MVS-c) for Use in Early Childhood," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 11(4), pages 1201-1216, August.
    11. Barbara Ryan & Rachel King, 2020. "How ready is ready? Measuring physical preparedness for severe storms," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 104(1), pages 171-199, October.
    12. Gádor Indra Hidalgo & Fermín Sánchez-Carracedo & Daniel Romero-Portillo, 2021. "COVID-19 Emergency Remote Teaching Opinions and Academic Performance of Undergraduate Students: Analysis of 4 Students’ Profiles. A Case Study," Mathematics, MDPI, vol. 9(17), pages 1-20, September.
    13. Yang Liu & Jan Hannig & Abhishek Pal Majumder, 2019. "Second-Order Probability Matching Priors for the Person Parameter in Unidimensional IRT Models," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 701-718, September.
    14. Elisabeth Deutskens & Ad Jong & Ko Ruyter & Martin Wetzels, 2006. "Comparing the generalizability of online and mail surveys in cross-national service quality research," Marketing Letters, Springer, vol. 17(2), pages 119-136, April.
    15. Ayako Kohno & Maznah Dahlui & Nik Daliana Nik Farid & Norlaili Abdul Aziz & Takeo Nakayama, 2021. "Development of Early Marriage Attitude Scale: A Multi-Dimensional Scale for Measuring the Attitudes Toward Child Marriage," SAGE Open, , vol. 11(3), pages 21582440211, August.
    16. Giancarlo Condello & Laura Capranica & Mojca Doupona & Kinga Varga & Verena Burk, 2019. "Dual-career through the elite university student-athletes’ lenses: The international FISU-EAS survey," PLOS ONE, Public Library of Science, vol. 14(10), pages 1-18, October.
    17. María del Mar Serrano-Arcos & Raquel Sánchez-Fernández & Juan Carlos Pérez-Mesa, 2021. "Analysis of Product-Country Image from Consumer’s Perspective: The Impact of Subjective Knowledge, Perceived Risk and Media Influence," Sustainability, MDPI, vol. 13(4), pages 1-25, February.
    18. Björn Hofman & Gerdien de Vries & Geerten van de Kaa, 2022. "Keeping Things as They Are: How Status Quo Biases and Traditions along with a Lack of Information Transparency in the Building Industry Slow Down the Adoption of Innovative Sustainable Technologies," Sustainability, MDPI, vol. 14(13), pages 1-20, July.
    19. Bader, Benjamin & Berg, Nicola & Holtbrügge, Dirk, 2015. "Expatriate performance in terrorism-endangered countries: The role of family and organizational support," International Business Review, Elsevier, vol. 24(5), pages 849-860.
    20. Wells, Victoria K. & Manika, Danae & Gregory-Smith, Diana & Taheri, Babak & McCowlen, Clair, 2015. "Heritage tourism, CSR and the role of employee environmental behaviour," Tourism Management, Elsevier, vol. 48(C), pages 399-413.

    More about this item

    JEL classification:

    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tiu:tiutis:bff59032-44ab-4be3-8c96-196a4e8d2444. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Richard Broekman (email available below). General contact details of provider: https://www.tilburguniversity.edu/about/schools/economics-and-management/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.