IDEAS home Printed from https://ideas.repec.org/a/eee/jbrese/v69y2016i8p3139-3148.html
   My bibliography  Save this article

A multi-group analysis of online survey respondent data quality: Comparing a regular USA consumer panel to MTurk samples

Author

Listed:
  • Smith, Scott M.
  • Roster, Catherine A.
  • Golden, Linda L.
  • Albaum, Gerald S.

Abstract

With the exploding use of Internet surveys, research efforts and data quality are increasingly subject to the effects of respondents who do not give the required attention to survey questions and who speed through the survey, or who intentionally cheat with their answers. We investigate respondent integrity and data quality for samples drawn from a “Regular” online panel and from Amazon's MTurk. New metrics for assessing sample integrity and online data quality are introduced. Overall, MTurk respondents in both respondent groups took less time to answer questions. The non-USA MTurk group deviated most from correct answers in attention filter questions and had more duplicate IP addresses. In addition, the results from the three Internet sample sources are substantively different. The choice of an Internet survey sample vendor is critical, as it can impact sample composition, respondent integrity, data quality, data structure and substantive results.

Suggested Citation

  • Smith, Scott M. & Roster, Catherine A. & Golden, Linda L. & Albaum, Gerald S., 2016. "A multi-group analysis of online survey respondent data quality: Comparing a regular USA consumer panel to MTurk samples," Journal of Business Research, Elsevier, vol. 69(8), pages 3139-3148.
    • Handle: RePEc:eee:jbrese:v:69:y:2016:i:8:p:3139-3148
    DOI: 10.1016/j.jbusres.2015.12.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S014829631500627X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jbusres.2015.12.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Robert Peterson & Dwight Merunka, 2014. "Convenience samples of college students and research reproducibility," Post-Print hal-01822317, HAL.
    2. Albaum, Gerald S. & Evangelista, Felicitas & Medina, Nila, 1998. "Role of Response Behavior Theory in Survey Research: A Cross-National Study," Journal of Business Research, Elsevier, vol. 42(2), pages 115-125, June.
    3. repec:cup:judgdm:v:5:y:2010:i:5:p:411-419 is not listed on IDEAS
    4. Lee, Hanjoon & Lindquist, Jay D. & Acito, Frank, 1997. "Managers' evaluation of research design and its impact on the use of research: An experimental approach," Journal of Business Research, Elsevier, vol. 39(3), pages 231-240, July.
    5. Peterson, Robert A. & Merunka, Dwight R., 2014. "Convenience samples of college students and research reproducibility," Journal of Business Research, Elsevier, vol. 67(5), pages 1035-1041.
    6. Braunsberger, Karin & Wybenga, Hans & Gates, Roger, 2007. "A comparison of reliability between telephone and web-based surveys," Journal of Business Research, Elsevier, vol. 60(7), pages 758-764, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cristina Robledo-Ardila & Juan Pablo Román-Calderón, 2022. "Potential: in search for meaning, theory and avenues for future research a systematic review," Management Review Quarterly, Springer, vol. 72(1), pages 149-186, February.
    2. Mohammad Nurul Alam & Osarodion Ogiemwonyi & Ibrahim. E. Hago & Noor Azlinna Azizan & Fariza Hashim & Md Sazzad Hossain, 2023. "Understanding Consumer Environmental Ethics and the Willingness to Use Green Products," SAGE Open, , vol. 13(1), pages 21582440221, January.
    3. Mireia Ercilla-Montserrat & David Sanjuan-Delmás & Esther Sanyé-Mengual & Laura Calvet-Mir & Karla Banderas & Joan Rieradevall & Xavier Gabarrell, 2019. "Analysis of the consumer’s perception of urban food products from a soilless system in rooftop greenhouses: a case study from the Mediterranean area of Barcelona (Spain)," Agriculture and Human Values, Springer;The Agriculture, Food, & Human Values Society (AFHVS), vol. 36(3), pages 375-393, September.
    4. José M. Ramírez-Hurtado & Alfredo G. Hernández-Díaz & Ana D. López-Sánchez & Víctor E. Pérez-León, 2021. "Measuring Online Teaching Service Quality in Higher Education in the COVID-19 Environment," IJERPH, MDPI, vol. 18(5), pages 1-14, March.
    5. Dwayne D. Gremler & Yves Vaerenbergh & Elisabeth C. Brüggen & Kevin P. Gwinner, 2020. "Understanding and managing customer relational benefits in services: a meta-analysis," Journal of the Academy of Marketing Science, Springer, vol. 48(3), pages 565-583, May.
    6. Aysun Ata Aktürk & Hasibe Özlen Demircan, 2018. "Development of Preschool Children Sibling Rivalry Scale (PSRS)," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 11(1), pages 117-136, February.
    7. Joana César Machado & Beatriz Fonseca & Carla Martins, 2021. "Brand logo and brand gender: examining the effects of natural logo designs and color on brand gender perceptions and affect," Journal of Brand Management, Palgrave Macmillan, vol. 28(2), pages 152-170, March.
    8. Lucinda Austin & Barbara Miller Gaither, 2019. "Redefining fit: examining CSR company-issue fit in stigmatized industries," Journal of Brand Management, Palgrave Macmillan, vol. 26(1), pages 9-20, January.
    9. Grigorios, Lamprinakos & Magrizos, Solon & Kostopoulos, Ioannis & Drossos, Dimitrios & Santos, David, 2022. "Overt and covert customer data collection in online personalized advertising: The role of user emotions," Journal of Business Research, Elsevier, vol. 141(C), pages 308-320.
    10. Rašković, Matevž & Ding, Zhonghui & Škare, Vatroslav & Ozretić Došen, Đurđana & Žabkar, Vesna, 2016. "Comparing consumer innovativeness and ethnocentrism of young-adult consumers," Journal of Business Research, Elsevier, vol. 69(9), pages 3682-3686.
    11. Teng, Weichen & Su, Yaohua & Liao, Tien-Tien & Wei, Cang-Liang, 2020. "An exploration of celebrity business ventures and their appeal to fans and non-fans," Journal of Retailing and Consumer Services, Elsevier, vol. 54(C).
    12. Seok Chan Jeong & Beom-Jin Choi, 2022. "Moderating Effects of Consumers’ Personal Innovativeness on the Adoption and Purchase Intention of Wearable Devices," SAGE Open, , vol. 12(4), pages 21582440221, November.
    13. Nadeem, Waqar & Al-Imamy, Saifeddin, 2020. "Do ethics drive value co-creation on digital sharing economy platforms?," Journal of Retailing and Consumer Services, Elsevier, vol. 55(C).
    14. Nadeem, Waqar & Juntunen, Mari & Shirazi, Farid & Hajli, Nick, 2020. "Consumers’ value co-creation in sharing economy: The role of social support, consumers’ ethical perceptions and relationship quality," Technological Forecasting and Social Change, Elsevier, vol. 151(C).
    15. Dunja Meštrović & Nina J. Zugic, 2018. "Students’ Perceptions of Service Quality, Satisfaction, and Word-Of-Mouth: Scale Adaptation and Validation on a Sample of ICT and STEM Students," Tržište/Market, Faculty of Economics and Business, University of Zagreb, vol. 30(2), pages 195-211.
    16. Rašković, Matevž & Ding, Zhonghui & Hirose, Morikazu & Žabkar, Vesna & Fam, Kim-Shyan, 2020. "Segmenting young-adult consumers in East Asia and Central and Eastern Europe – The role of consumer ethnocentrism and decision-making styles," Journal of Business Research, Elsevier, vol. 108(C), pages 496-507.
    17. Fagerstrøm, Asle & Eriksson, Niklas & Sigurdsson, Valdimar, 2020. "Investigating the impact of Internet of Things services from a smartphone app on grocery shopping," Journal of Retailing and Consumer Services, Elsevier, vol. 52(C).
    18. Gyeongcheol Cho & Christopher Schlaegel & Heungsun Hwang & Younyoung Choi & Marko Sarstedt & Christian M. Ringle, 2022. "Integrated Generalized Structured Component Analysis: On the Use of Model Fit Criteria in International Management Research," Management International Review, Springer, vol. 62(4), pages 569-609, August.
    19. Agustín J. Sánchez-Medina & Juan Manuel Benítez-del-Rosario & Félix Blázquez-Santana, 2017. "Anomia and Displacement of Responsibility as Determinants of Tourist Company Managers’ Non-Involvement in Alleviating Poverty," Sustainability, MDPI, vol. 9(5), pages 1-15, May.
    20. Dooyoung Choi & Ha Kyung Lee, 2020. "Beneficiary Foci Types and Performance Appeals in Green Advertising," Sustainability, MDPI, vol. 12(15), pages 1-15, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jbrese:v:69:y:2016:i:8:p:3139-3148. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jbusres .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.