IDEAS home Printed from https://ideas.repec.org/a/eee/jbrese/v69y2016i8p3139-3148.html
   My bibliography  Save this article

A multi-group analysis of online survey respondent data quality: Comparing a regular USA consumer panel to MTurk samples

Author

Listed:
  • Smith, Scott M.
  • Roster, Catherine A.
  • Golden, Linda L.
  • Albaum, Gerald S.

Abstract

With the exploding use of Internet surveys, research efforts and data quality are increasingly subject to the effects of respondents who do not give the required attention to survey questions and who speed through the survey, or who intentionally cheat with their answers. We investigate respondent integrity and data quality for samples drawn from a “Regular” online panel and from Amazon's MTurk. New metrics for assessing sample integrity and online data quality are introduced. Overall, MTurk respondents in both respondent groups took less time to answer questions. The non-USA MTurk group deviated most from correct answers in attention filter questions and had more duplicate IP addresses. In addition, the results from the three Internet sample sources are substantively different. The choice of an Internet survey sample vendor is critical, as it can impact sample composition, respondent integrity, data quality, data structure and substantive results.

Suggested Citation

  • Smith, Scott M. & Roster, Catherine A. & Golden, Linda L. & Albaum, Gerald S., 2016. "A multi-group analysis of online survey respondent data quality: Comparing a regular USA consumer panel to MTurk samples," Journal of Business Research, Elsevier, vol. 69(8), pages 3139-3148.
    • Handle: RePEc:eee:jbrese:v:69:y:2016:i:8:p:3139-3148
    DOI: 10.1016/j.jbusres.2015.12.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S014829631500627X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jbusres.2015.12.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. repec:cup:judgdm:v:5:y:2010:i:5:p:411-419 is not listed on IDEAS
    2. Lee, Hanjoon & Lindquist, Jay D. & Acito, Frank, 1997. "Managers' evaluation of research design and its impact on the use of research: An experimental approach," Journal of Business Research, Elsevier, vol. 39(3), pages 231-240, July.
    3. Peterson, Robert A. & Merunka, Dwight R., 2014. "Convenience samples of college students and research reproducibility," Journal of Business Research, Elsevier, vol. 67(5), pages 1035-1041.
    4. Braunsberger, Karin & Wybenga, Hans & Gates, Roger, 2007. "A comparison of reliability between telephone and web-based surveys," Journal of Business Research, Elsevier, vol. 60(7), pages 758-764, July.
    5. Robert Peterson & Dwight Merunka, 2014. "Convenience samples of college students and research reproducibility," Post-Print hal-01822317, HAL.
    6. Albaum, Gerald S. & Evangelista, Felicitas & Medina, Nila, 1998. "Role of Response Behavior Theory in Survey Research: A Cross-National Study," Journal of Business Research, Elsevier, vol. 42(2), pages 115-125, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. José M. Ramírez-Hurtado & Alfredo G. Hernández-Díaz & Ana D. López-Sánchez & Víctor E. Pérez-León, 2021. "Measuring Online Teaching Service Quality in Higher Education in the COVID-19 Environment," IJERPH, MDPI, vol. 18(5), pages 1-14, March.
    2. Dwayne D. Gremler & Yves Vaerenbergh & Elisabeth C. Brüggen & Kevin P. Gwinner, 2020. "Understanding and managing customer relational benefits in services: a meta-analysis," Journal of the Academy of Marketing Science, Springer, vol. 48(3), pages 565-583, May.
    3. Aysun Ata Aktürk & Hasibe Özlen Demircan, 2018. "Development of Preschool Children Sibling Rivalry Scale (PSRS)," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 11(1), pages 117-136, February.
    4. Joana César Machado & Beatriz Fonseca & Carla Martins, 2021. "Brand logo and brand gender: examining the effects of natural logo designs and color on brand gender perceptions and affect," Journal of Brand Management, Palgrave Macmillan, vol. 28(2), pages 152-170, March.
    5. Lucinda Austin & Barbara Miller Gaither, 2019. "Redefining fit: examining CSR company-issue fit in stigmatized industries," Journal of Brand Management, Palgrave Macmillan, vol. 26(1), pages 9-20, January.
    6. Grigorios, Lamprinakos & Magrizos, Solon & Kostopoulos, Ioannis & Drossos, Dimitrios & Santos, David, 2022. "Overt and covert customer data collection in online personalized advertising: The role of user emotions," Journal of Business Research, Elsevier, vol. 141(C), pages 308-320.
    7. Rašković, Matevž & Ding, Zhonghui & Škare, Vatroslav & Ozretić Došen, Đurđana & Žabkar, Vesna, 2016. "Comparing consumer innovativeness and ethnocentrism of young-adult consumers," Journal of Business Research, Elsevier, vol. 69(9), pages 3682-3686.
    8. Rašković, Matevž & Ding, Zhonghui & Hirose, Morikazu & Žabkar, Vesna & Fam, Kim-Shyan, 2020. "Segmenting young-adult consumers in East Asia and Central and Eastern Europe – The role of consumer ethnocentrism and decision-making styles," Journal of Business Research, Elsevier, vol. 108(C), pages 496-507.
    9. Gyeongcheol Cho & Christopher Schlaegel & Heungsun Hwang & Younyoung Choi & Marko Sarstedt & Christian M. Ringle, 2022. "Integrated Generalized Structured Component Analysis: On the Use of Model Fit Criteria in International Management Research," Management International Review, Springer, vol. 62(4), pages 569-609, August.
    10. Dooyoung Choi & Ha Kyung Lee, 2020. "Beneficiary Foci Types and Performance Appeals in Green Advertising," Sustainability, MDPI, vol. 12(15), pages 1-15, August.
    11. Chenyi He & Ruifeng Liu & Zhifeng Gao & Xin Zhao & Charles A. Sims & Rodolfo M. Nayga, 2021. "Does local label bias consumer taste buds and preference? Evidence of a strawberry sensory experiment," Agribusiness, John Wiley & Sons, Ltd., vol. 37(3), pages 550-568, July.
    12. Mandy Ryan & Nicolas Krucien & Frouke Hermens, 2018. "The eyes have it: Using eye tracking to inform information processing strategies in multi‐attributes choices," Health Economics, John Wiley & Sons, Ltd., vol. 27(4), pages 709-721, April.
    13. Sebastian Lehmann, 2014. "Toward an Understanding of the BDM: Predictive Validity, Gambling Effects, and Risk Attitude," FEMM Working Papers 150001, Otto-von-Guericke University Magdeburg, Faculty of Economics and Management.
    14. Qin, Hong & Peak, Daniel Alan & Prybutok, Victor, 2021. "A virtual market in your pocket: How does mobile augmented reality (MAR) influence consumer decision making?," Journal of Retailing and Consumer Services, Elsevier, vol. 58(C).
    15. Almudena Martínez-Campillo & María del Pilar Sierra-Fernández & Yolanda Fernández-Santos, 2019. "Service-Learning for Sustainability Entrepreneurship in Rural Areas: What Is Its Global Impact on Business University Students?," Sustainability, MDPI, vol. 11(19), pages 1-21, September.
    16. Logan S. Casey & Jesse Chandler & Adam Seth Levine & Andrew Proctor & Dara Z. Strolovitch, 2017. "Intertemporal Differences Among MTurk Workers: Time-Based Sample Variations and Implications for Online Data Collection," SAGE Open, , vol. 7(2), pages 21582440177, June.
    17. Höllig, Christoph E. & Tumasjan, Andranik & Welpe, Isabell M., 2020. "Individualizing gamified systems: The role of trait competitiveness and leaderboard design," Journal of Business Research, Elsevier, vol. 106(C), pages 288-303.
    18. Brian ‘t Hart & Ian Phau, 2022. "Conceptualising attitudes towards brand genuinuity: scale development and validation," Journal of Brand Management, Palgrave Macmillan, vol. 29(4), pages 327-340, July.
    19. Šerić, Maja & Ozretić-Došen, Đurđana & Škare, Vatroslav, 2020. "How can perceived consistency in marketing communications influence customer–brand relationship outcomes?," European Management Journal, Elsevier, vol. 38(2), pages 335-343.
    20. Weng Marc Lim & Gaurav Gupta & Baidyanath Biswas & Rohit Gupta, 2022. "Collaborative consumption continuance: a mixed-methods analysis of the service quality-loyalty relationship in ride-sharing services," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(3), pages 1463-1484, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jbrese:v:69:y:2016:i:8:p:3139-3148. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jbusres .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.