IDEAS home Printed from https://ideas.repec.org/a/bla/jamist/v61y2010i1p50-69.html
   My bibliography  Save this article

The development and evaluation of a survey to measure user engagement

Author

Listed:
  • Heather L. O'Brien
  • Elaine G. Toms

Abstract

Facilitating engaging user experiences is essential in the design of interactive systems. To accomplish this, it is necessary to understand the composition of this construct and how to evaluate it. Building on previous work that posited a theory of engagement and identified a core set of attributes that operationalized this construct, we constructed and evaluated a multidimensional scale to measure user engagement. In this paper we describe the development of the scale, as well as two large‐scale studies (N=440 and N=802) that were undertaken to assess its reliability and validity in online shopping environments. In the first we used Reliability Analysis and Exploratory Factor Analysis to identify six attributes of engagement: Perceived Usability, Aesthetics, Focused Attention, Felt Involvement, Novelty, and Endurability. In the second we tested the validity of and relationships among those attributes using Structural Equation Modeling. The result of this research is a multidimensional scale that may be used to test the engagement of software applications. In addition, findings indicate that attributes of engagement are highly intertwined, a complex interplay of user‐system interaction variables. Notably, Perceived Usability played a mediating role in the relationship between Endurability and Novelty, Aesthetics, Felt Involvement, and Focused Attention.

Suggested Citation

  • Heather L. O'Brien & Elaine G. Toms, 2010. "The development and evaluation of a survey to measure user engagement," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(1), pages 50-69, January.
    • Handle: RePEc:bla:jamist:v:61:y:2010:i:1:p:50-69
    DOI: 10.1002/asi.21229
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/asi.21229
    Download Restriction: no
    File URL: https://libkey.io/10.1002/asi.21229?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pagani, Margherita & Malacarne, Giovanni, 2017. "Experiential Engagement and Active vs. Passive Behavior in Mobile Location-based Social Networks: The Moderating Role of Privacy," Journal of Interactive Marketing, Elsevier, vol. 37(C), pages 133-148.
    2. Bitrián, Paula & Buil, Isabel & Catalán, Sara, 2021. "Enhancing user engagement: The role of gamification in mobile apps," Journal of Business Research, Elsevier, vol. 132(C), pages 170-185.
    3. Carolina Herrando & Julio Jiménez-Martínez & María José Martín-De Hoyos, 2017. "Passion at first sight: how to engage users in social commerce contexts," Electronic Commerce Research, Springer, vol. 17(4), pages 701-720, December.
    4. Nripesh Trivedi & Daniel Adomako Asamoah & Derek Doran, 0. "Keep the conversations going: engagement-based customer segmentation on online social service platforms," Information Systems Frontiers, Springer, vol. 0, pages 1-19.
    5. Makram Soui & Mabrouka Chouchane & Narjes Bessghaier & Mohamed Wiem Mkaouer & Marouane Kessentini, 2022. "On the Impact of Aesthetic Defects on the Maintainability of Mobile Graphical User Interfaces: An Empirical Study," Information Systems Frontiers, Springer, vol. 24(2), pages 659-676, April.
    6. Tamilla Triantoro & Ram Gopal & Raquel Benbunan-Fich & Guido Lang, 2020. "Personality and games: enhancing online surveys through gamification," Information Technology and Management, Springer, vol. 21(3), pages 169-178, September.
    7. Grant, Stephanie M. & Hodge, Frank D. & Sinha, Roshan K., 2018. "How disclosure medium affects investor reactions to CEO bragging, modesty, and humblebragging," Accounting, Organizations and Society, Elsevier, vol. 68, pages 118-134.
    8. Tamilla Triantoro & Ram Gopal & Raquel Benbunan-Fich & Guido Lang, 0. "Personality and games: enhancing online surveys through gamification," Information Technology and Management, Springer, vol. 0, pages 1-10.
    9. Zhuojun Gu & Ravi Bapna & Jason Chan & Alok Gupta, 2022. "Measuring the Impact of Crowdsourcing Features on Mobile App User Engagement and Retention: A Randomized Field Experiment," Management Science, INFORMS, vol. 68(2), pages 1297-1329, February.
    10. Orland Hoeber & Morgan Harvey & Shaheed Ahmed Dewan Sagar & Matthew Pointon, 2022. "The effects of simulated interruptions on mobile search tasks," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(6), pages 777-796, June.
    11. Omar Besbes & Yonatan Gur & Assaf Zeevi, 2016. "Optimization in Online Content Recommendation Services: Beyond Click-Through Rates," Manufacturing & Service Operations Management, INFORMS, vol. 18(1), pages 15-33, February.
    12. Caitríona Ní Shé & Eabhnat Ní Fhloinn & Ciarán Mac an Bhaird, 2023. "Student Engagement with Technology-Enhanced Resources in Mathematics in Higher Education: A Review," Mathematics, MDPI, vol. 11(3), pages 1-34, February.
    13. Tina Katika & Ioannis Karaseitanidis & Dimitra Tsiakou & Christos Makropoulos & Angelos Amditis, 2022. "Augmented Reality (AR) Supporting Citizen Engagement in Circular Economy," Circular Economy and Sustainability,, Springer.
    14. Zixing Shen & Michael J. Pritchard, 2022. "Cognitive engagement on social media: A study of the effects of visual cueing in educational videos," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(9), pages 1253-1267, September.
    15. Simona VINEREAN & Alin OPREANA, 2015. "Consumer Engagement in Online Settings: Conceptualization and Validation of Measurement Scales," Expert Journal of Marketing, Sprint Investify, vol. 3(2), pages 35-50.
    16. Shelly Rathee & Kritika Narula & Arul Mishra & Himanshu Mishra, 2023. "Alphanumeric vs. Numeric Token Systems and the Healthcare Experience: Field Evidence from Healthcare Delivery in India," Management Science, INFORMS, vol. 69(2), pages 1180-1221, February.
    17. Diana R. Sanchez & Markus Langer, 2020. "Video Game Pursuit (VGPu) Scale Development: Designing and Validating a Scale With Implications for Game-Based Learning and Assessment," Simulation & Gaming, , vol. 51(1), pages 55-86, February.
    18. Heather L. O'Brien, 2017. "Antecedents and learning outcomes of online news engagement," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(12), pages 2809-2820, December.
    19. Jong-Long Guo & Hsiao-Pei Hsu & Mei-Hsun Lin & Cheng-Yu Lin & Chiu-Mieh Huang, 2020. "Testing the Usability of Digital Educational Games for Encouraging Smoking Cessation," IJERPH, MDPI, vol. 17(8), pages 1-14, April.
    20. Nripesh Trivedi & Daniel Adomako Asamoah & Derek Doran, 2018. "Keep the conversations going: engagement-based customer segmentation on online social service platforms," Information Systems Frontiers, Springer, vol. 20(2), pages 239-257, April.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jamist:v:61:y:2010:i:1:p:50-69. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.asis.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.