IDEAS home Printed from https://ideas.repec.org/a/eee/socmed/v386y2025ics027795362500992x.html

Preference inversion in discrete choice experiments: A novel test of the emoji scale

Author

Listed:
  • Thai, Thao
  • Engel, Lidia
  • Ride, Jemimah
  • Mulhern, Brendan
  • Norman, Richard
  • Mihalopoulos, Cathrine

Abstract

Preference inversion - where preference estimates do not align with the hierarchical order of attribute levels-remains a concern in health valuation studies. To address this, we developed a novel visual aid, the emoji scale, comprising five Apple Unicode emoji characters to represent five levels of a health-related quality-of-life measure. Using a case study of the Recovering Quality of Life – Utility index instrument, we explored whether the emoji scale could reduce preference inversion and improve respondent experience in discrete choice experiments (DCEs). We conducted a between-respondent comparison involving 2024 Australian adults, randomly allocated to one of four arms:(1) standard text-only choice tasks, (2) emoji & text (combined), (3) emoji-only (no text), and (4) text with purple colour-coding. We estimated value sets derived from each format and assessed differences in perceived task difficulty, engagement, and preference inversion using t-tests and model comparisons. Results showed that utility decrements from the emoji & text were closely aligned with those from the standard text-only format. Importantly, the emoji & text reduced preference inversion and was associated with lower reported task difficulty and higher engagement compared to other formats. These findings suggest that pairing emojis with text may enhance respondent understanding and data quality in health valuation tasks, without compromising the validity of the resulting value sets. While further work is necessary to fully understand the legitimacy of the emoji scale as an effective visual aid for conveying attribute level ordering, the emoji & text format offers a promising approach to improving DCEs.

Suggested Citation

  • Thai, Thao & Engel, Lidia & Ride, Jemimah & Mulhern, Brendan & Norman, Richard & Mihalopoulos, Cathrine, 2025. "Preference inversion in discrete choice experiments: A novel test of the emoji scale," Social Science & Medicine, Elsevier, vol. 386(C).
    • Handle: RePEc:eee:socmed:v:386:y:2025:i:c:s027795362500992x
    DOI: 10.1016/j.socscimed.2025.118661
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S027795362500992X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.socscimed.2025.118661?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Haode Wang & Donna L. Rowen & John E. Brazier & Litian Jiang, 2023. "Discrete Choice Experiments in Health State Valuation: A Systematic Review of Progress and New Trends," Applied Health Economics and Health Policy, Springer, vol. 21(3), pages 405-418, May.
    2. Bansback, Nick & Brazier, John & Tsuchiya, Aki & Anis, Aslam, 2012. "Using a discrete choice experiment to estimate health state utility values," Journal of Health Economics, Elsevier, vol. 31(1), pages 306-318.
    3. Feng Xie & Eleanor Pullenayegum & Kathryn Gaebel & Mark Oppe & Paul Krabbe, 2014. "Eliciting preferences to the EQ-5D-5L health states: discrete choice experiment or multiprofile case of best–worst scaling?," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 15(3), pages 281-288, April.
    4. Richard Norman & Paula Cronin & Rosalie Viney, 2013. "A Pilot Discrete Choice Experiment to Explore Preferences for EQ-5D-5L Health States," Applied Health Economics and Health Policy, Springer, vol. 11(3), pages 287-298, June.
    5. Hess, Stephane & Train, Kenneth, 2017. "Correlation and scale in mixed logit models," Journal of choice modelling, Elsevier, vol. 23(C), pages 1-8.
    6. John M. Miyamoto & Peter P. Wakker & Han Bleichrodt & Hans J. M. Peters, 1998. "The Zero-Condition: A Simplifying Assumption in QALY Measurement and Multiattribute Utility," Management Science, INFORMS, vol. 44(6), pages 839-849, June.
    7. Mathieu F. Janssen & Ines Buchholz & Dominik Golicki & Gouke J. Bonsel, 2022. "Is EQ-5D-5L Better Than EQ-5D-3L Over Time? A Head-to-Head Comparison of Responsiveness of Descriptive Systems and Value Sets from Nine Countries," PharmacoEconomics, Springer, vol. 40(11), pages 1081-1093, November.
    8. Brendan Mulhern & Richard Norman & Deborah J. Street & Rosalie Viney, 2019. "One Method, Many Methodological Choices: A Structured Review of Discrete-Choice Experiments for Health State Valuation," PharmacoEconomics, Springer, vol. 37(1), pages 29-43, January.
    9. Brazier, John & Roberts, Jennifer & Deverill, Mark, 2002. "The estimation of a preference-based measure of health from the SF-36," Journal of Health Economics, Elsevier, vol. 21(2), pages 271-292, March.
    10. Mandy Ryan & Verity Watson & Vikki Entwistle, 2009. "Rationalising the ‘irrational’: a think aloud study of discrete choice experiment responses," Health Economics, John Wiley & Sons, Ltd., vol. 18(3), pages 321-336, March.
    11. Brendan Mulhern & Richard Norman & Paula Lorgelly & Emily Lancsar & Julie Ratcliffe & John Brazier & Rosalie Viney, 2017. "Is Dimension Order Important when Valuing Health States Using Discrete Choice Experiments Including Duration?," PharmacoEconomics, Springer, vol. 35(4), pages 439-451, April.
    12. Richard Norman & Brendan Mulhern & Emily Lancsar & Paula Lorgelly & Julie Ratcliffe & Deborah Street & Rosalie Viney, 2023. "The Use of a Discrete Choice Experiment Including Both Duration and Dead for the Development of an EQ-5D-5L Value Set for Australia," PharmacoEconomics, Springer, vol. 41(4), pages 427-438, April.
    13. Himmler, Sebastian & Soekhai, Vikas & van Exel, Job & Brouwer, Werner, 2021. "What works better for preference elicitation among older people? Cognitive burden of discrete choice experiment and case 2 best-worst scaling in an online setting," Journal of choice modelling, Elsevier, vol. 38(C).
    14. Marcel F. Jonker & Bas Donkers & Esther de Bekker‐Grob & Elly A. Stolk, 2019. "Attribute level overlap (and color coding) can reduce task complexity, improve choice consistency, and decrease the dropout rate in discrete choice experiments," Health Economics, John Wiley & Sons, Ltd., vol. 28(3), pages 350-363, March.
    15. Thao Thai & Michiel Bliemer & Gang Chen & Jean Spinks & Sonja de New & Emily Lancsar, 2023. "Comparison of a full and partial choice set design in a labeled discrete choice experiment," Health Economics, John Wiley & Sons, Ltd., vol. 32(6), pages 1284-1304, June.
    16. Amanda Cole & Koonal Shah & Brendan Mulhern & Yan Feng & Nancy Devlin, 2018. "Valuing EQ-5D-5L health states ‘in context’ using a discrete choice experiment," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 19(4), pages 595-605, May.
    17. Benjamin M. Craig & A. Simon Pickard & Elly Stolk & John E. Brazier, 2013. "US Valuation of the SF-6D," Medical Decision Making, , vol. 33(6), pages 793-803, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hosein Ameri & Thomas G. Poder, 2025. "Comparison of four approaches in eliciting health state utilities with SF-6Dv2," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 26(4), pages 589-604, June.
    2. Haode Wang & Donna L. Rowen & Yuen Chen & Clara Mukuria & Deborah Street & Richard Norman, 2026. "Valuing health and wellbeing using discrete choice experiment: exploring feasibility, design effect and international preference similarity," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 27(2), pages 339-353, March.
    3. Mina Bahrampour & Joshua Byrnes & Richard Norman & Paul A. Scuffham & Martin Downes, 2020. "Discrete choice experiments to generate utility values for multi-attribute utility instruments: a systematic review of methods," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 21(7), pages 983-992, September.
    4. Poder, Thomas G. & Ameri, Hosein, 2025. "Comparison of methods for anchoring latent values on the full health-dead utility scale," Social Science & Medicine, Elsevier, vol. 383(C).
    5. Sullivan, Trudy & McCarty, Georgia & Ombler, Franz & Turner, Robin & Mulhern, Brendan & Hansen, Paul, 2024. "Creating an SF-6Dv2 social value set for New Zealand," Social Science & Medicine, Elsevier, vol. 354(C).
    6. Brendan Mulhern & Richard Norman & Deborah J. Street & Rosalie Viney, 2019. "One Method, Many Methodological Choices: A Structured Review of Discrete-Choice Experiments for Health State Valuation," PharmacoEconomics, Springer, vol. 37(1), pages 29-43, January.
    7. Richard Norman & Rebecca Mercieca‐Bebber & Donna Rowen & John E. Brazier & David Cella & A. Simon Pickard & Deborah J. Street & Rosalie Viney & Dennis Revicki & Madeleine T. King & On behalf of the Eu, 2019. "U.K. utility weights for the EORTC QLU‐C10D," Health Economics, John Wiley & Sons, Ltd., vol. 28(12), pages 1385-1401, December.
    8. Spencer, Anne & Rivero-Arias, Oliver & Wong, Ruth & Tsuchiya, Aki & Bleichrodt, Han & Edwards, Rhiannon Tudor & Norman, Richard & Lloyd, Andrew & Clarke, Philip, 2022. "The QALY at 50: One story many voices," Social Science & Medicine, Elsevier, vol. 296(C).
    9. Amanda Cole & Koonal Shah & Brendan Mulhern & Yan Feng & Nancy Devlin, 2018. "Valuing EQ-5D-5L health states ‘in context’ using a discrete choice experiment," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 19(4), pages 595-605, May.
    10. Thai, Thao & Lancsar, Emily & Spinks, Jean & Freeman, Christopher & Chen, Gang, 2024. "Understanding Australian pharmacy degree holders’ job preferences through the lens of motivation-hygiene theory," Social Science & Medicine, Elsevier, vol. 348(C).
    11. Osman, Ahmed M.Y. & Wu, Jing & He, Xiaoning & Chen, Gang, 2021. "Eliciting SF-6Dv2 health state utilities using an anchored best-worst scaling technique," Social Science & Medicine, Elsevier, vol. 279(C).
    12. Poder, Thomas G. & Ameri, Hosein, 2024. "A novel approach for health state valuation: Multiple bounded dichotomous choice compared to the traditional standard gamble," Social Science & Medicine, Elsevier, vol. 357(C).
    13. Bansback, Nick & Hole, Arne Risa & Mulhern, Brendan & Tsuchiya, Aki, 2014. "Testing a discrete choice experiment including duration to value health states for large descriptive systems: Addressing design and sampling issues," Social Science & Medicine, Elsevier, vol. 114(C), pages 38-48.
    14. Elliott, Jack & Tsuchiya, Aki, 2022. "Do they just know more, or do they also have different preferences? An exploratory analysis of the effects of self-reporting serious health problems on health state valuation," Social Science & Medicine, Elsevier, vol. 315(C).
    15. Nicolas Krucien & Jonathan Sicsic & Mandy Ryan, 2019. "For better or worse? Investigating the validity of best–worst discrete choice experiments in health," Health Economics, John Wiley & Sons, Ltd., vol. 28(4), pages 572-586, April.
    16. John Brazier & Donna Rowen & Yaling Yang & Aki Tsuchiya, 2012. "Comparison of health state utility values derived using time trade-off, rank and discrete choice data anchored on the full health-dead scale," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 13(5), pages 575-587, October.
    17. Bram Roudijk & A. Rogier T. Donders & Peep F.M. Stalmeier, 2018. "Setting Dead at Zero: Applying Scale Properties to the QALY Model," Medical Decision Making, , vol. 38(6), pages 627-634, August.
    18. Nicolas Krucien & Verity Watson & Mandy Ryan, 2017. "Is Best–Worst Scaling Suitable for Health State Valuation? A Comparison with Discrete Choice Experiments," Health Economics, John Wiley & Sons, Ltd., vol. 26(12), pages 1-16, December.
    19. Marcel F. Jonker & Richard Norman, 2022. "Not all respondents use a multiplicative utility function in choice experiments for health state valuations, which should be reflected in the elicitation format (or statistical analysis)," Health Economics, John Wiley & Sons, Ltd., vol. 31(2), pages 431-439, February.
    20. Vikas Soekhai & Esther W. Bekker-Grob & Alan R. Ellis & Caroline M. Vass, 2019. "Discrete Choice Experiments in Health Economics: Past, Present and Future," PharmacoEconomics, Springer, vol. 37(2), pages 201-226, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:socmed:v:386:y:2025:i:c:s027795362500992x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/315/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.