IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0090635.html
   My bibliography  Save this article

A Think Aloud Study Comparing the Validity and Acceptability of Discrete Choice and Best Worst Scaling Methods

Author

Listed:
  • Jennifer A Whitty
  • Ruth Walker
  • Xanthe Golenko
  • Julie Ratcliffe

Abstract

Objectives: This study provides insights into the validity and acceptability of Discrete Choice Experiment (DCE) and profile-case Best Worst Scaling (BWS) methods for eliciting preferences for health care in a priority-setting context. Methods: An adult sample (N = 24) undertook a traditional DCE and a BWS choice task as part of a wider survey on Health Technology Assessment decision criteria. A ‘think aloud’ protocol was applied, whereby participants verbalized their thinking while making choices. Internal validity and acceptability were assessed through a thematic analysis of the decision-making process emerging from the qualitative data and a repeated choice task. Results: A thematic analysis of the decision-making process demonstrated clear evidence of ‘trading’ between multiple attribute/levels for the DCE, and to a lesser extent for the BWS task. Limited evidence consistent with a sequential decision-making model was observed for the BWS task. For the BWS task, some participants found choosing the worst attribute/level conceptually challenging. A desire to provide a complete ranking from best to worst was observed. The majority (18,75%) of participants indicated a preference for DCE, as they felt this enabled comparison of alternative full profiles. Those preferring BWS were averse to choosing an undesirable characteristic that was part of a ‘package’, or perceived BWS to be less ethically conflicting or burdensome. In a repeated choice task, more participants were consistent for the DCE (22,92%) than BWS (10,42%) (p = 0.002). Conclusions: This study supports the validity and acceptability of the traditional DCE format. Findings relating to the application of BWS profile methods are less definitive. Research avenues to further clarify the comparative merits of these preference elicitation methods are identified.

Suggested Citation

  • Jennifer A Whitty & Ruth Walker & Xanthe Golenko & Julie Ratcliffe, 2014. "A Think Aloud Study Comparing the Validity and Acceptability of Discrete Choice and Best Worst Scaling Methods," PLOS ONE, Public Library of Science, vol. 9(4), pages 1-9, April.
    • Handle: RePEc:plo:pone00:0090635
    DOI: 10.1371/journal.pone.0090635
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0090635
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0090635&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0090635?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Yoo, Hong Il & Doiron, Denise, 2013. "The use of alternative preference elicitation methods in complex discrete choice experiments," Journal of Health Economics, Elsevier, vol. 32(6), pages 1166-1179.
    2. Flynn, Terry N. & Louviere, Jordan J. & Peters, Tim J. & Coast, Joanna, 2007. "Best-worst scaling: What it can do for health care research and how to do it," Journal of Health Economics, Elsevier, vol. 26(1), pages 171-189, January.
    3. Joanna Coast, 1999. "The appropriate uses of qualitative methods in health economics," Health Economics, John Wiley & Sons, Ltd., vol. 8(4), pages 345-353, June.
    4. Jordan Louviere & Terry Flynn, 2010. "Using Best-Worst Scaling Choice Experiments to Measure Public Perceptions and Preferences for Healthcare Reform in Australia," The Patient: Patient-Centered Outcomes Research, Springer;International Academy of Health Preference Research, vol. 3(4), pages 275-283, December.
    5. Esther W. de Bekker‐Grob & Mandy Ryan & Karen Gerard, 2012. "Discrete choice experiments in health economics: a review of the literature," Health Economics, John Wiley & Sons, Ltd., vol. 21(2), pages 145-172, February.
    6. Potoglou, Dimitris & Burge, Peter & Flynn, Terry & Netten, Ann & Malley, Juliette & Forder, Julien & Brazier, John E., 2011. "Best-worst scaling vs. discrete choice experiments: An empirical comparison using social care data," Social Science & Medicine, Elsevier, vol. 72(10), pages 1717-1727, May.
    7. Lancsar, Emily & Wildman, John & Donaldson, Cam & Ryan, Mandy & Baker, Rachel, 2011. "Deriving distributional weights for QALYs through discrete choice experiments," Journal of Health Economics, Elsevier, vol. 30(2), pages 466-478, March.
    8. Richard Norman & Jane Hall & Deborah Street & Rosalie Viney, 2013. "Efficiency And Equity: A Stated Preference Approach," Health Economics, John Wiley & Sons, Ltd., vol. 22(5), pages 568-581, May.
    9. Julie Ratcliffe & Terry Flynn & Frances Terlich & Katherine Stevens & John Brazier & Michael Sawyer, 2012. "Developing Adolescent-Specific Health State Values for Economic Evaluation," PharmacoEconomics, Springer, vol. 30(8), pages 713-727, August.
    10. Coast, Joanna & Flynn, Terry N. & Natarajan, Lucy & Sproston, Kerry & Lewis, Jane & Louviere, Jordan J. & Peters, Tim J., 2008. "Valuing the ICECAP capability index for older people," Social Science & Medicine, Elsevier, vol. 67(5), pages 874-882, September.
    11. Mandy Ryan & Karen Gerard & Gillian Currie, 2012. "Using Discrete Choice Experiments in Health Economics," Chapters, in: Andrew M. Jones (ed.), The Elgar Companion to Health Economics, Second Edition, chapter 41, Edward Elgar Publishing.
    12. Anthony H. Harris & Suzanne R. Hill & Geoffrey Chin & Jing Jing Li & Emily Walkom, 2008. "The Role of Value for Money in Public Insurance Coverage Decisions for Drugs in Australia: A Retrospective Analysis 1994-2004," Medical Decision Making, , vol. 28(5), pages 713-722, September.
    13. Julie Ratcliffe & Leah Couzner & Terry Flynn & Michael Sawyer & Katherine Stevens & John Brazier & Leonie Burgess, 2011. "Valuing child health utility 9D health states with a young adolescent sample," Applied Health Economics and Health Policy, Springer, vol. 9(1), pages 15-27, January.
    14. Jennifer Whitty & Paul Scuffham & Sharyn Rundle-Thielee, 2011. "Public and decision maker stated preferences for pharmaceutical subsidy decisions," Applied Health Economics and Health Policy, Springer, vol. 9(2), pages 73-79, March.
    15. Nancy Devlin & David Parkin, 2004. "Does NICE have a cost‐effectiveness threshold and what other factors influence its decisions? A binary choice analysis," Health Economics, John Wiley & Sons, Ltd., vol. 13(5), pages 437-452, May.
    16. N. Flynn, Terry & J. Peters, Tim & Coast, Joanna, 2013. "Quantifying response shift or adaptation effects in quality of life by synthesising best-worst scaling and discrete choice data," Journal of choice modelling, Elsevier, vol. 6(C), pages 34-43.
    17. Mandy Ryan & Verity Watson & Vikki Entwistle, 2009. "Rationalising the ‘irrational’: a think aloud study of discrete choice experiment responses," Health Economics, John Wiley & Sons, Ltd., vol. 18(3), pages 321-336, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Milte, Rachel & Huynh, Elisabeth & Ratcliffe, Julie, 2019. "Assessing quality of care in nursing homes using discrete choice experiments: How does the level of cognitive functioning impact upon older people's preferences?," Social Science & Medicine, Elsevier, vol. 238(C), pages 1-1.
    2. Ratcliffe, Julie & Huynh, Elisabeth & Chen, Gang & Stevens, Katherine & Swait, Joffre & Brazier, John & Sawyer, Michael & Roberts, Rachel & Flynn, Terry, 2016. "Valuing the Child Health Utility 9D: Using profile case best worst scaling methods to develop a new adolescent specific scoring algorithm," Social Science & Medicine, Elsevier, vol. 157(C), pages 48-59.
    3. Smeele, Nicholas V.R. & Chorus, Caspar G. & Schermer, Maartje H.N. & de Bekker-Grob, Esther W., 2023. "Towards machine learning for moral choice analysis in health economics: A literature review and research agenda," Social Science & Medicine, Elsevier, vol. 326(C).
    4. Caroline Vass & Dan Rigby & Kelly Tate & Andrew Stewart & Katherine Payne, 2018. "An Exploratory Application of Eye-Tracking Methods in a Discrete Choice Experiment," Medical Decision Making, , vol. 38(6), pages 658-672, August.
    5. Hutchinson, Claire & Worley, Anthea & Khadka, Jyoti & Milte, Rachel & Cleland, Jenny & Ratcliffe, Julie, 2022. "Do we agree or disagree? A systematic review of the application of preference-based instruments in self and proxy reporting of quality of life in older people," Social Science & Medicine, Elsevier, vol. 305(C).
    6. Tomasz Gajderowicz & Maciej Jakubowski & Sylwia Wrona & Ghadah Alkhadim, 2023. "Is students’ teamwork a dreamwork? A new DCE-based multidimensional approach to preferences towards group work," Palgrave Communications, Palgrave Macmillan, vol. 10(1), pages 1-13, December.
    7. Nicolet, Anna & Perraudin, Clémence & Krucien, Nicolas & Wagner, Joël & Peytremann-Bridevaux, Isabelle & Marti, Joachim, 2023. "Preferences of older adults for healthcare models designed to improve care coordination: Evidence from Western Switzerland," Health Policy, Elsevier, vol. 132(C).
    8. Galárraga, Omar & Kuo, Caroline & Mtukushe, Bulelwa & Maughan-Brown, Brendan & Harrison, Abigail & Hoare, Jackie, 2020. "iSAY (incentives for South African youth): Stated preferences of young people living with HIV," Social Science & Medicine, Elsevier, vol. 265(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lancsar, Emily & Louviere, Jordan & Donaldson, Cam & Currie, Gillian & Burgess, Leonie, 2013. "Best worst discrete choice experiments in health: Methods and an application," Social Science & Medicine, Elsevier, vol. 76(C), pages 74-82.
    2. Nicolas Krucien & Verity Watson & Mandy Ryan, 2017. "Is Best–Worst Scaling Suitable for Health State Valuation? A Comparison with Discrete Choice Experiments," Health Economics, John Wiley & Sons, Ltd., vol. 26(12), pages 1-16, December.
    3. Denise Doiron & Hong Il Yoo, 2020. "Stated preferences over job characteristics: A panel study," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 53(1), pages 43-82, February.
    4. Aizaki, Hideo & Fogarty, James, 2019. "An R package and tutorial for case 2 best–worst scaling," Journal of choice modelling, Elsevier, vol. 32(C), pages 1-1.
    5. Emily Lancsar & Peter Burge, 2014. "Choice modelling research in health economics," Chapters, in: Stephane Hess & Andrew Daly (ed.), Handbook of Choice Modelling, chapter 28, pages 675-687, Edward Elgar Publishing.
    6. Marta Trapero-Bertran & Beatriz Rodríguez-Martín & Julio López-Bastida, 2019. "What attributes should be included in a discrete choice experiment related to health technologies? A systematic literature review," PLOS ONE, Public Library of Science, vol. 14(7), pages 1-15, July.
    7. Jiang, Shan & Gu, Yuanyuan & Yang, Fan & Wu, Tao & Wang, Hui & Cutler, Henry & Zhang, Lufa, 2020. "Tertiary hospitals or community clinics? An enquiry into the factors affecting patients' choice for healthcare facilities in urban China," China Economic Review, Elsevier, vol. 63(C).
    8. Osman, Ahmed M.Y. & Wu, Jing & He, Xiaoning & Chen, Gang, 2021. "Eliciting SF-6Dv2 health state utilities using an anchored best-worst scaling technique," Social Science & Medicine, Elsevier, vol. 279(C).
    9. Yoo, Hong Il & Doiron, Denise, 2013. "The use of alternative preference elicitation methods in complex discrete choice experiments," Journal of Health Economics, Elsevier, vol. 32(6), pages 1166-1179.
    10. Gu, Yuanyuan & Lancsar, Emily & Ghijben, Peter & Butler, James RG & Donaldson, Cam, 2015. "Attributes and weights in health care priority setting: A systematic review of what counts and to what extent," Social Science & Medicine, Elsevier, vol. 146(C), pages 41-52.
    11. Terry N. Flynn & Elisabeth Huynh & Tim J. Peters & Hareth Al‐Janabi & Sam Clemens & Alison Moody & Joanna Coast, 2015. "Scoring the Icecap‐a Capability Instrument. Estimation of a UK General Population Tariff," Health Economics, John Wiley & Sons, Ltd., vol. 24(3), pages 258-269, March.
    12. Milte, Rachel & Huynh, Elisabeth & Ratcliffe, Julie, 2019. "Assessing quality of care in nursing homes using discrete choice experiments: How does the level of cognitive functioning impact upon older people's preferences?," Social Science & Medicine, Elsevier, vol. 238(C), pages 1-1.
    13. Whitty, Jennifer A. & Littlejohns, Peter, 2015. "Social values and health priority setting in Australia: An analysis applied to the context of health technology assessment," Health Policy, Elsevier, vol. 119(2), pages 127-136.
    14. Mina Bahrampour & Joshua Byrnes & Richard Norman & Paul A. Scuffham & Martin Downes, 2020. "Discrete choice experiments to generate utility values for multi-attribute utility instruments: a systematic review of methods," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 21(7), pages 983-992, September.
    15. Caroline Vass & Dan Rigby & Katherine Payne, 2017. "The Role of Qualitative Research Methods in Discrete Choice Experiments," Medical Decision Making, , vol. 37(3), pages 298-313, April.
    16. Huynh, Elisabeth & Coast, Joanna & Rose, John & Kinghorn, Philip & Flynn, Terry, 2017. "Values for the ICECAP-Supportive Care Measure (ICECAP-SCM) for use in economic evaluation at end of life," Social Science & Medicine, Elsevier, vol. 189(C), pages 114-128.
    17. Qinxin Guo & Junyi Shen, 2019. "An Empirical Comparison Between Discrete Choice Experiment and Best-worst Scaling: A Case Study of Mobile Payment Choice," Discussion Paper Series DP2019-14, Research Institute for Economics & Business Administration, Kobe University.
    18. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    19. Nikita Arora & Matthew Quaife & Kara Hanson & Mylene Lagarde & Dorka Woldesenbet & Abiy Seifu & Romain Crastes dit Sourd, 2022. "Discrete choice analysis of health worker job preferences in Ethiopia: Separating attribute non‐attendance from taste heterogeneity," Health Economics, John Wiley & Sons, Ltd., vol. 31(5), pages 806-819, May.
    20. Hajji, Assma & Trukeschitz, Birgit & Malley, Juliette & Batchelder, Laurie & Saloniki, Eirini & Linnosmaa, Ismo & Lu, Hui, 2020. "Population-based preference weights for the Adult Social Care Outcomes Toolkit (ASCOT) for service users for Austria: Findings from a best-worst experiment," Social Science & Medicine, Elsevier, vol. 250(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0090635. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.