IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0090635.html
   My bibliography  Save this article

A Think Aloud Study Comparing the Validity and Acceptability of Discrete Choice and Best Worst Scaling Methods

Author

Listed:
  • Jennifer A Whitty
  • Ruth Walker
  • Xanthe Golenko
  • Julie Ratcliffe

Abstract

Objectives: This study provides insights into the validity and acceptability of Discrete Choice Experiment (DCE) and profile-case Best Worst Scaling (BWS) methods for eliciting preferences for health care in a priority-setting context. Methods: An adult sample (N = 24) undertook a traditional DCE and a BWS choice task as part of a wider survey on Health Technology Assessment decision criteria. A ‘think aloud’ protocol was applied, whereby participants verbalized their thinking while making choices. Internal validity and acceptability were assessed through a thematic analysis of the decision-making process emerging from the qualitative data and a repeated choice task. Results: A thematic analysis of the decision-making process demonstrated clear evidence of ‘trading’ between multiple attribute/levels for the DCE, and to a lesser extent for the BWS task. Limited evidence consistent with a sequential decision-making model was observed for the BWS task. For the BWS task, some participants found choosing the worst attribute/level conceptually challenging. A desire to provide a complete ranking from best to worst was observed. The majority (18,75%) of participants indicated a preference for DCE, as they felt this enabled comparison of alternative full profiles. Those preferring BWS were averse to choosing an undesirable characteristic that was part of a ‘package’, or perceived BWS to be less ethically conflicting or burdensome. In a repeated choice task, more participants were consistent for the DCE (22,92%) than BWS (10,42%) (p = 0.002). Conclusions: This study supports the validity and acceptability of the traditional DCE format. Findings relating to the application of BWS profile methods are less definitive. Research avenues to further clarify the comparative merits of these preference elicitation methods are identified.

Suggested Citation

  • Jennifer A Whitty & Ruth Walker & Xanthe Golenko & Julie Ratcliffe, 2014. "A Think Aloud Study Comparing the Validity and Acceptability of Discrete Choice and Best Worst Scaling Methods," PLOS ONE, Public Library of Science, vol. 9(4), pages 1-9, April.
  • Handle: RePEc:plo:pone00:0090635
    DOI: 10.1371/journal.pone.0090635
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0090635
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0090635&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0090635?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Yoo, Hong Il & Doiron, Denise, 2013. "The use of alternative preference elicitation methods in complex discrete choice experiments," Journal of Health Economics, Elsevier, vol. 32(6), pages 1166-1179.
    2. Flynn, Terry N. & Louviere, Jordan J. & Peters, Tim J. & Coast, Joanna, 2007. "Best-worst scaling: What it can do for health care research and how to do it," Journal of Health Economics, Elsevier, vol. 26(1), pages 171-189, January.
    3. Joanna Coast, 1999. "The appropriate uses of qualitative methods in health economics," Health Economics, John Wiley & Sons, Ltd., vol. 8(4), pages 345-353, June.
    4. Mandy Ryan & Karen Gerard & Gillian Currie, 2012. "Using Discrete Choice Experiments in Health Economics," Chapters, in: Andrew M. Jones (ed.), The Elgar Companion to Health Economics, Second Edition, chapter 41, Edward Elgar Publishing.
    5. Jordan Louviere & Terry Flynn, 2010. "Using Best-Worst Scaling Choice Experiments to Measure Public Perceptions and Preferences for Healthcare Reform in Australia," The Patient: Patient-Centered Outcomes Research, Springer;International Academy of Health Preference Research, vol. 3(4), pages 275-283, December.
    6. Esther W. de Bekker‐Grob & Mandy Ryan & Karen Gerard, 2012. "Discrete choice experiments in health economics: a review of the literature," Health Economics, John Wiley & Sons, Ltd., vol. 21(2), pages 145-172, February.
    7. Potoglou, Dimitris & Burge, Peter & Flynn, Terry & Netten, Ann & Malley, Juliette & Forder, Julien & Brazier, John E., 2011. "Best-worst scaling vs. discrete choice experiments: An empirical comparison using social care data," Social Science & Medicine, Elsevier, vol. 72(10), pages 1717-1727, May.
    8. Lancsar, Emily & Wildman, John & Donaldson, Cam & Ryan, Mandy & Baker, Rachel, 2011. "Deriving distributional weights for QALYs through discrete choice experiments," Journal of Health Economics, Elsevier, vol. 30(2), pages 466-478, March.
    9. Jennifer Whitty & Paul Scuffham & Sharyn Rundle-Thielee, 2011. "Public and decision maker stated preferences for pharmaceutical subsidy decisions," Applied Health Economics and Health Policy, Springer, vol. 9(2), pages 73-79, March.
    10. Richard Norman & Jane Hall & Deborah Street & Rosalie Viney, 2013. "Efficiency And Equity: A Stated Preference Approach," Health Economics, John Wiley & Sons, Ltd., vol. 22(5), pages 568-581, May.
    11. Nancy Devlin & David Parkin, 2004. "Does NICE have a cost‐effectiveness threshold and what other factors influence its decisions? A binary choice analysis," Health Economics, John Wiley & Sons, Ltd., vol. 13(5), pages 437-452, May.
    12. N. Flynn, Terry & J. Peters, Tim & Coast, Joanna, 2013. "Quantifying response shift or adaptation effects in quality of life by synthesising best-worst scaling and discrete choice data," Journal of choice modelling, Elsevier, vol. 6(C), pages 34-43.
    13. Coast, Joanna & Flynn, Terry N. & Natarajan, Lucy & Sproston, Kerry & Lewis, Jane & Louviere, Jordan J. & Peters, Tim J., 2008. "Valuing the ICECAP capability index for older people," Social Science & Medicine, Elsevier, vol. 67(5), pages 874-882, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Caroline Vass & Dan Rigby & Kelly Tate & Andrew Stewart & Katherine Payne, 2018. "An Exploratory Application of Eye-Tracking Methods in a Discrete Choice Experiment," Medical Decision Making, , vol. 38(6), pages 658-672, August.
    2. Galárraga, Omar & Kuo, Caroline & Mtukushe, Bulelwa & Maughan-Brown, Brendan & Harrison, Abigail & Hoare, Jackie, 2020. "iSAY (incentives for South African youth): Stated preferences of young people living with HIV," Social Science & Medicine, Elsevier, vol. 265(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Denise Doiron & Hong Il Yoo, 2020. "Stated preferences over job characteristics: A panel study," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 53(1), pages 43-82, February.
    2. Lancsar, Emily & Louviere, Jordan & Donaldson, Cam & Currie, Gillian & Burgess, Leonie, 2013. "Best worst discrete choice experiments in health: Methods and an application," Social Science & Medicine, Elsevier, vol. 76(C), pages 74-82.
    3. Nicolas Krucien & Verity Watson & Mandy Ryan, 2017. "Is Best–Worst Scaling Suitable for Health State Valuation? A Comparison with Discrete Choice Experiments," Health Economics, John Wiley & Sons, Ltd., vol. 26(12), pages 1-16, December.
    4. Aizaki, Hideo & Fogarty, James, 2019. "An R package and tutorial for case 2 best–worst scaling," Journal of choice modelling, Elsevier, vol. 32(C), pages 1-1.
    5. Emily Lancsar & Peter Burge, 2014. "Choice modelling research in health economics," Chapters, in: Stephane Hess & Andrew Daly (ed.), Handbook of Choice Modelling, chapter 28, pages 675-687, Edward Elgar Publishing.
    6. Jiang, Shan & Gu, Yuanyuan & Yang, Fan & Wu, Tao & Wang, Hui & Cutler, Henry & Zhang, Lufa, 2020. "Tertiary hospitals or community clinics? An enquiry into the factors affecting patients' choice for healthcare facilities in urban China," China Economic Review, Elsevier, vol. 63(C).
    7. Caroline Vass & Dan Rigby & Katherine Payne, 2017. "The Role of Qualitative Research Methods in Discrete Choice Experiments," Medical Decision Making, , vol. 37(3), pages 298-313, April.
    8. Yoo, Hong Il & Doiron, Denise, 2013. "The use of alternative preference elicitation methods in complex discrete choice experiments," Journal of Health Economics, Elsevier, vol. 32(6), pages 1166-1179.
    9. Huynh, Elisabeth & Coast, Joanna & Rose, John & Kinghorn, Philip & Flynn, Terry, 2017. "Values for the ICECAP-Supportive Care Measure (ICECAP-SCM) for use in economic evaluation at end of life," Social Science & Medicine, Elsevier, vol. 189(C), pages 114-128.
    10. Gu, Yuanyuan & Lancsar, Emily & Ghijben, Peter & Butler, James RG & Donaldson, Cam, 2015. "Attributes and weights in health care priority setting: A systematic review of what counts and to what extent," Social Science & Medicine, Elsevier, vol. 146(C), pages 41-52.
    11. Qinxin Guo & Junyi Shen, 2019. "An Empirical Comparison Between Discrete Choice Experiment and Best-worst Scaling: A Case Study of Mobile Payment Choice," Discussion Paper Series DP2019-14, Research Institute for Economics & Business Administration, Kobe University.
    12. Marta Trapero-Bertran & Beatriz Rodríguez-Martín & Julio López-Bastida, 2019. "What attributes should be included in a discrete choice experiment related to health technologies? A systematic literature review," PLOS ONE, Public Library of Science, vol. 14(7), pages 1-15, July.
    13. Hajji, Assma & Trukeschitz, Birgit & Malley, Juliette & Batchelder, Laurie & Saloniki, Eirini & Linnosmaa, Ismo & Lu, Hui, 2020. "Population-based preference weights for the Adult Social Care Outcomes Toolkit (ASCOT) for service users for Austria: Findings from a best-worst experiment," Social Science & Medicine, Elsevier, vol. 250(C).
    14. Genie, Mesfin G. & Nicoló, Antonio & Pasini, Giacomo, 2020. "The role of heterogeneity of patients’ preferences in kidney transplantation," Journal of Health Economics, Elsevier, vol. 72(C).
    15. Mitchell, Paul Mark & Roberts, Tracy E. & Barton, Pelham M. & Coast, Joanna, 2015. "Assessing sufficient capability: A new approach to economic evaluation," Social Science & Medicine, Elsevier, vol. 139(C), pages 71-79.
    16. Osman, Ahmed M.Y. & Wu, Jing & He, Xiaoning & Chen, Gang, 2021. "Eliciting SF-6Dv2 health state utilities using an anchored best-worst scaling technique," Social Science & Medicine, Elsevier, vol. 279(C).
    17. Milte, Rachel & Huynh, Elisabeth & Ratcliffe, Julie, 2019. "Assessing quality of care in nursing homes using discrete choice experiments: How does the level of cognitive functioning impact upon older people's preferences?," Social Science & Medicine, Elsevier, vol. 238(C), pages 1-1.
    18. Whitty, Jennifer A. & Littlejohns, Peter, 2015. "Social values and health priority setting in Australia: An analysis applied to the context of health technology assessment," Health Policy, Elsevier, vol. 119(2), pages 127-136.
    19. Bansback, Nick & Hole, Arne Risa & Mulhern, Brendan & Tsuchiya, Aki, 2014. "Testing a discrete choice experiment including duration to value health states for large descriptive systems: Addressing design and sampling issues," Social Science & Medicine, Elsevier, vol. 114(C), pages 38-48.
    20. Marti, Joachim, 2012. "A best–worst scaling survey of adolescents' level of concern for health and non-health consequences of smoking," Social Science & Medicine, Elsevier, vol. 75(1), pages 87-97.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0090635. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://journals.plos.org/plosone/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.