IDEAS home Printed from https://ideas.repec.org/a/wly/hlthec/v28y2019i3p350-363.html
   My bibliography  Save this article

Attribute level overlap (and color coding) can reduce task complexity, improve choice consistency, and decrease the dropout rate in discrete choice experiments

Author

Listed:
  • Marcel F. Jonker
  • Bas Donkers
  • Esther de Bekker‐Grob
  • Elly A. Stolk

Abstract

A randomized controlled discrete choice experiment (DCE) with 3,320 participating respondents was used to investigate the individual and combined impact of level overlap and color coding on task complexity, choice consistency, survey satisfaction scores, and dropout rates. The systematic differences between the study arms allowed for a direct comparison of dropout rates and cognitive debriefing scores and accommodated the quantitative comparison of respondents' choice consistency using a heteroskedastic mixed logit model. Our results indicate that the introduction of level overlap made it significantly easier for respondents to identify the differences and choose between the choice options. As a stand‐alone design strategy, attribute level overlap reduced the dropout rate by 30%, increased the level of choice consistency by 30%, and avoided learning effects in the initial choice tasks of the DCE. The combination of level overlap and color coding was even more effective: It reduced the dropout rate by 40% to 50% and increased the level of choice consistency by more than 60%. Hence, we can recommend attribute level overlap, with color coding to amplify its impact, as a standard design strategy in DCEs.

Suggested Citation

  • Marcel F. Jonker & Bas Donkers & Esther de Bekker‐Grob & Elly A. Stolk, 2019. "Attribute level overlap (and color coding) can reduce task complexity, improve choice consistency, and decrease the dropout rate in discrete choice experiments," Health Economics, John Wiley & Sons, Ltd., vol. 28(3), pages 350-363, March.
  • Handle: RePEc:wly:hlthec:v:28:y:2019:i:3:p:350-363
    DOI: 10.1002/hec.3846
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/hec.3846
    Download Restriction: no

    File URL: https://libkey.io/10.1002/hec.3846?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Bliemer, Michiel C.J. & Rose, John M., 2010. "Construction of experimental designs for mixed logit models allowing for correlation across choice observations," Transportation Research Part B: Methodological, Elsevier, vol. 44(6), pages 720-734, July.
    2. David Hensher, 2006. "Revealing Differences in Willingness to Pay due to the Dimensionality of Stated Choice Designs: An Initial Assessment," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 34(1), pages 7-44, May.
    3. Marcel F. Jonker & Arthur E. Attema & Bas Donkers & Elly A. Stolk & Matthijs M. Versteegh, 2017. "Are Health State Valuations from the General Public Biased? A Test of Health State Reference Dependency Using Self‐assessed Health and an Efficient Discrete Choice Experiment," Health Economics, John Wiley & Sons, Ltd., vol. 26(12), pages 1534-1547, December.
    4. Bosworth, Ryan & Cameron, Trudy Ann & DeShazo, J.R., 2015. "Willingness to pay for public health policies to treat illnesses," Journal of Health Economics, Elsevier, vol. 39(C), pages 74-88.
    5. DeShazo, J. R. & Fermo, German, 2002. "Designing Choice Sets for Stated Preference Methods: The Effects of Complexity on Choice Consistency," Journal of Environmental Economics and Management, Elsevier, vol. 44(1), pages 123-143, July.
    6. Swait, Joffre & Adamowicz, Wiktor, 2001. "The Influence of Task Complexity on Consumer Choice: A Latent Class Model of Decision Strategy Switching," Journal of Consumer Research, Oxford University Press, vol. 28(1), pages 135-148, June.
    7. Nicolas Krucien & Amiram Gafni & Nathalie Pelletier‐Fleury, 2015. "Empirical Testing of the External Validity of a Discrete Choice Experiment to Determine Preferred Treatment Option: The Case of Sleep Apnea," Health Economics, John Wiley & Sons, Ltd., vol. 24(8), pages 951-965, August.
    8. Fredrik Carlsson & Peter Martinsson, 2008. "How Much is Too Much?," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 40(2), pages 165-176, June.
    9. Emmanouil Mentzakis & Mandy Ryan & Paul McNamee, 2011. "Using discrete choice experiments to value informal care tasks: exploring preference heterogeneity," Health Economics, John Wiley & Sons, Ltd., vol. 20(8), pages 930-944, August.
    10. Caussade, Sebastián & Ortúzar, Juan de Dios & Rizzi, Luis I. & Hensher, David A., 2005. "Assessing the influence of design dimensions on stated choice experiment estimates," Transportation Research Part B: Methodological, Elsevier, vol. 39(7), pages 621-640, August.
    11. Brendan Mulhern & Richard Norman & Paula Lorgelly & Emily Lancsar & Julie Ratcliffe & John Brazier & Rosalie Viney, 2017. "Is Dimension Order Important when Valuing Health States Using Discrete Choice Experiments Including Duration?," PharmacoEconomics, Springer, vol. 35(4), pages 439-451, April.
    12. Verity Watson & Frauke Becker & Esther de Bekker‐Grob, 2017. "Discrete Choice Experiment Response Rates: A Meta‐analysis," Health Economics, John Wiley & Sons, Ltd., vol. 26(6), pages 810-817, June.
    13. Richard Yao & Riccardo Scarpa & John Rose & James Turner, 2015. "Experimental Design Criteria and Their Behavioural Efficiency: An Evaluation in the Field," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 62(3), pages 433-455, November.
    14. KESSELS, Roselinde & BRADLEY, Jones & GOOS, Peter, 2012. "A comparison of partial profile designs for discrete choice experiments with an application in software development," Working Papers 2012004, University of Antwerp, Faculty of Business and Economics.
    15. Rolfe, John & Bennett, Jeff, 2009. "The impact of offering two versus three alternatives in choice modelling experiments," Ecological Economics, Elsevier, vol. 68(4), pages 1140-1148, February.
    16. Mickael Bech & Trine Kjaer & Jørgen Lauridsen, 2011. "Does the number of choice sets matter? Results from a web survey applying a discrete choice experiment," Health Economics, John Wiley & Sons, Ltd., vol. 20(3), pages 273-286, March.
    17. Rosalie Viney & Richard Norman & John Brazier & Paula Cronin & Madeleine T. King & Julie Ratcliffe & Deborah Street, 2014. "An Australian Discrete Choice Experiment To Value Eq‐5d Health States," Health Economics, John Wiley & Sons, Ltd., vol. 23(6), pages 729-742, June.
    18. Trine Kjær & Mickael Bech & Dorte Gyrd‐Hansen & Kristian Hart‐Hansen, 2006. "Ordering effect and price sensitivity in discrete choice experiments: need we worry?," Health Economics, John Wiley & Sons, Ltd., vol. 15(11), pages 1217-1228, November.
    19. Tara Maddala & Kathryn A. Phillips & F. Reed Johnson, 2003. "An experiment on simplifying conjoint analysis designs for measuring preferences," Health Economics, John Wiley & Sons, Ltd., vol. 12(12), pages 1035-1047, December.
    20. Jordan J. Louviere & Towhidul Islam & Nada Wasi & Deborah Street & Leonie Burgess, 2008. "Designing Discrete Choice Experiments: Do Optimal Designs Come at a Price?," Journal of Consumer Research, Oxford University Press, vol. 35(2), pages 360-375, March.
    21. Axel Mühlbacher & Susanne Bethge, 2015. "Reduce Mortality Risk Above All Else: A Discrete-Choice Experiment in Acute Coronary Syndrome Patients," PharmacoEconomics, Springer, vol. 33(1), pages 71-81, January.
    22. Rosalie Viney & Elizabeth Savage & Jordan Louviere, 2005. "Empirical investigation of experimental design properties of discrete choice experiments in health care," Health Economics, John Wiley & Sons, Ltd., vol. 14(4), pages 349-362, April.
    23. Terry Flynn & Marcel Bilger & Chetna Malhotra & Eric Finkelstein, 2016. "Are Efficient Designs Used in Discrete Choice Experiments Too Difficult for Some Respondents? A Case Study Eliciting Preferences for End-of-Life Care," PharmacoEconomics, Springer, vol. 34(3), pages 273-284, March.
    24. Terry N. Flynn & Marcel Bilger & Chetna Malhotra & Eric A. Finkelstein, 2016. "Are Efficient Designs Used in Discrete Choice Experiments Too Difficult for Some Respondents? A Case Study Eliciting Preferences for End-of-Life Care," PharmacoEconomics, Springer, vol. 34(3), pages 273-284, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lucas M. A. Goossens & Marcel F. Jonker & Maureen P. M. H. Rutten-van Mölken & Melinde R. S. Boland & Annerika H. M. Slok & Philippe L. Salomé & Onno C. P. van Schayck & Johannes C. C. M. in ‘t Ve, 2019. "The Fold-in, Fold-out Design for DCE Choice Tasks: Application to Burden of Disease," Medical Decision Making, , vol. 39(4), pages 450-460, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yang, Jui-Chen & Johnson, F. Reed & Kilambi, Vikram & Mohamed, Ateesha F., 2015. "Sample size and utility-difference precision in discrete-choice experiments: A meta-simulation approach," Journal of choice modelling, Elsevier, vol. 16(C), pages 50-57.
    2. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    3. Elizabeth Kinter & Thomas Prior & Christopher Carswell & John Bridges, 2012. "A Comparison of Two Experimental Design Approaches in Applying Conjoint Analysis in Patient-Centered Outcomes Research," The Patient: Patient-Centered Outcomes Research, Springer;International Academy of Health Preference Research, vol. 5(4), pages 279-294, December.
    4. Jürgen Meyerhoff & Malte Oehlmann & Priska Weller, 2015. "The Influence of Design Dimensions on Stated Choices in an Environmental Context," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 61(3), pages 385-407, July.
    5. Mandy Ryan & Nicolas Krucien & Frouke Hermens, 2018. "The eyes have it: Using eye tracking to inform information processing strategies in multi‐attributes choices," Health Economics, John Wiley & Sons, Ltd., vol. 27(4), pages 709-721, April.
    6. Mickael Bech & Trine Kjaer & Jørgen Lauridsen, 2011. "Does the number of choice sets matter? Results from a web survey applying a discrete choice experiment," Health Economics, John Wiley & Sons, Ltd., vol. 20(3), pages 273-286, March.
    7. Weng, Weizhe & Morrison, Mark D. & Boyle, Kevin J. & Boxall, Peter C. & Rose, John, 2021. "Effects of the number of alternatives in public good discrete choice experiments," Ecological Economics, Elsevier, vol. 182(C).
    8. Windle, Jill & Rolfe, John, 2010. "Restricted versus unrestricted choice in labelled choice experiments: exploring the tradeoffs of expanding choice dimensions," Research Reports 95072, Australian National University, Environmental Economics Research Hub.
    9. Sever, Ivan & Verbič, Miroslav & Klarić Sever, Eva, 2019. "Cost attribute in health care DCEs: Just adding another attribute or a trigger of change in the stated preferences?," Journal of choice modelling, Elsevier, vol. 32(C), pages 1-1.
    10. Ali Ardeshiri & Spring Sampson & Joffre Swait, 2019. "Seasonality Effects on Consumers Preferences Over Quality Attributes of Different Beef Products," Papers 1902.02419, arXiv.org.
    11. Mamine, Fateh & Fares, M'hand & Minviel, Jean Joseph, 2020. "Contract Design for Adoption of Agrienvironmental Practices: A Meta-analysis of Discrete Choice Experiments," Ecological Economics, Elsevier, vol. 176(C).
    12. Fraser, Iain & Balcombe, Kelvin & Williams, Louis & McSorley, Eugene, 2021. "Preference stability in discrete choice experiments. Some evidence using eye-tracking," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 94(C).
    13. Windle, Jill & Rolfe, John, 2014. "Valuation framing and attribute scope variation in a choice experiment to asses the impacts of changing land use from agriculture to mining," 2014 Conference (58th), February 4-7, 2014, Port Macquarie, Australia 165888, Australian Agricultural and Resource Economics Society.
    14. Alessandro Mengoni & Chiara Seghieri & Sabina Nuti, 2013. "The application of discrete choice experiments in health economics: a systematic review of the literature," Working Papers 201301, Scuola Superiore Sant'Anna of Pisa, Istituto di Management.
    15. Tobias Börger & Oliver Frör & Sören Weiß, 2017. "The relationship between perceived difficulty and randomness in discrete choice experiments: Investigating reasons for and consequences of difficulty," Discussion Papers in Environment and Development Economics 2017-03, University of St. Andrews, School of Geography and Sustainable Development.
    16. Thijs Dekker & Paul Koster & Roy Brouwer, 2014. "Changing with the Tide: Semiparametric Estimation of Preference Dynamics," Land Economics, University of Wisconsin Press, vol. 90(4), pages 717-745.
    17. Kosenius, Anna-Kaisa, 2013. "Preference discontinuity in choice experiment: Determinants and implications," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 45(C), pages 138-145.
    18. Rolfe, John & Windle, Jill, 2012. "Testing benefit transfer of reef protection values between local case studies: The Great Barrier Reef in Australia," Ecological Economics, Elsevier, vol. 81(C), pages 60-69.
    19. Weng, Weizhe & Morrison, Mark & Boyle, Kevin J. & Boxall, Peter C., 2017. "The effect of the number of alternatives in choice experiment questions," 2017 Annual Meeting, July 30-August 1, Chicago, Illinois 259179, Agricultural and Applied Economics Association.
    20. Oehlmann, Malte & Meyerhoff, Jürgen & Mariel, Petr & Weller, Priska, 2017. "Uncovering context-induced status quo effects in choice experiments," Journal of Environmental Economics and Management, Elsevier, vol. 81(C), pages 59-73.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:hlthec:v:28:y:2019:i:3:p:350-363. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/5749 .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/5749 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.