IDEAS home Printed from https://ideas.repec.org/a/spr/aphecp/v22y2024i3d10.1007_s40258-023-00865-x.html
   My bibliography  Save this article

Are Web-Based Valuation Surveys for Preference-Based Measures as Reliable as Face-to-Face Surveys? TTO, DCE and DCE with Duration

Author

Listed:
  • Takeru Shiroiwa

    (National Institute of Public Health)

  • Takashi Fukuda

    (National Institute of Public Health)

Abstract

Background Valuation surveys of preference-based measures are typically conducted face-to-face or on web panels. In this survey, we considered whether face-to-face and online surveys were reliable using three tasks: composite time trade-off (cTTO), discrete choice experiment (DCE), and DCE with duration. Methods Respondents (aged 20–69 years) for both face-to-face (N = 1000, target sample size) and web surveys were selected through quota sampling by sex and age from each panel of the general population in Japan. They were then allocated to one of the three tasks and divided into six groups (two survey modes × three tasks, N = 334 per group). For the cTTO, respondents were asked to rate ten health states described by the EQ-5D-5L. For the DCE and DCE with duration surveys, respondents were asked about 15 health–state pairs. For all participants, as in the second-stage survey, a similar process was repeated two weeks after the first survey. Reliability was evaluated by calculating the percentage of agreement and intraclass correlation coefficients. Results The cTTO scores of the face-to-face and web surveys were systematically different. Between the face-to-face and web surveys, the agreement of the TTO survey was not good. The intraclass correlation coefficient (ICC) was 0.37 for the face-to-face test-test and 0.59 for the web test-retest. Discrete choice experiment (DCE) and DCE with duration had similarly good agreement (more than 70%), regardless of face-to-face or web surveys. However, between the first and second surveys (test-retest) of DCE and DCE with duration, the agreement depends on whether the positions of the two cards (health states) are identical. Conclusion If the face-to-face cTTO score is the gold standard, a web-based survey of cTTO is not recommended regardless of the ICC. If a DCE survey is performed, positioning effects should be considered.

Suggested Citation

  • Takeru Shiroiwa & Takashi Fukuda, 2024. "Are Web-Based Valuation Surveys for Preference-Based Measures as Reliable as Face-to-Face Surveys? TTO, DCE and DCE with Duration," Applied Health Economics and Health Policy, Springer, vol. 22(3), pages 391-400, May.
    • Handle: RePEc:spr:aphecp:v:22:y:2024:i:3:d:10.1007_s40258-023-00865-x
    DOI: 10.1007/s40258-023-00865-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s40258-023-00865-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s40258-023-00865-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Nancy Devlin & Ken Buckingham & Koonal Shah & Aki Tsuchiya & Carl Tilling & Grahame Wilkinson & Ben van Hout, 2013. "A Comparison Of Alternative Variants Of The Lead And Lag Time Tto," Health Economics, John Wiley & Sons, Ltd., vol. 22(5), pages 517-532, May.
    2. Suzanne Robinson, 2011. "Test–retest reliability of health state valuation techniques: the time trade off and person trade off," Health Economics, John Wiley & Sons, Ltd., vol. 20(11), pages 1379-1391, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Juan Ramos-Goñi & Oliver Rivero-Arias & María Errea & Elly Stolk & Michael Herdman & Juan Cabasés, 2013. "Dealing with the health state ‘dead’ when using discrete choice experiments to obtain values for EQ-5D-5L heath states," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 33-42, July.
    2. Jen-Yu Amy Chang & Chien-Ning Hsu & Juan Manuel Ramos-Goñi & Nan Luo & Hsiang-Wen Lin & Fang-Ju Lin, 2024. "Beyond 10-year lead-times in EQ-5D-5L: leveraging alternative lead-times in willingness-to-accept questions to capture preferences for worse-than-dead states and their implication," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 25(6), pages 1041-1055, August.
    3. Nan Luo & Minghui Li & Elly Stolk & Nancy Devlin, 2013. "The effects of lead time and visual aids in TTO valuation: a study of the EQ-VT framework," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 15-24, July.
    4. Stefan A. Lipman & Liying Zhang & Koonal K. Shah & Arthur E. Attema, 2023. "Time and lexicographic preferences in the valuation of EQ-5D-Y with time trade-off methodology," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 24(2), pages 293-305, March.
    5. Eva Rodríguez-Míguez & Antonio Sampayo, 2025. "Comparison of Caregiver and General Population Preferences for Dependency-Related Health States," Applied Health Economics and Health Policy, Springer, vol. 23(1), pages 105-117, January.
    6. Bansback, Nick & Hole, Arne Risa & Mulhern, Brendan & Tsuchiya, Aki, 2014. "Testing a discrete choice experiment including duration to value health states for large descriptive systems: Addressing design and sampling issues," Social Science & Medicine, Elsevier, vol. 114(C), pages 38-48.
    7. Arthur Attema & Yvette Edelaar-Peeters & Matthijs Versteegh & Elly Stolk, 2013. "Time trade-off: one methodology, different methods," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 53-64, July.
    8. José Luis Pinto-Prades & Eva Rodríguez-Míguez, 2015. "The Lead Time Tradeoff," Medical Decision Making, , vol. 35(3), pages 305-315, April.
    9. Fernando Lera-Lopez & Andrea Ollo-López & Mirian Garrués-Irisarri & Juan M. Cabasés & Eduardo Sánchez, 2019. "How the relationship between physical activity and health changes with age," European Journal of Ageing, Springer, vol. 16(1), pages 3-15, March.
    10. Nancy J. Devlin & Koonal K. Shah & Yan Feng & Brendan Mulhern & Ben van Hout, 2018. "Valuing health‐related quality of life: An EQ‐5D‐5L value set for England," Health Economics, John Wiley & Sons, Ltd., vol. 27(1), pages 7-22, January.
    11. Koonal Shah & Andrew Lloyd & Mark Oppe & Nancy Devlin, 2013. "One-to-one versus group setting for conducting computer-assisted TTO studies: findings from pilot studies in England and the Netherlands," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 65-73, July.
    12. Koonal K. Shah & Juan Manuel Ramos-Goñi & Simone Kreimeier & Nancy J. Devlin, 2020. "An exploration of methods for obtaining 0 = dead anchors for latent scale EQ-5D-Y values," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 21(7), pages 1091-1103, September.
    13. Versteegh, MM & Attema, AE & Oppe, M & Devlin, NJ & Stolk, EA, 2012. "Time to tweak the TTO. But how?," MPRA Paper 37989, University Library of Munich, Germany.
    14. Chris Sampson & David Parkin & Nancy Devlin, 2024. "Is anchoring at ‘dead’ a theoretical requirement for health state valuation?," Health Economics, John Wiley & Sons, Ltd., vol. 33(9), pages 1929-1935, September.
    15. P. Wang & M. Li & G. Liu & J. Thumboo & N. Luo, 2015. "Do Chinese have similar health-state preferences? A comparison of mainland Chinese and Singaporean Chinese," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 16(8), pages 857-863, November.
    16. Mark Oppe & Kim Rand-Hendriksen & Koonal Shah & Juan M. Ramos‐Goñi & Nan Luo, 2016. "EuroQol Protocols for Time Trade-Off Valuation of Health Outcomes," PharmacoEconomics, Springer, vol. 34(10), pages 993-1004, October.
    17. Yan Feng & Arne Risa Hole & Milad Karimi & Aki Tsuchiya & Ben van Hout, 2018. "An exploration of the non‐iterative time trade‐off method to value health states," Health Economics, John Wiley & Sons, Ltd., vol. 27(8), pages 1247-1263, August.
    18. Anja Schwalm & You-Shan Feng & Jörn Moock & Thomas Kohlmann, 2015. "Differences in EQ-5D-3L health state valuations among patients with musculoskeletal diseases, health care professionals and healthy volunteers," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 16(8), pages 865-877, November.
    19. Matthijs Versteegh & Arthur Attema & Mark Oppe & Nancy Devlin & Elly Stolk, 2013. "Time to tweak the TTO: results from a comparison of alternative specifications of the TTO," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 43-51, July.
    20. Finch, Aureliano Paolo & Meregaglia, Michela & Ciani, Oriana & Roudijk, Bram & Jommi, Claudio, 2022. "An EQ-5D-5L value set for Italy using videoconferencing interviews and feasibility of a new mode of administration," Social Science & Medicine, Elsevier, vol. 292(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:aphecp:v:22:y:2024:i:3:d:10.1007_s40258-023-00865-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.