IDEAS home Printed from https://ideas.repec.org/a/spr/eujhec/v22y2021i1d10.1007_s10198-020-01224-6.html
   My bibliography  Save this article

Increasing respondent engagement in composite time trade-off tasks by imposing three minimum trade-offs to improve data quality

Author

Listed:
  • Ruixuan Jiang

    (Merck & Co.)

  • Thomas Kohlmann

    (Medical University Greifswald)

  • Todd A. Lee

    (University of Illinois at Chicago College of Pharmacy)

  • Axel Mühlbacher

    (Hochschule Neubrandenburg)

  • James Shaw

    (Bristol-Myers Squibb)

  • Surrey Walton

    (University of Illinois at Chicago College of Pharmacy)

  • A. Simon Pickard

    (University of Illinois at Chicago College of Pharmacy)

Abstract

Background Web-based surveys are increasingly utilized for health valuation studies but may be more prone to lack of engagement and, therefore, poor data validity. The objective of this study was to evaluate the effect of imposed engagement (i.e., at least three trade-offs) in the composite time trade-off (cTTO) task. Methods The EQ-5D-5L valuation study protocol and study design were adapted for online, unsupervised completion in two arms: base case and engagement. Validity of preferences was assessed using the prevalence of inconsistent valuations and expected patterns of TTO values. Respondent task engagement was measured using time per task. Value sets were generated using linear regression with a random intercept (RILR). Results The base case (n = 501) and engagement arms (n = 504) clustered at different TTO values: [base case] 0, 1; [engagement] -0.5, 0.45, 0.6. Mean TTO values were lower for the engagement arm. Engagement respondents did not spend more time per TTO task: [base case] 63.3 s (SD 77.9 s); [engagement] 64.7 s (SD 73.3 s); p = 0.36. No significant difference was found between arms for prevalence of respondents with at least one inconsistent TTO value: [base case] 61.1%; [engagement] 63.5%; p = 0.43. Both value sets had significant intercepts far from 1: [base case] 0.846; [engagement] 0.783. The relative importance of the EQ-5D dimensions also differed between arms. Conclusions Both online arms had poor quality data. A minimum trade-off threshold did not improve engagement nor face validity of the data, indicating that modifications to the number of iterations are insufficient alone to improve data quality/validity of online TTO studies.

Suggested Citation

  • Ruixuan Jiang & Thomas Kohlmann & Todd A. Lee & Axel Mühlbacher & James Shaw & Surrey Walton & A. Simon Pickard, 2021. "Increasing respondent engagement in composite time trade-off tasks by imposing three minimum trade-offs to improve data quality," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 22(1), pages 17-33, February.
    • Handle: RePEc:spr:eujhec:v:22:y:2021:i:1:d:10.1007_s10198-020-01224-6
    DOI: 10.1007/s10198-020-01224-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10198-020-01224-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10198-020-01224-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Nancy Devlin & Ken Buckingham & Koonal Shah & Aki Tsuchiya & Carl Tilling & Grahame Wilkinson & Ben van Hout, 2013. "A Comparison Of Alternative Variants Of The Lead And Lag Time Tto," Health Economics, John Wiley & Sons, Ltd., vol. 22(5), pages 517-532, May.
    2. Leslie A. Lenert & Daniel J. Cher & Mary K. Goldstein & Merlynn R. Bergen & Alan Garber, 1998. "The Effect of Search Procedures on Utility Elicitations," Medical Decision Making, , vol. 18(1), pages 76-83, January.
    3. Leonard J. Paas & Meike Morren, 2018. "PLease do not answer if you are reading this: respondent attention in online panels," Marketing Letters, Springer, vol. 29(1), pages 13-21, March.
    4. Yvette Edelaar-Peeters & Anne M. Stiggelbout & Wilbert B. Van Den Hout, 2014. "Qualitative and Quantitative Analysis of Interviewer Help Answering the Time Tradeoff," Medical Decision Making, , vol. 34(5), pages 655-665, July.
    5. Leslie A. Lenert & Ann Sturley & Marcia Rupnow, 2003. "Toward Improved Methods for Measurement of Utility: Automated Repair of Errors in Elicitations," Medical Decision Making, , vol. 23(1), pages 67-75, January.
    6. Nan Luo & Minghui Li & Elly Stolk & Nancy Devlin, 2013. "The effects of lead time and visual aids in TTO valuation: a study of the EQ-VT framework," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 15-24, July.
    7. Matthijs Versteegh & Arthur Attema & Mark Oppe & Nancy Devlin & Elly Stolk, 2013. "Time to tweak the TTO: results from a comparison of alternative specifications of the TTO," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 43-51, July.
    8. Mark Oppe & Kim Rand-Hendriksen & Koonal Shah & Juan M. Ramos‐Goñi & Nan Luo, 2016. "EuroQol Protocols for Time Trade-Off Valuation of Health Outcomes," PharmacoEconomics, Springer, vol. 34(10), pages 993-1004, October.
    9. Nick Bansback & Aki Tsuchiya & John Brazier & Aslam Anis, 2012. "Canadian Valuation of EQ-5D Health States: Preliminary Value Set and Considerations for Future Valuation Studies," PLOS ONE, Public Library of Science, vol. 7(2), pages 1-11, February.
    10. Feng Xie & Eleanor Pullenayegum & A. Simon Pickard & Juan Manuel Ramos Goñi & Min‐woo Jo & Ataru Igarashi, 2017. "Transforming Latent Utilities to Health Utilities: East Does Not Meet West," Health Economics, John Wiley & Sons, Ltd., vol. 26(12), pages 1524-1533, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. Rita Faria’s journal round-up for 8th February 2021
      by Rita Faria in The Academic Health Economists' Blog on 2021-02-08 12:00:01

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Arthur Attema & Yvette Edelaar-Peeters & Matthijs Versteegh & Elly Stolk, 2013. "Time trade-off: one methodology, different methods," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 53-64, July.
    2. Stefan A. Lipman & Liying Zhang & Koonal K. Shah & Arthur E. Attema, 2023. "Time and lexicographic preferences in the valuation of EQ-5D-Y with time trade-off methodology," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 24(2), pages 293-305, March.
    3. Koonal K. Shah & Juan Manuel Ramos-Goñi & Simone Kreimeier & Nancy J. Devlin, 2020. "An exploration of methods for obtaining 0 = dead anchors for latent scale EQ-5D-Y values," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 21(7), pages 1091-1103, September.
    4. Finch, Aureliano Paolo & Meregaglia, Michela & Ciani, Oriana & Roudijk, Bram & Jommi, Claudio, 2022. "An EQ-5D-5L value set for Italy using videoconferencing interviews and feasibility of a new mode of administration," Social Science & Medicine, Elsevier, vol. 292(C).
    5. Versteegh, MM & Attema, AE & Oppe, M & Devlin, NJ & Stolk, EA, 2012. "Time to tweak the TTO. But how?," MPRA Paper 37989, University Library of Munich, Germany.
    6. P. Wang & M. Li & G. Liu & J. Thumboo & N. Luo, 2015. "Do Chinese have similar health-state preferences? A comparison of mainland Chinese and Singaporean Chinese," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 16(8), pages 857-863, November.
    7. Matthijs Versteegh & Arthur Attema & Mark Oppe & Nancy Devlin & Elly Stolk, 2013. "Time to tweak the TTO: results from a comparison of alternative specifications of the TTO," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 43-51, July.
    8. Hobbins, Anna P. & Barry, Luke & Kelleher, Dan & Shah, Koonal & Devlin, Nancy & Ramos Goni, Juan Manuel & O’Neill, Ciaran, 2020. "Do people with private health insurance attach a higher value to health than those without insurance? Results from an EQ-5D-5 L valuation study in Ireland," Health Policy, Elsevier, vol. 124(6), pages 639-646.
    9. Baier-Fuentes, Hugo & Andrade-Valbuena, Nelson A. & Huertas Gonzalez-Serrano, Maria & Gaviria-Marin, Magaly, 2023. "Bricolage as an effective tool for the survival of owner-managed SMEs during crises," Journal of Business Research, Elsevier, vol. 157(C).
    10. Juan Ramos-Goñi & Oliver Rivero-Arias & María Errea & Elly Stolk & Michael Herdman & Juan Cabasés, 2013. "Dealing with the health state ‘dead’ when using discrete choice experiments to obtain values for EQ-5D-5L heath states," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 33-42, July.
    11. Han Bleichrodt, 2002. "A new explanation for the difference between time trade‐off utilities and standard gamble utilities," Health Economics, John Wiley & Sons, Ltd., vol. 11(5), pages 447-456, July.
    12. Nan Luo & Minghui Li & Elly Stolk & Nancy Devlin, 2013. "The effects of lead time and visual aids in TTO valuation: a study of the EQ-VT framework," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 15-24, July.
    13. Markian Pahuta & Aaron Frombach & Emile Hashem & Stewart Spence & Christina Sun & Eugene K. Wai & Joel Werier & Carl Walraven & Doug Coyle, 2019. "The Psychometric Properties of a Self-Administered, Open-Source Module for Valuing Metastatic Epidural Spinal Cord Compression Utilities," PharmacoEconomics - Open, Springer, vol. 3(2), pages 197-204, June.
    14. Agata Łaszewska & Ayesha Sajjad & Jan Busschbach & Judit Simon & Leona Hakkaart-van Roijen, 2022. "Conceptual Framework for Optimised Proxy Value Set Selection Through Supra-National Value Set Development for the EQ-5D Instruments," PharmacoEconomics, Springer, vol. 40(12), pages 1221-1234, December.
    15. Jose-Luis Pinto-Prades & Jorge-Eduardo Martinez-Perez & Jose-Maria Abellan-Perpinan, 2006. "The influence of the ratio bias phenomenon on the elicitation of health states utilities," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 1, pages 118-133, November.
    16. Munir A. Khan & Jeff Richardson, 2019. "Is the Validity of Cost Utility Analysis Improved When Utility is Measured by an Instrument with ‘Home-Country’ Weights? Evidence from Six Western Countries," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 145(1), pages 1-15, August.
    17. Asrul Akmal Shafie & Annushiah Vasan Thakumar, 2020. "Multiplicative modelling of EQ-5D-3L TTO and VAS values," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 21(9), pages 1411-1420, December.
    18. Anne‐Laure Samson & Erik Schokkaert & Clémence Thébaut & Brigitte Dormont & Marc Fleurbaey & Stéphane Luchini & Carine Van de Voorde, 2018. "Fairness in cost‐benefit analysis: A methodology for health technology assessment," Health Economics, John Wiley & Sons, Ltd., vol. 27(1), pages 102-114, January.
    19. Mathieu F. Janssen & Gouke J. Bonsel & Nan Luo, 2018. "Is EQ-5D-5L Better Than EQ-5D-3L? A Head-to-Head Comparison of Descriptive Systems and Value Sets from Seven Countries," PharmacoEconomics, Springer, vol. 36(6), pages 675-697, June.
    20. repec:cup:judgdm:v:1:y:2006:i::p:118-133 is not listed on IDEAS
    21. Mihir Gandhi & Marcus Ang & Kelvin Teo & Chee Wai Wong & Yvonne Chung-Hsi Wei & Rachel Lee-Yin Tan & Mathieu F. Janssen & Nan Luo, 2020. "A vision ‘bolt-on’ increases the responsiveness of EQ-5D: preliminary evidence from a study of cataract surgery," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 21(4), pages 501-511, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:eujhec:v:22:y:2021:i:1:d:10.1007_s10198-020-01224-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.