IDEAS home Printed from https://ideas.repec.org/a/spr/eujhec/v22y2021i1d10.1007_s10198-020-01224-6.html
   My bibliography  Save this article

Increasing respondent engagement in composite time trade-off tasks by imposing three minimum trade-offs to improve data quality

Author

Listed:
  • Ruixuan Jiang

    (Merck & Co.)

  • Thomas Kohlmann

    (Medical University Greifswald)

  • Todd A. Lee

    (University of Illinois at Chicago College of Pharmacy)

  • Axel Mühlbacher

    (Hochschule Neubrandenburg)

  • James Shaw

    (Bristol-Myers Squibb)

  • Surrey Walton

    (University of Illinois at Chicago College of Pharmacy)

  • A. Simon Pickard

    (University of Illinois at Chicago College of Pharmacy)

Abstract

Background Web-based surveys are increasingly utilized for health valuation studies but may be more prone to lack of engagement and, therefore, poor data validity. The objective of this study was to evaluate the effect of imposed engagement (i.e., at least three trade-offs) in the composite time trade-off (cTTO) task. Methods The EQ-5D-5L valuation study protocol and study design were adapted for online, unsupervised completion in two arms: base case and engagement. Validity of preferences was assessed using the prevalence of inconsistent valuations and expected patterns of TTO values. Respondent task engagement was measured using time per task. Value sets were generated using linear regression with a random intercept (RILR). Results The base case (n = 501) and engagement arms (n = 504) clustered at different TTO values: [base case] 0, 1; [engagement] -0.5, 0.45, 0.6. Mean TTO values were lower for the engagement arm. Engagement respondents did not spend more time per TTO task: [base case] 63.3 s (SD 77.9 s); [engagement] 64.7 s (SD 73.3 s); p = 0.36. No significant difference was found between arms for prevalence of respondents with at least one inconsistent TTO value: [base case] 61.1%; [engagement] 63.5%; p = 0.43. Both value sets had significant intercepts far from 1: [base case] 0.846; [engagement] 0.783. The relative importance of the EQ-5D dimensions also differed between arms. Conclusions Both online arms had poor quality data. A minimum trade-off threshold did not improve engagement nor face validity of the data, indicating that modifications to the number of iterations are insufficient alone to improve data quality/validity of online TTO studies.

Suggested Citation

  • Ruixuan Jiang & Thomas Kohlmann & Todd A. Lee & Axel Mühlbacher & James Shaw & Surrey Walton & A. Simon Pickard, 2021. "Increasing respondent engagement in composite time trade-off tasks by imposing three minimum trade-offs to improve data quality," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 22(1), pages 17-33, February.
  • Handle: RePEc:spr:eujhec:v:22:y:2021:i:1:d:10.1007_s10198-020-01224-6
    DOI: 10.1007/s10198-020-01224-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10198-020-01224-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10198-020-01224-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Nancy Devlin & Ken Buckingham & Koonal Shah & Aki Tsuchiya & Carl Tilling & Grahame Wilkinson & Ben van Hout, 2013. "A Comparison Of Alternative Variants Of The Lead And Lag Time Tto," Health Economics, John Wiley & Sons, Ltd., vol. 22(5), pages 517-532, May.
    2. Nan Luo & Minghui Li & Elly Stolk & Nancy Devlin, 2013. "The effects of lead time and visual aids in TTO valuation: a study of the EQ-VT framework," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 15-24, July.
    3. Mark Oppe & Kim Rand-Hendriksen & Koonal Shah & Juan M. Ramos‐Goñi & Nan Luo, 2016. "EuroQol Protocols for Time Trade-Off Valuation of Health Outcomes," PharmacoEconomics, Springer, vol. 34(10), pages 993-1004, October.
    4. Leonard J. Paas & Meike Morren, 2018. "PLease do not answer if you are reading this: respondent attention in online panels," Marketing Letters, Springer, vol. 29(1), pages 13-21, March.
    5. Feng Xie & Eleanor Pullenayegum & A. Simon Pickard & Juan Manuel Ramos Goñi & Min‐woo Jo & Ataru Igarashi, 2017. "Transforming Latent Utilities to Health Utilities: East Does Not Meet West," Health Economics, John Wiley & Sons, Ltd., vol. 26(12), pages 1524-1533, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. Rita Faria’s journal round-up for 8th February 2021
      by Rita Faria in The Academic Health Economists' Blog on 2021-02-08 12:00:01

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Devlin, N. & Shah, K.K & Buckingham, K., 2017. "What is the Normative Basis for Selecting the Measure of 'Average' Preferences for Use in Social Choices?," Research Papers 001798, Office of Health Economics.
    2. Koonal K. Shah & Juan Manuel Ramos-Goñi & Simone Kreimeier & Nancy J. Devlin, 2020. "An exploration of methods for obtaining 0 = dead anchors for latent scale EQ-5D-Y values," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 21(7), pages 1091-1103, September.
    3. P. Wang & M. Li & G. Liu & J. Thumboo & N. Luo, 2015. "Do Chinese have similar health-state preferences? A comparison of mainland Chinese and Singaporean Chinese," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 16(8), pages 857-863, November.
    4. Koonal K. Shah & Juan Manuel Ramos-Goñi & Simone Kreimeier & Nancy J. Devlin, 0. "An exploration of methods for obtaining 0 = dead anchors for latent scale EQ-5D-Y values," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 0, pages 1-13.
    5. Hobbins, Anna P. & Barry, Luke & Kelleher, Dan & Shah, Koonal & Devlin, Nancy & Ramos Goni, Juan Manuel & O’Neill, Ciaran, 2020. "Do people with private health insurance attach a higher value to health than those without insurance? Results from an EQ-5D-5 L valuation study in Ireland," Health Policy, Elsevier, vol. 124(6), pages 639-646.
    6. Juan Ramos-Goñi & Oliver Rivero-Arias & María Errea & Elly Stolk & Michael Herdman & Juan Cabasés, 2013. "Dealing with the health state ‘dead’ when using discrete choice experiments to obtain values for EQ-5D-5L heath states," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 33-42, July.
    7. Nan Luo & Minghui Li & Elly Stolk & Nancy Devlin, 2013. "The effects of lead time and visual aids in TTO valuation: a study of the EQ-VT framework," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 15-24, July.
    8. Yan Feng & Nancy J. Devlin & Koonal K. Shah & Brendan Mulhern & Ben van Hout, 2018. "New methods for modelling EQ‐5D‐5L value sets: An application to English data," Health Economics, John Wiley & Sons, Ltd., vol. 27(1), pages 23-38, January.
    9. Asrul Akmal Shafie & Annushiah Vasan Thakumar, 2020. "Multiplicative modelling of EQ-5D-3L TTO and VAS values," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 21(9), pages 1411-1420, December.
    10. Anne‐Laure Samson & Erik Schokkaert & Clémence Thébaut & Brigitte Dormont & Marc Fleurbaey & Stéphane Luchini & Carine Van de Voorde, 2018. "Fairness in cost‐benefit analysis: A methodology for health technology assessment," Health Economics, John Wiley & Sons, Ltd., vol. 27(1), pages 102-114, January.
    11. Mathieu F. Janssen & Gouke J. Bonsel & Nan Luo, 2018. "Is EQ-5D-5L Better Than EQ-5D-3L? A Head-to-Head Comparison of Descriptive Systems and Value Sets from Seven Countries," PharmacoEconomics, Springer, vol. 36(6), pages 675-697, June.
    12. Mihir Gandhi & Marcus Ang & Kelvin Teo & Chee Wai Wong & Yvonne Chung-Hsi Wei & Rachel Lee-Yin Tan & Mathieu F. Janssen & Nan Luo, 2020. "A vision ‘bolt-on’ increases the responsiveness of EQ-5D: preliminary evidence from a study of cataract surgery," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 21(4), pages 501-511, June.
    13. Bansback, Nick & Hole, Arne Risa & Mulhern, Brendan & Tsuchiya, Aki, 2014. "Testing a discrete choice experiment including duration to value health states for large descriptive systems: Addressing design and sampling issues," Social Science & Medicine, Elsevier, vol. 114(C), pages 38-48.
    14. Kristina Ludwig & J.-Matthias Graf von der Schulenburg & Wolfgang Greiner, 2018. "German Value Set for the EQ-5D-5L," PharmacoEconomics, Springer, vol. 36(6), pages 663-674, June.
    15. Arthur Attema & Yvette Edelaar-Peeters & Matthijs Versteegh & Elly Stolk, 2013. "Time trade-off: one methodology, different methods," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 14(1), pages 53-64, July.
    16. Nancy J. Devlin & Koonal K. Shah & Yan Feng & Brendan Mulhern & Ben van Hout, 2018. "Valuing health‐related quality of life: An EQ‐5D‐5L value set for England," Health Economics, John Wiley & Sons, Ltd., vol. 27(1), pages 7-22, January.
    17. José Luis Pinto-Prades & Eva Rodríguez-Míguez, 2015. "The Lead Time Tradeoff," Medical Decision Making, , vol. 35(3), pages 305-315, April.
    18. Fredrick Dermawan Purba & Joke A. M. Hunfeld & Aulia Iskandarsyah & Titi Sahidah Fitriana & Sawitri Supardi Sadarjoen & Juan Manuel Ramos-Goñi & Jan Passchier & Jan J. V. Busschbach, 2017. "The Indonesian EQ-5D-5L Value Set," PharmacoEconomics, Springer, vol. 35(11), pages 1153-1165, November.
    19. Fernando Lera-Lopez & Andrea Ollo-López & Mirian Garrués-Irisarri & Juan M. Cabasés & Eduardo Sánchez, 2019. "How the relationship between physical activity and health changes with age," European Journal of Ageing, Springer, vol. 16(1), pages 3-15, March.
    20. Ian Waudby-Smith & A. Simon Pickard & Feng Xie & Eleanor M. Pullenayegum, 2020. "Using Both Time Tradeoff and Discrete Choice Experiments in Valuing the EQ-5D: Impact of Model Misspecification on Value Sets," Medical Decision Making, , vol. 40(4), pages 483-497, May.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:eujhec:v:22:y:2021:i:1:d:10.1007_s10198-020-01224-6. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.springer.com .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.