IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v39y2019i7p827-841.html
   My bibliography  Save this article

Mode and Frame Matter: Assessing the Impact of Survey Mode and Sample Frame in Choice Experiments

Author

Listed:
  • Verity Watson

    (Health Economics Research Unit, University of Aberdeen, Foresterhill, Aberdeen, Scotland)

  • Terry Porteous

    (Health Economics Research Unit, University of Aberdeen, Foresterhill, Aberdeen, Scotland)

  • Tim Bolt

    (Faculty of Economics, Saitama University, Sakura-ku, Saitama, Japan)

  • Mandy Ryan

    (Health Economics Research Unit, University of Aberdeen, Foresterhill, Aberdeen, Scotland)

Abstract

Background. Choice experiments (CE) are applied in health economics to elicit public preferences and willingness to pay (WTP). CEs are frequently administered as Internet-based surveys. Internet surveys have recognized advantages, but concerns exist about the representativeness of Internet samples, data quality, and the impact on elicited values. Aim. We conducted the first study in health comparing an Internet-based CE survey with the more traditional general population mail survey. We also compared the Internet-based and mail CE surveys with computer-assisted personal interviews (CAPIs), which are commonly used to elicit health state valuations. Methods. Two separate samples were drawn from 2 United Kingdom (UK) volunteer Internet panels (IPs), CAPIs were undertaken with respondents sampled from UK Census Output Areas, and mail surveys were sent to UK households drawn from the postcode address file (PAF). Each mode received more than 1000 respondents. We compared modes and frames using objective measures (response rate, sample representativeness of the UK population, elicited values, theoretical validity, and cost per response) and subjective/self-reported measures (time taken to complete the study, perceived study consequentiality, and stated attribute nonattendance). This study intentionally confounded the survey modes and sample frame by choosing sample frames that are typically used by researchers for each mode. Results. Estimated WTP differs across mode-frame pairs. On most measures, CAPIs dominated. They are more expensive, however. On all measures, except response rates, Internet surveys dominated the mail survey. They were also cheaper. Conclusion. Researchers using IPs should pay attention to response rates and be aware that the quality of IPs differs. Given the importance of perceived consequentiality and attribute attendance in CEs, future research should address their impact across modes and frames.

Suggested Citation

  • Verity Watson & Terry Porteous & Tim Bolt & Mandy Ryan, 2019. "Mode and Frame Matter: Assessing the Impact of Survey Mode and Sample Frame in Choice Experiments," Medical Decision Making, , vol. 39(7), pages 827-841, October.
    • Handle: RePEc:sae:medema:v:39:y:2019:i:7:p:827-841
    DOI: 10.1177/0272989X19871035
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X19871035
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0272989X19871035?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Donna Rowen & John Brazier & Anju Keetharuth & Aki Tsuchiya & Clara Mukuria, 2016. "Comparison of Modes of Administration and Alternative Formats for Eliciting Societal Preferences for Burden of Illness," Applied Health Economics and Health Policy, Springer, vol. 14(1), pages 89-104, February.
    2. Vossler, Christian A. & Watson, Sharon B., 2013. "Understanding the consequences of consequentiality: Testing the validity of stated preferences in the field," Journal of Economic Behavior & Organization, Elsevier, vol. 86(C), pages 137-147.
    3. Arne Risa Hole, 2007. "A comparison of approaches to estimating confidence intervals for willingness to pay measures," Health Economics, John Wiley & Sons, Ltd., vol. 16(8), pages 827-840, August.
    4. Madeleine T. King & Jane Hall & Emily Lancsar & Denzil Fiebig & Ishrat Hossain & Jordan Louviere & Helen K. Reddel & Christine R. Jenkins, 2007. "Patient preferences for managing asthma: results from a discrete choice experiment," Health Economics, John Wiley & Sons, Ltd., vol. 16(7), pages 703-717, July.
    5. Richard Carson & Theodore Groves, 2007. "Incentive and informational properties of preference questions," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 37(1), pages 181-210, May.
    6. Verity Watson & Frauke Becker & Esther de Bekker‐Grob, 2017. "Discrete Choice Experiment Response Rates: A Meta‐analysis," Health Economics, John Wiley & Sons, Ltd., vol. 26(6), pages 810-817, June.
    7. Michael Keane & Nada Wasi, 2013. "Comparing Alternative Models Of Heterogeneity In Consumer Choice Behavior," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(6), pages 1018-1045, September.
    8. J. D. Snowball & K. G. Willis, 2011. "Interview versus self-completion questionnaires in discrete choice experiments," Applied Economics Letters, Taylor & Francis Journals, vol. 18(16), pages 1521-1525.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ishant Sharma & Sabyasachee Mishra, 2023. "Ranking preferences towards adopting autonomous vehicles based on peer inputs and advertisements," Transportation, Springer, vol. 50(6), pages 2139-2192, December.
    2. Zawojska, Ewa & Gastineau, Pascal & Mahieu, Pierre-Alexandre & Cheze, Benoit & Paris, Anthony, 2021. "Measuring policy consequentiality perceptions in stated preference surveys," 2021 Annual Meeting, August 1-3, Austin, Texas 313977, Agricultural and Applied Economics Association.
    3. MuLong Mao & Paul James, 2020. "Evaluating Customer Satisfaction Using the ACSI Model in a Themed Restaurant in Bangkok," Business Management and Strategy, Macrothink Institute, vol. 11(1), pages 55-71, June.
    4. Mandy Ryan & Emmanouil Mentzakis & Catriona Matheson & Christine Bond, 2020. "Survey modes comparison in contingent valuation: Internet panels and mail surveys," Health Economics, John Wiley & Sons, Ltd., vol. 29(2), pages 234-242, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gordillo, Fernando & Elsasser, Peter & Günter, Sven, 2019. "Willingness to pay for forest conservation in Ecuador: Results from a nationwide contingent valuation survey in a combined “referendum” – “Consequential open-ended” design," Forest Policy and Economics, Elsevier, vol. 105(C), pages 28-39.
    2. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    3. Frondel Manuel & Kutzschbauch Ole & Sommer Stephan & Traub Stefan, 2017. "Die Gerechtigkeitslücke in der Verteilung der Kosten der Energiewende auf die privaten Haushalte," Perspektiven der Wirtschaftspolitik, De Gruyter, vol. 18(4), pages 335-347, November.
    4. Sardaro, Ruggiero & La Sala, Piermichele & De Pascale, Gianluigi & Faccilongo, Nicola, 2021. "The conservation of cultural heritage in rural areas: Stakeholder preferences regarding historical rural buildings in Apulia, southern Italy," Land Use Policy, Elsevier, vol. 109(C).
    5. Thapa, Samir & Morrison, Mark & Parton, Kevin A, 2021. "Willingness to pay for domestic biogas plants and distributing carbon revenues to influence their purchase: A case study in Nepal," Energy Policy, Elsevier, vol. 158(C).
    6. Pengfei Liu & Lingling Hou & Dongqing Li & Shi Min & Yueying Mu, 2021. "Determinants of Livestock Insurance Demand: Experimental Evidence from Chinese Herders," Journal of Agricultural Economics, Wiley Blackwell, vol. 72(2), pages 430-451, June.
    7. Andor, Mark A. & Lange, Andreas & Sommer, Stephan, 2022. "Fairness and the support of redistributive environmental policies," Journal of Environmental Economics and Management, Elsevier, vol. 114(C).
    8. Malte Welling & Ewa Zawojska & Julian Sagebiel, 2022. "Information, Consequentiality and Credibility in Stated Preference Surveys: A Choice Experiment on Climate Adaptation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 82(1), pages 257-283, May.
    9. Timothy C. Haab & Matthew G. Interis & Daniel R. Petrolia & John C. Whitehead, 2013. "From Hopeless to Curious? Thoughts on Hausman's 'Dubious to Hopeless' Critique of Contingent Valuation," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 35(4), pages 593-612.
    10. Zawojska, Ewa & Bartczak, Anna & Czajkowski, Mikotaj, 2017. "Disentangling impacts of payment and provision consequentiality and risk attitudes on stated preferences," Annual Meeting, 2017, June 18-21, Montreal, Canada 258602, Canadian Agricultural Economics Society.
    11. Malte Oehlmann & Jürgen Meyerhoff, 2017. "Stated preferences towards renewable energy alternatives in Germany – do the consequentiality of the survey and trust in institutions matter?," Journal of Environmental Economics and Policy, Taylor & Francis Journals, vol. 6(1), pages 1-16, January.
    12. Pamela Wicker & John C Whitehead & Daniel S Mason & Bruce K Johnson, 2017. "Public support for hosting the Olympic Summer Games in Germany: The CVM approach," Urban Studies, Urban Studies Journal Limited, vol. 54(15), pages 3597-3614, November.
    13. Richard T. Carson & Theodore Groves & John A. List, 2014. "Consequentiality: A Theoretical and Experimental Exploration of a Single Binary Choice," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 1(1), pages 171-207.
    14. Drichoutis, Andreas C. & Vassilopoulos, Achilleas & Lusk, Jayson & Nayga, Rodolfo M., 2015. "Fair farming: Preferences for fair labor certification using four elicitation methods," MPRA Paper 62546, University Library of Munich, Germany.
    15. Mamadzhanov, Alisher & McCluskey, Jill J. & Li, Tongzhe, 2019. "Willingness to pay for a second-generation bioethanol: A case study of Korea," Energy Policy, Elsevier, vol. 127(C), pages 464-474.
    16. West, Grant H. & Snell, Heather & Kovacs, Kent & Nayga, Rodolfo M., 2020. "Estimation of the preferences for the intertemporal services from groundwater," 2020 Annual Meeting, July 26-28, Kansas City, Missouri 304220, Agricultural and Applied Economics Association.
    17. Zawojska, Ewa & Bartczak, Anna & Czajkowski, Mikołaj, 2019. "Disentangling the effects of policy and payment consequentiality and risk attitudes on stated preferences," Journal of Environmental Economics and Management, Elsevier, vol. 93(C), pages 63-84.
    18. Schaafsma, M. & van Beukering, P.J.H. & Oskolokaite, I., 2017. "Combining focus group discussions and choice experiments for economic valuation of peatland restoration: A case study in Central Kalimantan, Indonesia," Ecosystem Services, Elsevier, vol. 27(PA), pages 150-160.
    19. Christian A. Vossler & Stéphane Bergeron & Maurice Doyon & Daniel Rondeau, 2023. "Revisiting the Gap between the Willingness to Pay and Willingness to Accept for Public Goods," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 10(2), pages 413-445.
    20. Marco Lerro & Riccardo Vecchio & Francesco Caracciolo & Stefano Pascucci & Luigi Cembalo, 2018. "Consumers' heterogeneous preferences for corporate social responsibility in the food industry," Corporate Social Responsibility and Environmental Management, John Wiley & Sons, vol. 25(6), pages 1050-1061, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:39:y:2019:i:7:p:827-841. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.