IDEAS home Printed from https://ideas.repec.org/p/ten/wpaper/2025-03.html
   My bibliography  Save this paper

From simple to complex: A revealed preference test of discrete choice experiment designs

Author

Listed:

Abstract

Researchers employing discrete choice experiments to value publicly provided goods face several experimental design decisions. Fundamental amongst them are whether to ask participants one or many choice questions to elicit valuations, and how many choice options to include in each. To provide guidance for researchers tackling these decisions, and policy makers charged with interpreting welfare estimates based on these decisions, we conducted a financially incentivized online field experiment to provide a ground truth comparison of three leading elicitation approaches. A single binary choice question and a sequence of binary choice questions yield equal willingness-to-pay estimates. A sequence of trinary choices results in lower demand estimates. The latter approach, while dominant in the stated preference literature, encourages serial status quo choices due to increased task complexity, and is prone to framing effects in that the value for one good depends on the other good included in the choice set. These behavioral effects more than offset the theoretical efficiency advantage of this elicitation approach.

Suggested Citation

  • Christian A. Vossler & Ewa Zawojska, 2025. "From simple to complex: A revealed preference test of discrete choice experiment designs," Working Papers 2025-03, University of Tennessee, Department of Economics.
    • Handle: RePEc:ten:wpaper:2025-03
    as

    Download full text from publisher

    File URL: https://web.utk.edu/~jhollad3/RePEc/2025-03.pdf
    File Function: First version, 2025
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Weng, Weizhe & Morrison, Mark D. & Boyle, Kevin J. & Boxall, Peter C. & Rose, John, 2021. "Effects of the number of alternatives in public good discrete choice experiments," Ecological Economics, Elsevier, vol. 182(C).
    2. Christian A. Vossler & Maurice Doyon & Daniel Rondeau, 2012. "Truth in Consequentiality: Theory and Field Evidence on Discrete Choice Experiments," American Economic Journal: Microeconomics, American Economic Association, vol. 4(4), pages 145-171, November.
    3. Christian A. Vossler & David A. Keiser & Catherine L. Kling & Daniel J. Phaneuf, 2025. "Information Scripts and the Incentive Compatibility of Discrete Choice Experiments," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 12(2), pages 459-492.
    4. David Hensher, 2006. "Revealing Differences in Willingness to Pay due to the Dimensionality of Stated Choice Designs: An Initial Assessment," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 34(1), pages 7-44, May.
    5. Frank Lupi & Daniel J. Phaneuf & Roger H. von Haefen, 2020. "Best Practices for Implementing Recreation Demand Models," Review of Environmental Economics and Policy, University of Chicago Press, vol. 14(2), pages 302-323.
    6. Daniel Rondeau & Christian A. Vossler, 2024. "Incentive compatibility and respondent beliefs: Consequentiality and game form," Working Papers 2024-02, University of Tennessee, Department of Economics.
    7. Poe, Gregory L., 2016. "Behavioral Anomalies in Contingent Values and Actual Choices," Agricultural and Resource Economics Review, Cambridge University Press, vol. 45(2), pages 246-269, August.
    8. Richard Carson & Jordan Louviere, 2011. "A Common Nomenclature for Stated Preference Elicitation Approaches," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 49(4), pages 539-559, August.
    9. Jing Zhang & Wiktor L. Adamowicz, 2011. "Unraveling the Choice Format Effect: A Context-Dependent Random Utility Model," Land Economics, University of Wisconsin Press, vol. 87(4), pages 730-743.
    10. John Rolfe & Jill Windle, 2012. "Restricted versus Unrestricted Choice in Labeled Choice Experiments: Exploring the Trade‐Offs of Expanding Choice Dimensions," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 60(1), pages 53-70, March.
    11. Caussade, Sebastián & Ortúzar, Juan de Dios & Rizzi, Luis I. & Hensher, David A., 2005. "Assessing the influence of design dimensions on stated choice experiment estimates," Transportation Research Part B: Methodological, Elsevier, vol. 39(7), pages 621-640, August.
    12. Mikolaj Czajkowski & Marek Giergiczny & William H. Greene, 2014. "Learning and Fatigue Effects Revisited: Investigating the Effects of Accounting for Unobservable Preference and Scale Heterogeneity," Land Economics, University of Wisconsin Press, vol. 90(2), pages 324-351.
    13. Patrick Lloyd-Smith & Ewa Zawojska & Wiktor Adamowicz, 2020. "Moving beyond the Contingent Valuation versus Choice Experiment Debate: Presentation Effects in Stated Preference," Land Economics, University of Wisconsin Press, vol. 96(1), pages 1-24.
    14. David A. Hensher, 2004. "Identifying the Influence of Stated Choice Design Dimensionality on Willingness to Pay for Travel Time Savings," Journal of Transport Economics and Policy, University of Bath, vol. 38(3), pages 425-446, September.
    15. Rose, John M. & Hensher, David A. & Caussade, Sebastian & Ortúzar, Juan de Dios & Jou, Rong-Chang, 2009. "Identifying differences in willingness to pay due to dimensionality in stated choice experiments: a cross country analysis," Journal of Transport Geography, Elsevier, vol. 17(1), pages 21-29.
    16. Leonhard K. Lades & Ewa Zawojska & Robert J. Johnston & Nick Hanley & Liam Delaney & Mikołaj Czajkowski, 2025. "Anomalies or Expected Behaviors? Understanding Stated Preferences and Welfare Implications in Light of Contemporary Behavioral Economics," Review of Environmental Economics and Policy, University of Chicago Press, vol. 19(1), pages 48-68.
    17. Yaron Azrieli & Christopher P. Chambers & Paul J. Healy, 2018. "Incentives in Experiments: A Theoretical Analysis," Journal of Political Economy, University of Chicago Press, vol. 126(4), pages 1472-1503.
    18. Patricia A. Champ & Richard C. Bishop, 2006. "Is Willingness to Pay for a Public Good Sensitive to the Elicitation Format?," Land Economics, University of Wisconsin Press, vol. 82(2), pages 162-173.
    19. Welsh, Michael P. & Poe, Gregory L., 1998. "Elicitation Effects in Contingent Valuation: Comparisons to a Multiple Bounded Discrete Choice Approach," Journal of Environmental Economics and Management, Elsevier, vol. 36(2), pages 170-185, September.
    20. Ian J. Bateman & Alistair Munro & Gregory L. Poe, 2008. "Decoy Effects in Choice Experiments and Contingent Valuation: Asymmetric Dominance," Land Economics, University of Wisconsin Press, vol. 84(1), pages 115-127.
    21. Richard Carson & Theodore Groves, 2007. "Incentive and informational properties of preference questions," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 37(1), pages 181-210, May.
    22. Regier, Dean A. & Watson, Verity & Burnett, Heather & Ungar, Wendy J., 2014. "Task complexity and response certainty in discrete choice experiments: An application to drug treatments for juvenile idiopathic arthritis," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 50(C), pages 40-49.
    23. Wiktor Adamowicz & Peter Boxall & Michael Williams & Jordan Louviere, 1998. "Stated Preference Approaches for Measuring Passive Use Values: Choice Experiments and Contingent Valuation," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 80(1), pages 64-75.
    24. Adamowicz, Wiktor & Dupont, Diane & Krupnick, Alan & Zhang, Jing, 2011. "Valuation of cancer and microbial disease risk reductions in municipal drinking water: An analysis of risk context using multiple valuation methods," Journal of Environmental Economics and Management, Elsevier, vol. 61(2), pages 213-226, March.
    25. Richard T. Carson & Theodore Groves & John A. List, 2014. "Consequentiality: A Theoretical and Experimental Exploration of a Single Binary Choice," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 1(1), pages 171-207.
    26. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    27. David Revelt & Kenneth Train, 1998. "Mixed Logit With Repeated Choices: Households' Choices Of Appliance Efficiency Level," The Review of Economics and Statistics, MIT Press, vol. 80(4), pages 647-657, November.
    28. Day, Brett & Bateman, Ian J. & Carson, Richard T. & Dupont, Diane & Louviere, Jordan J. & Morimoto, Sanae & Scarpa, Riccardo & Wang, Paul, 2012. "Ordering effects and choice set awareness in repeat-response stated preference studies," Journal of Environmental Economics and Management, Elsevier, vol. 63(1), pages 73-91.
    29. Boxall, Peter C. & Adamowicz, Wiktor L. & Swait, Joffre & Williams, Michael & Louviere, Jordan, 1996. "A comparison of stated preference methods for environmental valuation," Ecological Economics, Elsevier, vol. 18(3), pages 243-253, September.
    30. Jürgen Meyerhoff & Morten Mørkbak & Søren Olsen, 2014. "A Meta-study Investigating the Sources of Protest Behaviour in Stated Preference Surveys," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 58(1), pages 35-57, May.
    31. Roger H. von Haefen & D. Matthew Massey & Wiktor L. Adamowicz, 2005. "Serial Nonparticipation in Repeated Discrete Choice Models," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 87(4), pages 1061-1076.
    32. repec:ten:wpaper:2024-02 is not listed on IDEAS
    33. Oehlmann, Malte & Meyerhoff, Jürgen & Mariel, Petr & Weller, Priska, 2017. "Uncovering context-induced status quo effects in choice experiments," Journal of Environmental Economics and Management, Elsevier, vol. 81(C), pages 59-73.
    34. Ulrich J Frey & Frauke Pirscher, 2019. "Distinguishing protest responses in contingent valuation: A conceptualization of motivations and attitudes behind them," PLOS ONE, Public Library of Science, vol. 14(1), pages 1-20, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Weng, Weizhe & Morrison, Mark D. & Boyle, Kevin J. & Boxall, Peter C. & Rose, John, 2021. "Effects of the number of alternatives in public good discrete choice experiments," Ecological Economics, Elsevier, vol. 182(C).
    2. Zawojska, Ewa & Czajkowski, Mikolaj & Giergiczny, Marek, "undated". "Valuing tap water quality improvements using stated preference methods. Does the number of discrete choice options matter?," 2018 Annual Meeting, August 5-7, Washington, D.C. 274019, Agricultural and Applied Economics Association.
    3. Oehlmann, Malte & Meyerhoff, Jürgen & Mariel, Petr & Weller, Priska, 2017. "Uncovering context-induced status quo effects in choice experiments," Journal of Environmental Economics and Management, Elsevier, vol. 81(C), pages 59-73.
    4. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    5. Daniel Rondeau & Christian A. Vossler, 2024. "Incentive compatibility and respondent beliefs: Consequentiality and game form," Working Papers 2024-02, University of Tennessee, Department of Economics.
    6. Weng, Weizhe & Morrison, Mark & Boyle, Kevin J. & Boxall, Peter C., "undated". "The effect of the number of alternatives in choice experiment questions," 2017 Annual Meeting, July 30-August 1, Chicago, Illinois 259179, Agricultural and Applied Economics Association.
    7. Weng, Weizhe & Morrison, Mark & Boyle, Kevin & Boxall, Peter, 2017. "The effect of the number of alternatives in a choice experiment with an application to the Macquarie Marshes, AU," 2017 Annual Meeting, February 4-7, 2017, Mobile, Alabama 252836, Southern Agricultural Economics Association.
    8. Christian A. Vossler & Ewa Zawojska, 2018. "Toward a better understanding of elicitation effects in stated preference studies," Working Papers 2018-01, University of Tennessee, Department of Economics.
    9. Boxebeld, Sander, 2024. "Ordering effects in discrete choice experiments: A systematic literature review across domains," Journal of choice modelling, Elsevier, vol. 51(C).
    10. Jürgen Meyerhoff & Malte Oehlmann & Priska Weller, 2015. "The Influence of Design Dimensions on Stated Choices in an Environmental Context," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 61(3), pages 385-407, July.
    11. West, Grant H. & Snell, Heather & Kovacs, Kent & Nayga, Rodolfo M., 2020. "Estimation of the preferences for the intertemporal services from groundwater," 2020 Annual Meeting, July 26-28, Kansas City, Missouri 304220, Agricultural and Applied Economics Association.
    12. Haghani, Milad & Bliemer, Michiel C.J. & Rose, John M. & Oppewal, Harmen & Lancsar, Emily, 2021. "Hypothetical bias in stated choice experiments: Part II. Conceptualisation of external validity, sources and explanations of bias and effectiveness of mitigation methods," Journal of choice modelling, Elsevier, vol. 41(C).
    13. Ahi, Jülide Ceren & Aanesen, Margrethe & Kipperberg, Gorm, 2023. "Testing the sensitivity of stated environmental preferences to variations in choice architecture," Ecological Economics, Elsevier, vol. 205(C).
    14. Patrick Lloyd-Smith & Ewa Zawojska & Wiktor Adamowicz, 2020. "Moving beyond the Contingent Valuation versus Choice Experiment Debate: Presentation Effects in Stated Preference," Land Economics, University of Wisconsin Press, vol. 96(1), pages 1-24.
    15. Mikołaj Czajkowski & Marek Giergiczny & William H. Greene, 2012. "Learning and Fatigue Effects Revisited. The Impact of Accounting for Unobservable Preference and Scale Heterogeneity on Perceived Ordering Effects in Multiple Choice Task Discrete Choice Experiments," Working Papers 2012-08, Faculty of Economic Sciences, University of Warsaw.
    16. Shr, Yau-Huo (Jimmy) & Zhang, Wendong, 2024. "Omitted downstream attributes and the benefits of nutrient reductions: Implications for choice experiments," Ecological Economics, Elsevier, vol. 222(C).
    17. Nguyen, Thanh Cong & Le, Hoa Thu & Nguyen, Hang Dieu & Ngo, Mai Thanh & Nguyen, Hong Quang, 2021. "Examining ordering effects and strategic behaviour in a discrete choice experiment," Economic Analysis and Policy, Elsevier, vol. 70(C), pages 394-413.
    18. Nick Hanley & Mikołaj Czajkowski, 2017. "Stated Preference valuation methods: an evolving tool for understanding choices and informing policy," Working Papers 2017-01, Faculty of Economic Sciences, University of Warsaw.
    19. Haghani, Milad & Bliemer, Michiel C.J. & Hensher, David A., 2021. "The landscape of econometric discrete choice modelling research," Journal of choice modelling, Elsevier, vol. 40(C).
    20. Xie, Lusi & Adamowicz, Wiktor & Kecinski, Maik & Fooks, Jacob R., 2022. "Using economic experiments to assess the validity of stated preference contingent behavior responses," Journal of Environmental Economics and Management, Elsevier, vol. 114(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • D61 - Microeconomics - - Welfare Economics - - - Allocative Efficiency; Cost-Benefit Analysis
    • D82 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Asymmetric and Private Information; Mechanism Design
    • H4 - Public Economics - - Publicly Provided Goods
    • Q51 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Environmental Economics - - - Valuation of Environmental Effects

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ten:wpaper:2025-03. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Scott Holladay (email available below). General contact details of provider: https://edirc.repec.org/data/ecutkus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.