IDEAS home Printed from https://ideas.repec.org/p/ags/aaea25/360817.html
   My bibliography  Save this paper

The Efficacy of Seven Hypothetical Bias Mitigation Methods

Author

Listed:
  • Penn, Jerrod
  • Jiang, Qi
  • Hu, Wuyang

Abstract

Hypothetical Bias is the difference between hypothetical and real behaviors, a central issue of convergent validity in stated preference methods. Numerous methods have been developed to mitigate hypothetical bias. We test seven such ex-ante, simul, and ex-post methods: Honest priming, cheap talk, oath, repeated opt-out reminder, default, unsure, and certainty follow-up and compare these outcomes to a hypothetical-control and incentivized treatment. We find that certainty follow-up successfully eliminates hypothetical bias. Default shows some success in reducing HB though does not eliminate it. All other methods show much more limited success at reducing HB.

Suggested Citation

  • Penn, Jerrod & Jiang, Qi & Hu, Wuyang, 2025. "The Efficacy of Seven Hypothetical Bias Mitigation Methods," 2025 AAEA & WAEA Joint Annual Meeting, July 27-29, 2025, Denver, CO 360817, Agricultural and Applied Economics Association.
    • Handle: RePEc:ags:aaea25:360817
    DOI: 10.22004/ag.econ.360817
    as

    Download full text from publisher

    File URL: https://ageconsearch.umn.edu/record/360817/files/75209_95528_105300_Penn-Seven_HB_Mitigation_Methods_AAEA_2025.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.22004/ag.econ.360817?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Di Fang & Rodolfo M. Nayga & Grant H. West & Claudia Bazzani & Wei Yang & Benjamin C. Lok & Charles E. Levy & Heather A. Snell, 2021. "On the Use of Virtual Reality in Mitigating Hypothetical Bias in Choice Experiments," American Journal of Agricultural Economics, John Wiley & Sons, vol. 103(1), pages 142-161, January.
    2. Jacquemet, Nicolas & Joule, Robert-Vincent & Luchini, Stéphane & Shogren, Jason F., 2013. "Preference elicitation under oath," Journal of Environmental Economics and Management, Elsevier, vol. 65(1), pages 110-132.
    3. Jonas Schmidt & Tammo H. A. Bijmolt, 2020. "Accurately measuring willingness to pay for consumer goods: a meta-analysis of the hypothetical bias," Journal of the Academy of Marketing Science, Springer, vol. 48(3), pages 499-518, May.
    4. Francisco Alpizar & Fredrik Carlsson & Olof Johansson-Stenman, 2008. "Does context matter more for hypothetical than for actual contributions? Evidence from a natural field experiment," Experimental Economics, Springer;Economic Science Association, vol. 11(3), pages 299-314, September.
    5. Vossler, Christian A. & Watson, Sharon B., 2013. "Understanding the consequences of consequentiality: Testing the validity of stated preferences in the field," Journal of Economic Behavior & Organization, Elsevier, vol. 86(C), pages 137-147.
    6. Uz, Dilek & Mamkhezri, Jamal, 2024. "Household willingness to pay for various attributes of residential solar panels: Evidence from a discrete choice experiment," Energy Economics, Elsevier, vol. 130(C).
    7. Jamal Mamkhezri & Jennifer A. Thacher & Janie M. Chermak & Robert P. Berrens, 2020. "Does the solemn oath lower WTP responses in a discrete choice experiment application to solar energy?," Journal of Environmental Economics and Policy, Taylor & Francis Journals, vol. 9(4), pages 447-473, October.
    8. Nathan P Kemper & Jennie S Popp & Rodolfo M Nayga, 2020. "A query theory account of a discrete choice experiment under oath," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 47(3), pages 1133-1172.
    9. Ladenburg, Jacob & Olsen, Søren Bøye, 2014. "Augmenting short Cheap Talk scripts with a repeated Opt-Out Reminder in Choice Experiment surveys," Resource and Energy Economics, Elsevier, vol. 37(C), pages 39-63.
    10. Ronald G. Cummings & Laura Osborne Taylor, 1998. "Does Realism Matter in Contingent Valuation Surveys?," Land Economics, University of Wisconsin Press, vol. 74(2), pages 203-215.
    11. Loomis, John B., 2014. "2013 WAEA Keynote Address: Strategies for Overcoming Hypothetical Bias in Stated Preference Surveys," Journal of Agricultural and Resource Economics, Western Agricultural Economics Association, vol. 39(01), pages 1-13, April.
    12. Christian A. Vossler & David A. Keiser & Catherine L. Kling & Daniel J. Phaneuf, 2025. "Information Scripts and the Incentive Compatibility of Discrete Choice Experiments," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 12(2), pages 459-492.
    13. Grebitus, Carola & Lusk, Jayson L. & Nayga, Rodolfo M., 2013. "Explaining differences in real and hypothetical experimental auctions and choice experiments with personality," Journal of Economic Psychology, Elsevier, vol. 36(C), pages 11-26.
    14. Nick Hanley & Christopher Boyce & Mikołaj Czajkowski & Steve Tucker & Charles Noussair & Michael Townsend, 2017. "Sad or Happy? The Effects of Emotions on Stated Preferences for Environmental Goods," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 68(4), pages 821-846, December.
    15. James Murphy & P. Allen & Thomas Stevens & Darryl Weatherhead, 2005. "A Meta-analysis of Hypothetical Bias in Stated Preference Valuation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 30(3), pages 313-325, March.
    16. Prakashan Chellattan Veettil & Yashodha Yashodha & Joseph Vecci, 2025. "Hypothetical bias and cognitive ability: Farmers' preference for crop insurance products†," American Journal of Agricultural Economics, John Wiley & Sons, vol. 107(3), pages 888-924, May.
    17. Tiziana de-Magistris & Azucena Gracia & Rodolfo M. Nayga, 2013. "On the Use of Honesty Priming Tasks to Mitigate Hypothetical Bias in Choice Experiments," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 95(5), pages 1136-1154.
    18. Gregory Howard & Brian E. Roe & Erik C. Nisbet & Jay F. Martin, 2017. "Hypothetical Bias Mitigation Techniques in Choice Experiments: Do Cheap Talk and Honesty Priming Effects Fade with Repeated Choices?," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 543-573.
    19. repec:hal:pseose:halshs-00731244 is not listed on IDEAS
    20. Alpizar, Francisco & Carlsson, Fredrik & Johansson-Stenman, Olof, 2008. "Full title Does Context Matter More for Hypothetical Than for Actual Contributions? Evidence from a Natural Field Experiment," RFF Working Paper Series dp-08-02-efd, Resources for the Future.
    21. Richard C. Ready & Patricia A. Champ & Jennifer L. Lawton, 2010. "Using Respondent Uncertainty to Mitigate Hypothetical Bias in a Stated Choice Experiment," Land Economics, University of Wisconsin Press, vol. 86(2), pages 363-381.
    22. Bulte, Erwin & Gerking, Shelby & List, John A. & de Zeeuw, Aart, 2005. "The effect of varying the causes of environmental problems on stated WTP values: evidence from a field study," Journal of Environmental Economics and Management, Elsevier, vol. 49(2), pages 330-342, March.
    23. Malone, Trey & Lusk, Jayson L., 2018. "Consequences of Participant Inattention with an Application to Carbon Taxes for Meat Products," Ecological Economics, Elsevier, vol. 145(C), pages 218-230.
    24. Glenn Blomquist & Karen Blumenschein & Magnus Johannesson, 2009. "Eliciting Willingness to Pay without Bias using Follow-up Certainty Statements: Comparisons between Probably/Definitely and a 10-point Certainty Scale," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 43(4), pages 473-502, August.
    25. Laura O. Taylor & Ronald G. Cummings, 1999. "Unbiased Value Estimates for Environmental Goods: A Cheap Talk Design for the Contingent Valuation Method," American Economic Review, American Economic Association, vol. 89(3), pages 649-665, June.
    26. Amelia Ahles & Marco A. Palma & Andreas C. Drichoutis, 2024. "Testing the effectiveness of lottery incentives in online experiments," American Journal of Agricultural Economics, John Wiley & Sons, vol. 106(4), pages 1435-1453, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Haghani, Milad & Bliemer, Michiel C.J. & Rose, John M. & Oppewal, Harmen & Lancsar, Emily, 2021. "Hypothetical bias in stated choice experiments: Part II. Conceptualisation of external validity, sources and explanations of bias and effectiveness of mitigation methods," Journal of choice modelling, Elsevier, vol. 41(C).
    2. Jiang, Qi & Penn, Jerrod & Hu, Wuyang, 2022. "Real payment priming to reduce potential hypothetical bias," Journal of choice modelling, Elsevier, vol. 45(C).
    3. Milad Haghani & Michiel C. J. Bliemer & John M. Rose & Harmen Oppewal & Emily Lancsar, 2021. "Hypothetical bias in stated choice experiments: Part II. Macro-scale analysis of literature and effectiveness of bias mitigation methods," Papers 2102.02945, arXiv.org.
    4. Crastes dit Sourd, Romain & Zawojska, Ewa & Mahieu, Pierre-Alexandre & Louviere, Jordan, 2018. "Mitigating strategic misrepresentation of values in open-ended stated preference surveys by using negative reinforcement," Journal of choice modelling, Elsevier, vol. 28(C), pages 153-166.
    5. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    6. Mohammed H. Alemu & Søren B. Olsen, 2017. "Can a Repeated Opt-Out Reminder remove hypothetical bias in discrete choice experiments? An application to consumer valuation of novel food products," IFRO Working Paper 2017/05, University of Copenhagen, Department of Food and Resource Economics.
    7. Penn, Jerrod & Hu, Wuyang & Ye, Tao, 2024. "Efficacy of hypothetical bias mitigation techniques: A cross-country comparison," Journal of Environmental Economics and Management, Elsevier, vol. 125(C).
    8. Sergio Colombo & Wiktor Budziński & Mikołaj Czajkowski & Klaus Glenk, 2020. "Ex-ante and ex-post measures to mitigate hypothetical bias. Are they alternative or complementary tools to increase the reliability and validity of DCE estimates?," Working Papers 2020-20, Faculty of Economic Sciences, University of Warsaw.
    9. Wuepper, David & Clemm, Alexandra & Wree, Philipp, 2019. "The preference for sustainable coffee and a new approach for dealing with hypothetical bias," Journal of Economic Behavior & Organization, Elsevier, vol. 158(C), pages 475-486.
    10. Jie He & Jérôme Dupras & Thomas G. Poder, 2018. "Payment and Provision Consequentiality in Voluntary Contribution Mechanism: Single or Double “Knife-Edge” Evidence?," Cahiers de recherche 18-02, Departement d'économique de l'École de gestion à l'Université de Sherbrooke.
    11. Jiang, Qi & Vassalos, Michael & Nian, Yefan & Thayer, Anastasia & Silva, Felipe, 2025. "Evaluating the impact of alternative opt out strategies on potential hypothetical bias in discrete choice experiment," 2025 AAEA & WAEA Joint Annual Meeting, July 27-29, 2025, Denver, CO 360818, Agricultural and Applied Economics Association.
    12. Penn, Jerrod & Hu, Wuyang & Ye, Tao, "undated". "Cheap Talk, Consequentiality, and Certainty Follow-up as Hypothetical Bias Mitigation Techniques: A Cross Country Comparison," 2018 Annual Meeting, August 5-7, Washington, D.C. 274018, Agricultural and Applied Economics Association.
    13. Haghani, Milad & Bliemer, Michiel C.J. & Rose, John M. & Oppewal, Harmen & Lancsar, Emily, 2021. "Hypothetical bias in stated choice experiments: Part I. Macro-scale analysis of literature and integrative synthesis of empirical evidence from applied economics, experimental psychology and neuroimag," Journal of choice modelling, Elsevier, vol. 41(C).
    14. Sergio Colombo & Wiktor Budziński & Mikołaj Czajkowski & Klaus Glenk, 2022. "The relative performance of ex‐ante and ex‐post measures to mitigate hypothetical and strategic bias in a stated preference study," Journal of Agricultural Economics, Wiley Blackwell, vol. 73(3), pages 845-873, September.
    15. Kar Ho Lim & Wuyang Hu, 2023. "Contextual reference price in choice experiments," American Journal of Agricultural Economics, John Wiley & Sons, vol. 105(4), pages 1288-1306, August.
    16. Penn, Jerrod & Hu, Wuyang, 2019. "Cheap talk efficacy under potential and actual Hypothetical Bias: A meta-analysis," Journal of Environmental Economics and Management, Elsevier, vol. 96(C), pages 22-35.
    17. Ewa Zawojska & Pierre-Alexandre Mahieu & Romain Crastes & Jordan Louviere, 2016. "On a way to overcome strategic overbidding in open-ended stated preference surveys: A recoding approach," Working Papers 2016-34, Faculty of Economic Sciences, University of Warsaw.
    18. Penn, Jerrod & Hu, Wuyang, 2016. "Making the Most of Cheap Talk in an Online Survey," 2016 Annual Meeting, July 31-August 2, Boston, Massachusetts 236171, Agricultural and Applied Economics Association.
    19. Milad Haghani & Michiel C. J. Bliemer & John M. Rose & Harmen Oppewal & Emily Lancsar, 2021. "Hypothetical bias in stated choice experiments: Part I. Integrative synthesis of empirical evidence and conceptualisation of external validity," Papers 2102.02940, arXiv.org.
    20. Chavez, Daniel E. & Palma, Marco A. & Nayga, Rodolfo M., "undated". "When does real become consequential in non-hypothetical choice experiments?," 2018 Annual Meeting, August 5-7, Washington, D.C. 274040, Agricultural and Applied Economics Association.

    More about this item

    Keywords

    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ags:aaea25:360817. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: AgEcon Search (email available below). General contact details of provider: https://edirc.repec.org/data/aaeaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.