IDEAS home Printed from https://ideas.repec.org/a/spr/joamsc/v48y2020i3d10.1007_s11747-019-00666-6.html
   My bibliography  Save this article

Accurately measuring willingness to pay for consumer goods: a meta-analysis of the hypothetical bias

Author

Listed:
  • Jonas Schmidt

    (University of Muenster)

  • Tammo H. A. Bijmolt

    (University of Groningen)

Abstract

Consumers’ willingness to pay (WTP) is highly relevant to managers and academics, and the various direct and indirect methods used to measure it vary in their accuracy, defined as how closely the hypothetically measured WTP (HWTP) matches consumers’ real WTP (RWTP). The difference between HWTP and RWTP is the “hypothetical bias.” A prevalent assumption in marketing science is that indirect methods measure WTP more accurately than do direct methods. With a meta-analysis of 77 studies reported in 47 papers and resulting in 115 effect sizes, we test that assumption by assessing the hypothetical bias. The total sample consists of 24,347 included observations for HWTP and 20,656 for RWTP. Moving beyond extant meta-analyses in marketing, we introduce an effect size metric (i.e., response ratio) and a novel analysis method (i.e., multivariate mixed linear model) to analyze the stochastically dependent effect sizes. Our findings are relevant for academic researchers and managers. First, on average, the hypothetical bias is 21%, and this study provides a reference point for the expected magnitude of the hypothetical bias. Second, the deviation primarily depends on the use of a direct or indirect method for measuring HWTP. In contrast with conventional wisdom, indirect methods actually overestimate RWTP significantly stronger than direct methods. Third, the hypothetical bias is greater for higher valued products, specialty goods (cf. other product types), and within-subject designs (cf. between-subject designs), thus a stronger downward adjustment of HWTP values is necessary to reflect consumers’ RWTP.

Suggested Citation

  • Jonas Schmidt & Tammo H. A. Bijmolt, 2020. "Accurately measuring willingness to pay for consumer goods: a meta-analysis of the hypothetical bias," Journal of the Academy of Marketing Science, Springer, vol. 48(3), pages 499-518, May.
  • Handle: RePEc:spr:joamsc:v:48:y:2020:i:3:d:10.1007_s11747-019-00666-6
    DOI: 10.1007/s11747-019-00666-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11747-019-00666-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11747-019-00666-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Benjamin Scheibehenne & Rainer Greifeneder & Peter M. Todd, 2010. "Can There Ever Be Too Many Options? A Meta-Analytic Review of Choice Overload," Journal of Consumer Research, Oxford University Press, vol. 37(3), pages 409-425, October.
    2. Noussair, Charles & Robin, Stephane & Ruffieux, Bernard, 2004. "Revealing consumers' willingness-to-pay: A comparison of the BDM mechanism and the Vickrey auction," Journal of Economic Psychology, Elsevier, vol. 25(6), pages 725-741, December.
    3. Gensler, Sonja & Hinz, Oliver & Skiera, Bernd & Theysohn, Sven, 2012. "Willingness-to-pay estimation with choice-based conjoint analysis: Addressing extreme response behavior with individually adapted designs," European Journal of Operational Research, Elsevier, vol. 219(2), pages 368-378.
    4. Shogren, Jason F. & Margolis, Michael & Koo, Cannon & List, John A., 2001. "A random nth-price auction," Journal of Economic Behavior & Organization, Elsevier, vol. 46(4), pages 409-421, December.
    5. Hensher, David A., 2010. "Hypothetical bias, choice experiments and willingness to pay," Transportation Research Part B: Methodological, Elsevier, vol. 44(6), pages 735-752, July.
    6. James Murphy & P. Allen & Thomas Stevens & Darryl Weatherhead, 2005. "A Meta-analysis of Hypothetical Bias in Stated Preference Valuation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 30(3), pages 313-325, March.
    7. Charness, Gary & Gneezy, Uri & Kuhn, Michael A., 2012. "Experimental methods: Between-subject and within-subject design," Journal of Economic Behavior & Organization, Elsevier, vol. 81(1), pages 1-8.
    8. Helen R. Neill & Ronald G. Cummings & Philip T. Ganderton & Glenn W. Harrison & Thomas McGuckin, 1994. "Hypothetical Surveys and Real Economic Commitments," Land Economics, University of Wisconsin Press, vol. 70(2), pages 145-154.
    9. Craig R. Fox & Amos Tversky, 1995. "Ambiguity Aversion and Comparative Ignorance," The Quarterly Journal of Economics, Oxford University Press, vol. 110(3), pages 585-603.
    10. Viechtbauer, Wolfgang, 2010. "Conducting Meta-Analyses in R with the metafor Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 36(i03).
    11. Bolton, Gary E. & Ockenfels, Axel, 2014. "Does laboratory trading mirror behavior in real world markets? Fair bargaining and competitive bidding on eBay," Journal of Economic Behavior & Organization, Elsevier, vol. 97(C), pages 143-154.
    12. Eggers, Felix & Sattler, Henrik, 2009. "Hybrid individualized two-level choice-based conjoint (HIT-CBC): A new method for measuring preference structures with many attribute levels," International Journal of Research in Marketing, Elsevier, vol. 26(2), pages 108-118.
    13. Min Ding & Rajdeep Grewal & John Liechty, 2005. "Incentive-aligned conjoint analysis," Framed Field Experiments 00139, The Field Experiments Website.
    14. Kagel, John H & Harstad, Ronald M & Levin, Dan, 1987. "Information Impact and Allocation Rules in Auctions with Affiliated Private Values: A Laboratory Study," Econometrica, Econometric Society, vol. 55(6), pages 1275-1304, November.
    15. Brown, Thomas C. & Ajzen, Icek & Hrubes, Daniel, 2003. "Further tests of entreaties to avoid hypothetical bias in referendum contingent valuation," Journal of Environmental Economics and Management, Elsevier, vol. 46(2), pages 353-361, September.
    16. Arts, Joep W.C. & Frambach, Ruud T. & Bijmolt, Tammo H.A., 2011. "Generalizations on consumer innovation adoption: A meta-analysis on drivers of intention and behavior," International Journal of Research in Marketing, Elsevier, vol. 28(2), pages 134-144.
    17. Ockenfels, Axel & Roth, Alvin E., 2006. "Late and multiple bidding in second price Internet auctions: Theory and evidence concerning different rules for ending an auction," Games and Economic Behavior, Elsevier, vol. 55(2), pages 297-320, May.
    18. Jayson L. Lusk & Ted C. Schroeder, 2004. "Are Choice Experiments Incentive Compatible? A Test with Quality Differentiated Beef Steaks," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 86(2), pages 467-482.
    19. Dan Ariely & George Loewenstein & Drazen Prelec, 2003. ""Coherent Arbitrariness": Stable Demand Curves Without Stable Preferences," The Quarterly Journal of Economics, Oxford University Press, vol. 118(1), pages 73-106.
    20. Dan Ariely & Axel Ockenfels & Alvin E. Roth, 2005. "An Experimental Analysis of Ending Rules in Internet Auctions," RAND Journal of Economics, The RAND Corporation, vol. 36(4), pages 890-907, Winter.
    21. Thomas C. Brown & Patricia A. Champ & Richard C. Bishop & Daniel W. McCollum, 1996. "Which Response Format Reveals the Truth about Donations to a Public Good?," Land Economics, University of Wisconsin Press, vol. 72(2), pages 152-166.
    22. Tully, Stephanie M. & Winer, Russell S., 2014. "The Role of the Beneficiary in Willingness to Pay for Socially Responsible Products: A Meta-analysis," Journal of Retailing, Elsevier, vol. 90(2), pages 255-274.
    23. Gensler, Sonja & Neslin, Scott A. & Verhoef, Peter C., 2017. "The Showrooming Phenomenon: It's More than Just About Price," Journal of Interactive Marketing, Elsevier, vol. 38(C), pages 29-43.
    24. John List & Craig Gallet, 2001. "What Experimental Protocol Influence Disparities Between Actual and Hypothetical Stated Values?," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 20(3), pages 241-254, November.
    25. Richard T. Carson & Nicholas E. Flores & Kerry M. Martin & Jennifer L. Wright, 1996. "Contingent Valuation and Revealed Preference Methodologies: Comparing the Estimates for Quasi-Public Goods," Land Economics, University of Wisconsin Press, vol. 72(1), pages 80-99.
    26. Benjamin Bushong & Lindsay M. King & Colin F. Camerer & Antonio Rangel, 2010. "Pavlovian Processes in Consumer Choice: The Physical Presence of a Good Increases Willingness-to-Pay," American Economic Review, American Economic Association, vol. 100(4), pages 1556-1571, September.
    27. William Vickrey, 1961. "Counterspeculation, Auctions, And Competitive Sealed Tenders," Journal of Finance, American Finance Association, vol. 16(1), pages 8-37, March.
    28. Lilien, Gary L., 2016. "The B2B Knowledge Gap," International Journal of Research in Marketing, Elsevier, vol. 33(3), pages 543-556.
    29. Abhishek Borah & Xin (Shane) Wang & Jun Hyun (Joseph) Ryoo, 2018. "Understanding Influence of Marketing Thought on Practice: an Analysis of Business Journals Using Textual and Latent Dirichlet Allocation (LDA) Analysis," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 5(3), pages 146-161, December.
    30. Kamel Jedidi & Z. John Zhang, 2002. "Augmenting Conjoint Analysis to Estimate Consumer Reservation Price," Management Science, INFORMS, vol. 48(10), pages 1350-1368, October.
    31. Christian Schlereth & Bernd Skiera, 2017. "Two New Features in Discrete Choice Experiments to Improve Willingness-to-Pay Estimation That Result in SDR and SADR: Separated (Adaptive) Dual Response," Management Science, INFORMS, vol. 63(3), pages 829-842, March.
    32. Ariely, Dan & Loewenstein, George & Prelec, Drazen, 2006. "Tom Sawyer and the construction of value," Journal of Economic Behavior & Organization, Elsevier, vol. 60(1), pages 1-10, May.
    33. Michael Steiner & Andreas Eggert & Wolfgang Ulaga & Klaus Backhaus, 2016. "Do customized service packages impede value capture in industrial markets?," Journal of the Academy of Marketing Science, Springer, vol. 44(2), pages 151-165, March.
    34. Harrison, Glenn W. & Rutström, E. Elisabet, 2008. "Experimental Evidence on the Existence of Hypothetical Bias in Value Elicitation Methods," Handbook of Experimental Economics Results, in: Charles R. Plott & Vernon L. Smith (ed.), Handbook of Experimental Economics Results, edition 1, volume 1, chapter 81, pages 752-767, Elsevier.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mohammed H. Alemu & Søren B. Olsen, 2017. "Can a Repeated Opt-Out Reminder remove hypothetical bias in discrete choice experiments? An application to consumer valuation of novel food products," IFRO Working Paper 2017/05, University of Copenhagen, Department of Food and Resource Economics.
    2. Hofstetter, Reto & Miller, Klaus M. & Krohmer, Harley & Zhang, Z. John, 2021. "A de-biased direct question approach to measuring consumers' willingness to pay," International Journal of Research in Marketing, Elsevier, vol. 38(1), pages 70-84.
    3. Haghani, Milad & Bliemer, Michiel C.J. & Rose, John M. & Oppewal, Harmen & Lancsar, Emily, 2021. "Hypothetical bias in stated choice experiments: Part II. Conceptualisation of external validity, sources and explanations of bias and effectiveness of mitigation methods," Journal of choice modelling, Elsevier, vol. 41(C).
    4. Milad Haghani & Michiel C. J. Bliemer & John M. Rose & Harmen Oppewal & Emily Lancsar, 2021. "Hypothetical bias in stated choice experiments: Part I. Integrative synthesis of empirical evidence and conceptualisation of external validity," Papers 2102.02940, arXiv.org.
    5. Nicolas Jacquemet & Alexander James & Stéphane Luchini & Jason Shogren, 2011. "Social Psychology and Environmental Economics: A New Look at ex ante Corrections of Biased Preference Evaluation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 48(3), pages 413-433, March.
    6. Jacquemet, Nicolas & Joule, Robert-Vincent & Luchini, Stéphane & Shogren, Jason F., 2013. "Preference elicitation under oath," Journal of Environmental Economics and Management, Elsevier, vol. 65(1), pages 110-132.
    7. Haghani, Milad & Bliemer, Michiel C.J. & Rose, John M. & Oppewal, Harmen & Lancsar, Emily, 2021. "Hypothetical bias in stated choice experiments: Part I. Macro-scale analysis of literature and integrative synthesis of empirical evidence from applied economics, experimental psychology and neuroimag," Journal of choice modelling, Elsevier, vol. 41(C).
    8. Jie He & Jérôme Dupras & Thomas G. Poder, 2018. "Payment and Provision Consequentiality in Voluntary Contribution Mechanism: Single or Double “Knife-Edge” Evidence?," Cahiers de recherche 18-02, Departement d'économique de l'École de gestion à l'Université de Sherbrooke.
    9. Hensher, David A., 2010. "Hypothetical bias, choice experiments and willingness to pay," Transportation Research Part B: Methodological, Elsevier, vol. 44(6), pages 735-752, July.
    10. Haghani, Milad & Sarvi, Majid, 2018. "Hypothetical bias and decision-rule effect in modelling discrete directional choices," Transportation Research Part A: Policy and Practice, Elsevier, vol. 116(C), pages 361-388.
    11. Silva, Andres & Nayga, Rodolfo M., Jr. & Campbell, Benjamin L. & Park, John L., 2011. "Revisiting Cheap Talk with New Evidence from a Field Experiment," Journal of Agricultural and Resource Economics, Western Agricultural Economics Association, vol. 36(2), pages 1-12, August.
    12. Fifer, Simon & Rose, John M., 2016. "Can you ever be certain? Reducing hypothetical bias in stated choice experiments via respondent reported choice certaintyAuthor-Name: Beck, Matthew J," Transportation Research Part B: Methodological, Elsevier, vol. 89(C), pages 149-167.
    13. John K. Horowitz & Kenneth E. McConnell & James J. Murphy, 2013. "Behavioral foundations of environmental economics and valuation," Chapters, in: John A. List & Michael K. Price (ed.), Handbook on Experimental Economics and the Environment, chapter 4, pages 115-156, Edward Elgar Publishing.
    14. Kanya, Lucy & Saghera, Sabina & Lewin, Alex & Fox-Rushby, Julia, 2019. "The criterion validity of willingness to pay methods: a systematic review and meta-analysis of the evidence," LSE Research Online Documents on Economics 100741, London School of Economics and Political Science, LSE Library.
    15. Loomis, John B., 2014. "2013 WAEA Keynote Address: Strategies for Overcoming Hypothetical Bias in Stated Preference Surveys," Journal of Agricultural and Resource Economics, Western Agricultural Economics Association, vol. 39(1), pages 1-13, April.
    16. Franziska Voelckner, 2006. "An empirical comparison of methods for measuring consumers’ willingness to pay," Marketing Letters, Springer, vol. 17(2), pages 137-149, April.
    17. Roy Brouwer & Solomon Tarfasa, 2020. "Testing hypothetical bias in a framed field experiment," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 68(3), pages 343-357, September.
    18. Felix Munoz-Garcia & Sherzod B. Akhundjanov, 2014. "Firm Preferences for Environmental Policy: Industry Uniform or Firm Specific?," Working Papers 2014-8, School of Economic Sciences, Washington State University.
    19. Svenningsen, Lea S. & Jacobsen, Jette Bredahl, 2018. "Testing the effect of changes in elicitation format, payment vehicle and bid range on the hypothetical bias for moral goods," Journal of choice modelling, Elsevier, vol. 29(C), pages 17-32.
    20. Lusk Jayson L & Alexander Corinne & Rousu Matthew C., 2007. "Designing Experimental Auctions for Marketing Research: The Effect of Values, Distributions, and Mechanisms on Incentives for Truthful Bidding," Review of Marketing Science, De Gruyter, vol. 5(1), pages 1-32, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joamsc:v:48:y:2020:i:3:d:10.1007_s11747-019-00666-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.