IDEAS home Printed from https://ideas.repec.org/a/eee/socmed/v232y2019icp238-261.html
   My bibliography  Save this article

The criterion validity of willingness to pay methods: A systematic review and meta-analysis of the evidence

Author

Listed:
  • Kanya, Lucy
  • Sanghera, Sabina
  • Lewin, Alex
  • Fox-Rushby, Julia

Abstract

The contingent valuation (CV) method is used to estimate the willingness to pay (WTP) for services and products to inform cost benefit analyses (CBA). A long-standing criticism that stated WTP estimates may be poor indicators of actual WTP, calls into question their validity and the use of such estimates for welfare evaluation, especially in the health sector. Available evidence on the validity of CV studies so far is inconclusive. We systematically reviewed the literature to (1) synthesize the evidence on the criterion validity of WTP/willingness to accept (WTA), (2) undertake a meta-analysis, pooling evidence on the extent of variation between stated and actual WTP values and, (3) explore the reasons for the variation.

Suggested Citation

  • Kanya, Lucy & Sanghera, Sabina & Lewin, Alex & Fox-Rushby, Julia, 2019. "The criterion validity of willingness to pay methods: A systematic review and meta-analysis of the evidence," Social Science & Medicine, Elsevier, vol. 232(C), pages 238-261.
  • Handle: RePEc:eee:socmed:v:232:y:2019:i:c:p:238-261
    DOI: 10.1016/j.socscimed.2019.04.015
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0277953619302151
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.socscimed.2019.04.015?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Onwujekwe, Obinna & Hanson, Kara & Fox-Rushby, Julia, 2005. "Do divergences between stated and actual willingness to pay signify the existence of bias in contingent valuation surveys?," Social Science & Medicine, Elsevier, vol. 60(3), pages 525-536, February.
    2. Laurent Muller & Bernard Ruffieux, 2011. "Do price-tags influence consumers’ willingness to pay? On the external validity of using auctions for measuring value," Experimental Economics, Springer;Economic Science Association, vol. 14(2), pages 181-202, May.
    3. Kalle Seip & Jon Strand, 1992. "Willingness to pay for environmental goods in Norway: A contingent valuation study with real payment," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 2(1), pages 91-106, January.
    4. Loomis, John & Bell, Paul & Cooney, Helen & Asmus, Cheryl, 2009. "A Comparison of Actual and Hypothetical Willingness to Pay of Parents and Non-Parents for Protecting Infant Health: The Case of Nitrates in Drinking Water," Journal of Agricultural and Applied Economics, Cambridge University Press, vol. 41(3), pages 697-712, December.
    5. List, John A. & Shogren, Jason F., 1998. "Calibration of the difference between actual and hypothetical valuations in a field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 37(2), pages 193-205, October.
    6. Edward Balistreri & Gary McClelland & Gregory Poe & William Schulze, 2001. "Can Hypothetical Questions Reveal True Values? A Laboratory Comparison of Dichotomous Choice and Open-Ended Contingent Values with Auction Values," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 18(3), pages 275-292, March.
    7. Vossler, Christian A. & Kerkvliet, Joe & Polasky, Stephen & Gainutdinova, Olesya, 2003. "Externally validating contingent valuation: an open-space survey and referendum in Corvallis, Oregon," Journal of Economic Behavior & Organization, Elsevier, vol. 51(2), pages 261-277, June.
    8. Magnus Johannesson & Bengt Liljas & Per-Olov Johansson, 1998. "An experimental comparison of dichotomous choice contingent valuation questions and real purchase decisions," Applied Economics, Taylor & Francis Journals, vol. 30(5), pages 643-647.
    9. Frykblom, Peter, 1997. "Hypothetical Question Modes and Real Willingness to Pay," Journal of Environmental Economics and Management, Elsevier, vol. 34(3), pages 275-287, November.
    10. Vossler, Christian A. & Kerkvliet, Joe, 2003. "A criterion validity test of the contingent valuation method: comparing hypothetical and actual voting behavior for a public referendum," Journal of Environmental Economics and Management, Elsevier, vol. 45(3), pages 631-649, May.
    11. John A. Fox & Jason F. Shogren & Dermot J. Hayes & James B. Kliebenstein, 1998. "CVM-X: Calibrating Contingent Values with Experimental Auction Markets," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 80(3), pages 455-465.
    12. Cummings, Ronald G & Elliott, Steven & Harrison, Glenn W & Murphy, James, 1997. "Are Hypothetical Referenda Incentive Compatible?," Journal of Political Economy, University of Chicago Press, vol. 105(3), pages 609-621, June.
    13. Helen R. Neill & Ronald G. Cummings & Philip T. Ganderton & Glenn W. Harrison & Thomas McGuckin, 1994. "Hypothetical Surveys and Real Economic Commitments," Land Economics, University of Wisconsin Press, vol. 70(2), pages 145-154.
    14. James Murphy & Thomas Stevens & Lava Yadav, 2010. "A Comparison of Induced Value and Home-Grown Value Experiments to Test for Hypothetical Bias in Contingent Valuation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 47(1), pages 111-123, September.
    15. List, John A. & Shogren, Jason F., 2002. "Calibration of Willingness-to-Accept," Journal of Environmental Economics and Management, Elsevier, vol. 43(2), pages 219-233, March.
    16. John Loomis & Thomas Brown & Beatrice Lucero & George Peterson, 1997. "Evaluating the Validity of the Dichotomous Choice Question Format in Contingent Valuation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 10(2), pages 109-123, September.
    17. Laurent Muller & Bernard Ruffieux, 2011. "Do price-tags influence consumers’ willingness to pay? On the external validity of using auctions for measuring value," Post-Print hal-01799824, HAL.
    18. Johnston, Robert J., 2006. "Is hypothetical bias universal? Validating contingent valuation responses using a binding public referendum," Journal of Environmental Economics and Management, Elsevier, vol. 52(1), pages 469-481, July.
    19. Cummings, Ronald G & Harrison, Glenn W & Rutstrom, E Elisabet, 1995. "Homegrown Values and Hypothetical Surveys: Is the Dichotomous Choice Approach Incentive-Compatible?," American Economic Review, American Economic Association, vol. 85(1), pages 260-266, March.
    20. Anabela Botelho & Ligia Costa Pinto, 2002. "Hypothetical, real, and predicted real willingness to pay in open-ended surveys: experimental results," Applied Economics Letters, Taylor & Francis Journals, vol. 9(15), pages 993-996.
    21. Spencer, Michael A. & Swallow, Stephen K. & Miller, Christopher J., 1998. "Valuing Water Quality Monitoring: A Contingent Valuation Experiment Involving Hypothetical and Real Payments," Agricultural and Resource Economics Review, Cambridge University Press, vol. 27(1), pages 28-42, April.
    22. Eva Camacho-Cuena & Aurora García-Gallego & Nikolaos Georgantzís & Gerardo Sabater-Grande, 2004. "An Experimental Validation of Hypothetical WTP for a Recyclable Product," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 27(3), pages 313-335, March.
    23. Christian A. Vossler & Robert G. Ethier & Gregory L. Poe & Michael P. Welsh, 2003. "Payment Certainty in Discrete Choice Contingent Valuation Responses: Results from a Field Validity Test," Southern Economic Journal, John Wiley & Sons, vol. 69(4), pages 886-902, April.
    24. Brian Byrnes & Clive Jones† & Sandra Goodman‡, 1999. "Contingent Valuation and Real Economic Commitments: Evidence from Electric Utility Green Pricing Programmes," Journal of Environmental Planning and Management, Taylor & Francis Journals, vol. 42(2), pages 149-166.
    25. Onwujekwe, Obinna, 2004. "Criterion and content validity of a novel structured haggling contingent valuation question format versus the bidding game and binary with follow-up format," Social Science & Medicine, Elsevier, vol. 58(3), pages 525-537, February.
    26. Pallab Mozumder & Robert P. Berrens, 2007. "Investigating hypothetical bias: induced-value tests of the referendum voting mechanism with uncertainty," Applied Economics Letters, Taylor & Francis Journals, vol. 14(10), pages 705-709.
    27. John A. List, 2001. "Do Explicit Warnings Eliminate the Hypothetical Bias in Elicitation Procedures? Evidence from Field Auctions for Sportscards," American Economic Review, American Economic Association, vol. 91(5), pages 1498-1507, December.
    28. Massimo Paradiso & Antonella Trisorio, 2001. "The effect of knowledge on the disparity between hypothetical and real willingness to pay," Applied Economics, Taylor & Francis Journals, vol. 33(11), pages 1359-1364.
    29. Karen Blumenschein & Glenn C. Blomquist & Magnus Johannesson & Nancy Horn & Patricia Freeman, 2008. "Eliciting Willingness to Pay Without Bias: Evidence from a Field Experiment," Economic Journal, Royal Economic Society, vol. 118(525), pages 114-137, January.
    30. K. G. Willis & N. A. Powe, 1998. "Contingent Valuation and Real Economic Commitments: A Private Good Experiment," Journal of Environmental Planning and Management, Taylor & Francis Journals, vol. 41(5), pages 611-619.
    31. Thomas C. Brown & Patricia A. Champ & Richard C. Bishop & Daniel W. McCollum, 1996. "Which Response Format Reveals the Truth about Donations to a Public Good?," Land Economics, University of Wisconsin Press, vol. 72(2), pages 152-166.
    32. Slothuus, Ulla & Larsen, Mette L. & Junker, Peter, 2002. "The contingent ranking method--a feasible and valid method when eliciting preferences for health care?," Social Science & Medicine, Elsevier, vol. 54(10), pages 1601-1609, May.
    33. Thomas Klose, 2003. "A utility‐theoretic model for QALYs and willingness to pay," Health Economics, John Wiley & Sons, Ltd., vol. 12(1), pages 17-31, January.
    34. John List & Craig Gallet, 2001. "What Experimental Protocol Influence Disparities Between Actual and Hypothetical Stated Values?," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 20(3), pages 241-254, November.
    35. Vossler, Christian A. & Watson, Sharon B., 2013. "Understanding the consequences of consequentiality: Testing the validity of stated preferences in the field," Journal of Economic Behavior & Organization, Elsevier, vol. 86(C), pages 137-147.
    36. Bishop, Richard C. & Heberlein, Thomas A., 1979. "Measuring Values Of Extramarket Goods: Are Indirect Measures Biased?," 1979 Annual Meeting, July 29-August 1, Pullman, Washington 277818, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    37. Richard T. Carson & Nicholas E. Flores & Kerry M. Martin & Jennifer L. Wright, 1996. "Contingent Valuation and Revealed Preference Methodologies: Comparing the Estimates for Quasi-Public Goods," Land Economics, University of Wisconsin Press, vol. 72(1), pages 80-99.
    38. Marcella Veronesi & Anna Alberini & Joseph Cooper, 2011. "Implications of Bid Design and Willingness-To-Pay Distribution for Starting Point Bias in Double-Bounded Dichotomous Choice Contingent Valuation Surveys," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 49(2), pages 199-215, June.
    39. J. Lon Carlson, 2000. "Hypothetical surveys versus real commitments: further evidence," Applied Economics Letters, Taylor & Francis Journals, vol. 7(7), pages 447-450.
    40. Obinna Onwujekwe, 2001. "Searching for a better willingness to pay elicitation method in rural Nigeria: the binary question with follow‐up method versus the bidding game technique," Health Economics, John Wiley & Sons, Ltd., vol. 10(2), pages 147-158, March.
    41. Stirling Bryan & Sue Jowett, 2010. "Hypothetical versus real preferences: results from an opportunistic field experiment," Health Economics, John Wiley & Sons, Ltd., vol. 19(12), pages 1502-1509, December.
    42. Magnus Johannesson & Bengt Liljas & Richard O'Conor, 1997. "Hypothetical versus real willingness to pay: some experimental results," Applied Economics Letters, Taylor & Francis Journals, vol. 4(3), pages 149-151.
    43. Champ, Patricia A. & Bishop, Richard C. & Brown, Thomas C. & McCollum, Daniel W., 1997. "Using Donation Mechanisms to Value Nonuse Benefits from Public Goods," Journal of Environmental Economics and Management, Elsevier, vol. 33(2), pages 151-162, June.
    44. Bratt, John H., 2010. "Predicting impact of price increases on demand for reproductive health services: Can it be done well?," Health Policy, Elsevier, vol. 95(2-3), pages 159-165, May.
    45. John Loomis & Thomas Brown & Beatrice Lucero & George Peterson, 1996. "Improving Validity Experiments of Contingent Valuation Methods: Results of Efforts to Reduce the Disparity of Hypothetical and Actual Willingness to Pay," Land Economics, University of Wisconsin Press, vol. 72(4), pages 450-461.
    46. Magnus Johannesson, 1997. "Some further experimental results on hypothetical versus real willingness to pay," Applied Economics Letters, Taylor & Francis Journals, vol. 4(8), pages 535-536.
    47. M. Trapero‐Bertran & H. Mistry & J. Shen & J. Fox‐Rushby, 2013. "A Systematic Review And Meta‐Analysis Of Willingness‐To‐Pay Values: The Case Of Malaria Control Interventions," Health Economics, John Wiley & Sons, Ltd., vol. 22(4), pages 428-450, April.
    48. Blumenschein, Karen & Johannesson, Magnus & Yokoyama, Krista K. & Freeman, Patricia R., 2001. "Hypothetical versus real willingness to pay in the health care sector: results from a field experiment," Journal of Health Economics, Elsevier, vol. 20(3), pages 441-457, May.
    49. Obinna Onwujekwe & Benjamin Uzochukwu, 2004. "Stated and actual altruistic willingness to pay for insecticide‐treated nets in Nigeria: validity of open‐ended and binary with follow‐up questions," Health Economics, John Wiley & Sons, Ltd., vol. 13(5), pages 477-492, May.
    50. Ian J. Bateman & Richard T. Carson & Brett Day & Michael Hanemann & Nick Hanley & Tannis Hett & Michael Jones-Lee & Graham Loomes, 2002. "Economic Valuation with Stated Preference Techniques," Books, Edward Elgar Publishing, number 2639.
    51. Kealy, Mary Jo & Montgomery, Mark & Dovidio, John F., 1990. "Reliability and predictive validity of contingent values: Does the nature of the good matter?," Journal of Environmental Economics and Management, Elsevier, vol. 19(3), pages 244-263, November.
    52. M. R. Bhatia & J. A. Fox-Rushby, 2003. "Validity of Willingness to Pay: hypothetical versus actual payment," Applied Economics Letters, Taylor & Francis Journals, vol. 10(12), pages 737-740.
    53. Brown, Kelly M. & Taylor, Laura O., 2000. "Do as you say, say as you do: evidence on gender differences in actual and stated contributions to public goods," Journal of Economic Behavior & Organization, Elsevier, vol. 43(1), pages 127-139, September.
    54. Harrison, Glenn W. & Rutström, E. Elisabet, 2008. "Experimental Evidence on the Existence of Hypothetical Bias in Value Elicitation Methods," Handbook of Experimental Economics Results, in: Charles R. Plott & Vernon L. Smith (ed.), Handbook of Experimental Economics Results, edition 1, volume 1, chapter 81, pages 752-767, Elsevier.
    55. Knut Veisten & Ståle Navrud, 2006. "Contingent valuation and actual payment for voluntarily provided passive-use values: Assessing the effect of an induced truth-telling mechanism and elicitation formats," Applied Economics, Taylor & Francis Journals, vol. 38(7), pages 735-756.
    56. Richard C. Bishop & Thomas A. Heberlein, 1979. "Measuring Values of Extramarket Goods: Are Indirect Measures Biased?," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 61(5), pages 926-930.
    57. Michael Getzner, 2000. "Hypothetical and Real Economic Commitments, and Social Status, in Valuing a Species Protection Programme," Journal of Environmental Planning and Management, Taylor & Francis Journals, vol. 43(4), pages 541-559.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Anna Alberini & Milan Ščasný, 2021. "On the validity of the estimates of the VSL from contingent valuation: Evidence from the Czech Republic," Journal of Risk and Uncertainty, Springer, vol. 62(1), pages 55-87, February.
    2. Loubière, Sandrine & Taylor, Owen & Tinland, Aurelie & Vargas-Moniz, Maria & O'Shaughnessy, Branagh & Bokszczanin, Anna & Kallmen, Hakan & Bernad, Roberto & Wolf, Judith & Santinello, Massimo & Loundo, 2020. "Europeans’ willingness to pay for ending homelessness: A contingent valuation study," Social Science & Medicine, Elsevier, vol. 247(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kanya, Lucy & Saghera, Sabina & Lewin, Alex & Fox-Rushby, Julia, 2019. "The criterion validity of willingness to pay methods: a systematic review and meta-analysis of the evidence," LSE Research Online Documents on Economics 100741, London School of Economics and Political Science, LSE Library.
    2. James Murphy & P. Allen & Thomas Stevens & Darryl Weatherhead, 2005. "A Meta-analysis of Hypothetical Bias in Stated Preference Valuation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 30(3), pages 313-325, March.
    3. Shogren, Jason F., 2006. "Experimental Methods and Valuation," Handbook of Environmental Economics, in: K. G. Mäler & J. R. Vincent (ed.), Handbook of Environmental Economics, edition 1, volume 2, chapter 19, pages 969-1027, Elsevier.
    4. Liljas, Bengt & Blumenschein, Karen, 2000. "On hypothetical bias and calibration in cost-benefit studies," Health Policy, Elsevier, vol. 52(1), pages 53-70, May.
    5. John K. Horowitz & Kenneth E. McConnell & James J. Murphy, 2013. "Behavioral foundations of environmental economics and valuation," Chapters, in: John A. List & Michael K. Price (ed.), Handbook on Experimental Economics and the Environment, chapter 4, pages 115-156, Edward Elgar Publishing.
    6. John List & Craig Gallet, 2001. "What Experimental Protocol Influence Disparities Between Actual and Hypothetical Stated Values?," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 20(3), pages 241-254, November.
    7. repec:ebl:ecbull:v:3:y:2004:i:6:p:1-13 is not listed on IDEAS
    8. Atozou, Baoubadi & Tamini, Lota D. & Bergeronm, Stephane & Doyon, Maurice, 2020. "Factors Explaining the Hypothetical Bias: How to Improve Models for Meta-Analyses," Journal of Agricultural and Resource Economics, Western Agricultural Economics Association, vol. 45(2), March.
    9. Jie He & Jérôme Dupras & Thomas G. Poder, 2018. "Payment and Provision Consequentiality in Voluntary Contribution Mechanism: Single or Double “Knife-Edge” Evidence?," Cahiers de recherche 18-02, Departement d'Economique de l'École de gestion à l'Université de Sherbrooke.
    10. List, John A. & Shogren, Jason F., 2002. "Calibration of Willingness-to-Accept," Journal of Environmental Economics and Management, Elsevier, vol. 43(2), pages 219-233, March.
    11. Karen Blumenschein & GlennC. Blomquist & Magnus Johannesson & Nancy Horn & Patricia Freeman, 2008. "Eliciting Willingness to Pay Without Bias: Evidence from a Field Experiment," Economic Journal, Royal Economic Society, vol. 118(525), pages 114-137, January.
    12. Joseph Little & Robert Berrens, 2004. "Explaining Disparities between Actual and Hypothetical Stated Values: Further Investigation Using Meta-Analysis," Economics Bulletin, AccessEcon, vol. 3(6), pages 1-13.
    13. Whitehead, John C. & Cherry, Todd L., 2007. "Willingness to pay for a Green Energy program: A comparison of ex-ante and ex-post hypothetical bias mitigation approaches," Resource and Energy Economics, Elsevier, vol. 29(4), pages 247-261, November.
    14. Catherine L. Kling & Daniel J. Phaneuf & Jinhua Zhao, 2012. "From Exxon to BP: Has Some Number Become Better Than No Number?," Journal of Economic Perspectives, American Economic Association, vol. 26(4), pages 3-26, Fall.
    15. Murphy, James J. & Stevens, Thomas H., 2004. "Contingent Valuation, Hypothetical Bias, and Experimental Economics," Agricultural and Resource Economics Review, Cambridge University Press, vol. 33(2), pages 182-192, October.
    16. Johnston, Robert J., 2006. "Is hypothetical bias universal? Validating contingent valuation responses using a binding public referendum," Journal of Environmental Economics and Management, Elsevier, vol. 52(1), pages 469-481, July.
    17. Loomis, John B., 2014. "2013 WAEA Keynote Address: Strategies for Overcoming Hypothetical Bias in Stated Preference Surveys," Journal of Agricultural and Resource Economics, Western Agricultural Economics Association, vol. 39(1), pages 1-13, April.
    18. Ana Bedate & Luis Herrero & José Sanz, 2009. "Economic valuation of a contemporary art museum: correction of hypothetical bias using a certainty question," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 33(3), pages 185-199, August.
    19. Richard T. Carson, 2011. "Contingent Valuation," Books, Edward Elgar Publishing, number 2489, September.
    20. Hofstetter, Reto & Miller, Klaus M. & Krohmer, Harley & Zhang, Z. John, 2021. "A de-biased direct question approach to measuring consumers' willingness to pay," International Journal of Research in Marketing, Elsevier, vol. 38(1), pages 70-84.
    21. Mark A. Andor & Manuel Frondel & Colin Vance, 2017. "Mitigating Hypothetical Bias: Evidence on the Effects of Correctives from a Large Field Study," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 68(3), pages 777-796, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:socmed:v:232:y:2019:i:c:p:238-261. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/315/description#description .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/315/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.