IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/24514.html
   My bibliography  Save this paper

Using Massive Online Choice Experiments to Measure Changes in Well-being

Author

Listed:
  • Erik Brynjolfsson
  • Felix Eggers
  • Avinash Gannamaneni

Abstract

GDP and derived metrics (e.g., productivity) have been central to understanding economic progress and well-being. In principle, the change in consumer surplus (compensating expenditure) provides a superior, and more direct, measure of the change in well-being, especially for digital goods, but in practice, it has been difficult to measure. We explore the potential of massive online choice experiments to measure consumers’ willingness to accept compensation for losing access to various digital goods and thereby estimate the consumer surplus generated from these goods. We test the robustness of the approach and benchmark it against established methods, including incentive compatible choice experiments that require participants to give up Facebook for a certain period in exchange for compensation. The proposed choice experiments show convergent validity and are massively scalable. Our results indicate that digital goods have created large gains in well-being that are missed by conventional measures of GDP and productivity. By periodically querying a large, representative sample of goods and services, including those which are not priced in existing markets, changes in consumer surplus and other new measures of well-being derived from these online choice experiments have the potential for providing cost-effective supplements to existing national income and product accounts.

Suggested Citation

  • Erik Brynjolfsson & Felix Eggers & Avinash Gannamaneni, 2018. "Using Massive Online Choice Experiments to Measure Changes in Well-being," NBER Working Papers 24514, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:24514
    Note: EFG PR
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w24514.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Nils Wlömert & Felix Eggers, 2016. "Predicting new service adoption with conjoint analysis: external validity of BDM-based incentive-aligned and dual-response choice designs," Marketing Letters, Springer, vol. 27(1), pages 195-210, March.
    2. Greenstein, Shane & McDevitt, Ryan C., 2011. "The broadband bonus: Estimating broadband Internet's economic value," Telecommunications Policy, Elsevier, vol. 35(7), pages 617-632, August.
    3. James Murphy & P. Allen & Thomas Stevens & Darryl Weatherhead, 2005. "A Meta-analysis of Hypothetical Bias in Stated Preference Valuation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 30(3), pages 313-325, March.
    4. Charles I. Jones & Peter J. Klenow, 2016. "Beyond GDP? Welfare across Countries and Time," American Economic Review, American Economic Association, vol. 106(9), pages 2426-2457, September.
    5. Nevo, Aviv, 2001. "Measuring Market Power in the Ready-to-Eat Cereal Industry," Econometrica, Econometric Society, vol. 69(2), pages 307-342, March.
    6. Richard Carson & Robert Mitchell & Michael Hanemann & Raymond Kopp & Stanley Presser & Paul Ruud, 2003. "Contingent Valuation and Lost Passive Use: Damages from the Exxon Valdez Oil Spill," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 25(3), pages 257-286, July.
    7. Richard Schmalensee, 1978. "Entry Deterrence in the Ready-to-Eat Breakfast Cereal Industry," Bell Journal of Economics, The RAND Corporation, vol. 9(2), pages 305-327, Autumn.
    8. Jeremy Greenwood & Karen A. Kopecky, 2013. "Measuring The Welfare Gain From Personal Computers," Economic Inquiry, Western Economic Association International, vol. 51(1), pages 336-347, January.
    9. David M. Byrne & John G. Fernald & Marshall B. Reinsdorf, 2016. "Does the United States Have a Productivity Slowdown or a Measurement Problem?," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 47(1 (Spring), pages 109-182.
    10. Dale W. Jorgenson & J. Steven Landefeld & Paul Schreyer, 2014. "Measuring Economic Sustainability and Progress," NBER Books, National Bureau of Economic Research, Inc, number jorg12-1, March.
    11. Austan Goolsbee & Peter J. Klenow, 2006. "Valuing Consumer Products by the Time Spent Using Them: An Application to the Internet," American Economic Review, American Economic Association, vol. 96(2), pages 108-113, May.
    12. Richard T. Carson, 2012. "Contingent Valuation: A Practical Alternative When Prices Aren't Available," Journal of Economic Perspectives, American Economic Association, vol. 26(4), pages 27-42, Fall.
    13. Timothy C. Haab & Matthew G. Interis & Daniel R. Petrolia & John C. Whitehead, 2013. "From Hopeless to Curious? Thoughts on Hausman's "Dubious to Hopeless" Critique of Contingent Valuation," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 35(4), pages 593-612.
    14. Min Ding & Rajdeep Grewal & John Liechty, 2005. "Incentive-aligned conjoint analysis," Framed Field Experiments 00139, The Field Experiments Website.
    15. Winton Bates, 2009. "Gross national happiness," Asian-Pacific Economic Literature, Asia Pacific School of Economics and Government, The Australian National University, vol. 23(2), pages 1-16, November.
    16. Joel Waldfogel, 2012. "Copyright Protection, Technological Change, and the Quality of New Products: Evidence from Recorded Music since Napster," Journal of Law and Economics, University of Chicago Press, vol. 55(4), pages 715-740.
    17. John C. Whitehead, 2002. "Incentive Incompatibility and Starting-Point Bias in Iterative Valuation Questions," Land Economics, University of Wisconsin Press, vol. 78(2), pages 285-297.
    18. Richard Carson & Theodore Groves, 2007. "Incentive and informational properties of preference questions," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 37(1), pages 181-210, May.
    19. Dan Ariely & George Loewenstein & Drazen Prelec, 2003. ""Coherent Arbitrariness": Stable Demand Curves Without Stable Preferences," The Quarterly Journal of Economics, Oxford University Press, vol. 118(1), pages 73-106.
    20. Charles R. Hulten, 1978. "Growth Accounting with Intermediate Inputs," Review of Economic Studies, Oxford University Press, vol. 45(3), pages 511-518.
    21. Chad Syverson, 2017. "Challenges to Mismeasurement Explanations for the US Productivity Slowdown," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 165-186, Spring.
    22. Richard T. Carson & Theodore Groves & John A. List, 2014. "Consequentiality: A Theoretical and Experimental Exploration of a Single Binary Choice," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 1(1), pages 171-207.
    23. Erik Brynjolfsson & Daniel Rock & Chad Syverson, 2018. "Artificial Intelligence and the Modern Productivity Paradox: A Clash of Expectations and Statistics," NBER Chapters, in: The Economics of Artificial Intelligence: An Agenda, pages 23-57, National Bureau of Economic Research, Inc.
    24. Kahneman, Daniel & Knetsch, Jack L & Thaler, Richard H, 1990. "Experimental Tests of the Endowment Effect and the Coase Theorem," Journal of Political Economy, University of Chicago Press, vol. 98(6), pages 1325-1348, December.
    25. John List & Craig Gallet, 2001. "What Experimental Protocol Influence Disparities Between Actual and Hypothetical Stated Values?," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 20(3), pages 241-254, November.
    26. Daniel Kahneman & Jack L. Knetsch & Richard H. Thaler, 1991. "Anomalies: The Endowment Effect, Loss Aversion, and Status Quo Bias," Journal of Economic Perspectives, American Economic Association, vol. 5(1), pages 193-206, Winter.
    27. Bishop, Richard C. & Heberlein, Thomas A., 1979. "Measuring Values Of Extramarket Goods: Are Indirect Measures Biased?," 1979 Annual Meeting, July 29-August 1, Pullman, Washington 277818, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    28. Peter A. Diamond & Jerry A. Hausman, 1994. "Contingent Valuation: Is Some Number Better than No Number?," Journal of Economic Perspectives, American Economic Association, vol. 8(4), pages 45-64, Fall.
    29. Michael Spence & Bruce Owen, 1977. "Television Programming, Monopolistic Competition, and Welfare," The Quarterly Journal of Economics, Oxford University Press, vol. 91(1), pages 103-126.
    30. Kuznets, Simon, 1973. "Modern Economic Growth: Findings and Reflections," American Economic Review, American Economic Association, vol. 63(3), pages 247-258, June.
    31. Jerry Hausman, 2012. "Contingent Valuation: From Dubious to Hopeless," Journal of Economic Perspectives, American Economic Association, vol. 26(4), pages 43-56, Fall.
    32. Dale W. Jorgenson & J. Steven Landefeld & Paul Schreyer, 2014. "Introduction to "Measuring Economic Sustainability and Progress"," NBER Chapters, in: Measuring Economic Sustainability and Progress, pages 1-16, National Bureau of Economic Research, Inc.
    33. Richard T. Carson & Miko_aj Czajkowski, 2014. "The discrete choice experiment approach to environmental contingent valuation," Chapters, in: Stephane Hess & Andrew Daly (ed.), Handbook of Choice Modelling, chapter 9, pages 202-235, Edward Elgar Publishing.
    34. Hanemann, W Michael, 1991. "Willingness to Pay and Willingness to Accept: How Much Can They Differ?," American Economic Review, American Economic Association, vol. 81(3), pages 635-647, June.
    35. Richard C. Bishop & Thomas A. Heberlein, 1979. "Measuring Values of Extramarket Goods: Are Indirect Measures Biased?," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 61(5), pages 926-930.
    36. Peter Cohen & Robert Hahn & Jonathan Hall & Steven Levitt & Robert Metcalfe, 2016. "Using Big Data to Estimate Consumer Surplus: The Case of Uber," NBER Working Papers 22627, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Helga Fehr-Duda & Robin Schimmelpfennig, 2018. "Wider die Zahlengläubigkeit: Sind Befragungsergebnisse eine gute Grundlage für wirtschaftspolitische Entscheidungen?," ECON - Working Papers 297, Department of Economics - University of Zurich, revised Dec 2018.
    2. Amoah, Anthony & Ferrini, Silvia & Schaafsma, Marije, 2019. "Electricity outages in Ghana: Are contingent valuation estimates valid?," Energy Policy, Elsevier, vol. 135(C).
    3. Richard C. Bishop & Kevin J. Boyle, 2019. "Reliability and Validity in Nonmarket Valuation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 72(2), pages 559-582, February.
    4. John K. Horowitz & Kenneth E. McConnell & James J. Murphy, 2013. "Behavioral foundations of environmental economics and valuation," Chapters, in: John A. List & Michael K. Price (ed.), Handbook on Experimental Economics and the Environment, chapter 4, pages 115-156, Edward Elgar Publishing.
    5. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    6. Richard T. Carson & Miko_aj Czajkowski, 2014. "The discrete choice experiment approach to environmental contingent valuation," Chapters, in: Stephane Hess & Andrew Daly (ed.), Handbook of Choice Modelling, chapter 9, pages 202-235, Edward Elgar Publishing.
    7. Frondel, Manuel & Sommer, Stephan & Tomberg, Lukas, 2019. "WTA-WTP disparity: The role of perceived realism of the valuation setting," Ruhr Economic Papers 832, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    8. Milad Haghani & Michiel C. J. Bliemer & John M. Rose & Harmen Oppewal & Emily Lancsar, 2021. "Hypothetical bias in stated choice experiments: Part I. Integrative synthesis of empirical evidence and conceptualisation of external validity," Papers 2102.02940, arXiv.org.
    9. Svensson, Mikael, 2006. "The Value of a Statistical Life in Sweden Estimates from Two Studies using the "Certainty Approach" Calibration," Working Papers 2006:6, Örebro University, School of Business, revised 12 May 2009.
    10. Sawe, Nik, 2017. "Using neuroeconomics to understand environmental valuation," Ecological Economics, Elsevier, vol. 135(C), pages 1-9.
    11. Wainger, Lisa A. & Helcoski, Ryan & Farge, Kevin W. & Espinola, Brandy A. & Green, Gary T., 2018. "Evidence of a Shared Value for Nature," Ecological Economics, Elsevier, vol. 154(C), pages 107-116.
    12. Chavez, Daniel E. & Palma, Marco A. & Nayga, Rodolfo M. & Mjelde, James W., 2020. "Product availability in discrete choice experiments with private goods," Journal of choice modelling, Elsevier, vol. 36(C).
    13. Ivehammar, Pernilla, 2014. "Valuing environmental quality in actual travel time savings – The Haningeleden road project in Stockholm," Research in Transportation Economics, Elsevier, vol. 48(C), pages 349-356.
    14. Tonin, Stefania, 2018. "Citizens’ perspectives on marine protected areas as a governance strategy to effectively preserve marine ecosystem services and biodiversity," Ecosystem Services, Elsevier, vol. 34(PB), pages 189-200.
    15. Kees Vringer & Eline van der Heijden & Daan van Soest & Herman Vollebergh & Frank Dietz, 2017. "Sustainable Consumption Dilemmas," Sustainability, MDPI, Open Access Journal, vol. 9(6), pages 1-21, June.
    16. Bishop, Richard C., 2018. "Warm Glow, Good Feelings, and Contingent Valuation," Journal of Agricultural and Resource Economics, Western Agricultural Economics Association, vol. 43(3), September.
    17. Brynjolfsson, Erik & Collis, Avinash & Diewert, W. Erwin & Eggers, Felix & Fox, Kevin J., 2019. "GDP-B: Accounting for the Value of New and Free Goods in the Digital Economy," OSF Preprints sptfu, Center for Open Science.
    18. Banzhaf, H. Spencer, 2016. "Constructing markets: environmental economics and the contingent valuation controversy," MPRA Paper 78814, University Library of Munich, Germany.
    19. Bhattacharyya, Aditi & Kutlu, Levent & Sickles, Robin C., 2018. "Pricing Inputs and Outputs: Market prices versus shadow prices, market power, and welfare analysis," Working Papers 18-009, Rice University, Department of Economics.
    20. Baker, Rick & Ruting, Brad, 2014. "Environmental Policy Analysis: A Guide to Non‑Market Valuation," 2014 Conference (58th), February 4-7, 2014, Port Macquarie, Australia 165810, Australian Agricultural and Resource Economics Society.

    More about this item

    JEL classification:

    • E01 - Macroeconomics and Monetary Economics - - General - - - Measurement and Data on National Income and Product Accounts and Wealth; Environmental Accounts
    • E16 - Macroeconomics and Monetary Economics - - General Aggregative Models - - - Social Accounting Matrix
    • O1 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development
    • O4 - Economic Development, Innovation, Technological Change, and Growth - - Economic Growth and Aggregate Productivity

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:24514. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (). General contact details of provider: http://edirc.repec.org/data/nberrus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.