IDEAS home Printed from https://ideas.repec.org/a/nas/journl/v116y2019p7250-7255.html
   My bibliography  Save this article

Using massive online choice experiments to measure changes in well-being

Author

Listed:
  • Erik Brynjolfsson

    (Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA 02142; National Bureau of Economic Research, Cambridge, MA 02138)

  • Avinash Collis

    (Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA 02142)

  • Felix Eggers

    (Faculty of Economics and Business, University of Groningen, 9747 AE Groningen, The Netherlands)

Abstract

Gross domestic product (GDP) and derived metrics such as productivity have been central to our understanding of economic progress and well-being. In principle, changes in consumer surplus provide a superior, and more direct, measure of changes in well-being, especially for digital goods. In practice, these alternatives have been difficult to quantify. We explore the potential of massive online choice experiments to measure consumer surplus. We illustrate this technique via several empirical examples which quantify the valuations of popular digital goods and categories. Our examples include incentive-compatible discrete-choice experiments where online and laboratory participants receive monetary compensation if and only if they forgo goods for predefined periods. For example, the median user needed a compensation of about $48 to forgo Facebook for 1 mo. Our overall analyses reveal that digital goods have created large gains in well-being that are not reflected in conventional measures of GDP and productivity. By periodically querying a large, representative sample of goods and services, including those which are not priced in existing markets, changes in consumer surplus and other new measures of well-being derived from these online choice experiments have the potential for providing cost-effective supplements to the existing national income and product accounts.

Suggested Citation

  • Erik Brynjolfsson & Avinash Collis & Felix Eggers, 2019. "Using massive online choice experiments to measure changes in well-being," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 116(15), pages 7250-7255, April.
  • Handle: RePEc:nas:journl:v:116:y:2019:p:7250-7255
    as

    Download full text from publisher

    File URL: http://www.pnas.org/content/116/15/7250.full
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Nils Wlömert & Felix Eggers, 2016. "Predicting new service adoption with conjoint analysis: external validity of BDM-based incentive-aligned and dual-response choice designs," Marketing Letters, Springer, vol. 27(1), pages 195-210, March.
    2. Greenstein, Shane & McDevitt, Ryan C., 2011. "The broadband bonus: Estimating broadband Internet's economic value," Telecommunications Policy, Elsevier, vol. 35(7), pages 617-632, August.
    3. James Murphy & P. Allen & Thomas Stevens & Darryl Weatherhead, 2005. "A Meta-analysis of Hypothetical Bias in Stated Preference Valuation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 30(3), pages 313-325, March.
    4. Charles I. Jones & Peter J. Klenow, 2016. "Beyond GDP? Welfare across Countries and Time," American Economic Review, American Economic Association, vol. 106(9), pages 2426-2457, September.
    5. Nevo, Aviv, 2001. "Measuring Market Power in the Ready-to-Eat Cereal Industry," Econometrica, Econometric Society, vol. 69(2), pages 307-342, March.
    6. Richard Carson & Robert Mitchell & Michael Hanemann & Raymond Kopp & Stanley Presser & Paul Ruud, 2003. "Contingent Valuation and Lost Passive Use: Damages from the Exxon Valdez Oil Spill," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 25(3), pages 257-286, July.
    7. Richard Schmalensee, 1978. "Entry Deterrence in the Ready-to-Eat Breakfast Cereal Industry," Bell Journal of Economics, The RAND Corporation, vol. 9(2), pages 305-327, Autumn.
    8. Jeremy Greenwood & Karen A. Kopecky, 2013. "Measuring The Welfare Gain From Personal Computers," Economic Inquiry, Western Economic Association International, vol. 51(1), pages 336-347, January.
    9. David M. Byrne & John G. Fernald & Marshall B. Reinsdorf, 2016. "Does the United States Have a Productivity Slowdown or a Measurement Problem?," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 47(1 (Spring), pages 109-182.
    10. Dale W. Jorgenson & J. Steven Landefeld & Paul Schreyer, 2014. "Measuring Economic Sustainability and Progress," NBER Books, National Bureau of Economic Research, Inc, number jorg12-1, May.
    11. Austan Goolsbee & Peter J. Klenow, 2006. "Valuing Consumer Products by the Time Spent Using Them: An Application to the Internet," American Economic Review, American Economic Association, vol. 96(2), pages 108-113, May.
    12. Richard T. Carson, 2012. "Contingent Valuation: A Practical Alternative When Prices Aren't Available," Journal of Economic Perspectives, American Economic Association, vol. 26(4), pages 27-42, Fall.
    13. Timothy C. Haab & Matthew G. Interis & Daniel R. Petrolia & John C. Whitehead, 2013. "From Hopeless to Curious? Thoughts on Hausman's "Dubious to Hopeless" Critique of Contingent Valuation," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 35(4), pages 593-612.
    14. Min Ding & Rajdeep Grewal & John Liechty, 2005. "Incentive-aligned conjoint analysis," Framed Field Experiments 00139, The Field Experiments Website.
    15. Winton Bates, 2009. "Gross national happiness," Asian-Pacific Economic Literature, Asia Pacific School of Economics and Government, The Australian National University, vol. 23(2), pages 1-16, November.
    16. Joel Waldfogel, 2012. "Copyright Protection, Technological Change, and the Quality of New Products: Evidence from Recorded Music since Napster," Journal of Law and Economics, University of Chicago Press, vol. 55(4), pages 715-740.
    17. John C. Whitehead, 2002. "Incentive Incompatibility and Starting-Point Bias in Iterative Valuation Questions," Land Economics, University of Wisconsin Press, vol. 78(2), pages 285-297.
    18. Richard Carson & Theodore Groves, 2007. "Incentive and informational properties of preference questions," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 37(1), pages 181-210, May.
    19. Dan Ariely & George Loewenstein & Drazen Prelec, 2003. ""Coherent Arbitrariness": Stable Demand Curves Without Stable Preferences," The Quarterly Journal of Economics, Oxford University Press, vol. 118(1), pages 73-106.
    20. Charles R. Hulten, 1978. "Growth Accounting with Intermediate Inputs," Review of Economic Studies, Oxford University Press, vol. 45(3), pages 511-518.
    21. Chad Syverson, 2017. "Challenges to Mismeasurement Explanations for the US Productivity Slowdown," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 165-186, Spring.
    22. Richard T. Carson & Theodore Groves & John A. List, 2014. "Consequentiality: A Theoretical and Experimental Exploration of a Single Binary Choice," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 1(1), pages 171-207.
    23. Erik Brynjolfsson & Daniel Rock & Chad Syverson, 2018. "Artificial Intelligence and the Modern Productivity Paradox: A Clash of Expectations and Statistics," NBER Chapters, in: The Economics of Artificial Intelligence: An Agenda, pages 23-57, National Bureau of Economic Research, Inc.
    24. Kahneman, Daniel & Knetsch, Jack L & Thaler, Richard H, 1990. "Experimental Tests of the Endowment Effect and the Coase Theorem," Journal of Political Economy, University of Chicago Press, vol. 98(6), pages 1325-1348, December.
    25. John List & Craig Gallet, 2001. "What Experimental Protocol Influence Disparities Between Actual and Hypothetical Stated Values?," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 20(3), pages 241-254, November.
    26. Daniel Kahneman & Jack L. Knetsch & Richard H. Thaler, 1991. "Anomalies: The Endowment Effect, Loss Aversion, and Status Quo Bias," Journal of Economic Perspectives, American Economic Association, vol. 5(1), pages 193-206, Winter.
    27. Bishop, Richard C. & Heberlein, Thomas A., 1979. "Measuring Values Of Extramarket Goods: Are Indirect Measures Biased?," 1979 Annual Meeting, July 29-August 1, Pullman, Washington 277818, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    28. Peter A. Diamond & Jerry A. Hausman, 1994. "Contingent Valuation: Is Some Number Better than No Number?," Journal of Economic Perspectives, American Economic Association, vol. 8(4), pages 45-64, Fall.
    29. Michael Spence & Bruce Owen, 1977. "Television Programming, Monopolistic Competition, and Welfare," The Quarterly Journal of Economics, Oxford University Press, vol. 91(1), pages 103-126.
    30. Kuznets, Simon, 1973. "Modern Economic Growth: Findings and Reflections," American Economic Review, American Economic Association, vol. 63(3), pages 247-258, June.
    31. Jerry Hausman, 2012. "Contingent Valuation: From Dubious to Hopeless," Journal of Economic Perspectives, American Economic Association, vol. 26(4), pages 43-56, Fall.
    32. Dale W. Jorgenson & J. Steven Landefeld & Paul Schreyer, 2014. "Introduction to "Measuring Economic Sustainability and Progress"," NBER Chapters, in: Measuring Economic Sustainability and Progress, pages 1-16, National Bureau of Economic Research, Inc.
    33. Richard T. Carson & Miko_aj Czajkowski, 2014. "The discrete choice experiment approach to environmental contingent valuation," Chapters, in: Stephane Hess & Andrew Daly (ed.), Handbook of Choice Modelling, chapter 9, pages 202-235, Edward Elgar Publishing.
    34. Hanemann, W Michael, 1991. "Willingness to Pay and Willingness to Accept: How Much Can They Differ?," American Economic Review, American Economic Association, vol. 81(3), pages 635-647, June.
    35. Richard C. Bishop & Thomas A. Heberlein, 1979. "Measuring Values of Extramarket Goods: Are Indirect Measures Biased?," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 61(5), pages 926-930.
    36. Peter Cohen & Robert Hahn & Jonathan Hall & Steven Levitt & Robert Metcalfe, 2016. "Using Big Data to Estimate Consumer Surplus: The Case of Uber," NBER Working Papers 22627, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Helga Fehr-Duda & Robin Schimmelpfennig, 2018. "Wider die Zahlengläubigkeit: Sind Befragungsergebnisse eine gute Grundlage für wirtschaftspolitische Entscheidungen?," ECON - Working Papers 297, Department of Economics - University of Zurich, revised Dec 2018.
    2. Amoah, Anthony & Ferrini, Silvia & Schaafsma, Marije, 2019. "Electricity outages in Ghana: Are contingent valuation estimates valid?," Energy Policy, Elsevier, vol. 135(C).
    3. John K. Horowitz & Kenneth E. McConnell & James J. Murphy, 2013. "Behavioral foundations of environmental economics and valuation," Chapters, in: John A. List & Michael K. Price (ed.), Handbook on Experimental Economics and the Environment, chapter 4, pages 115-156, Edward Elgar Publishing.
    4. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    5. Richard T. Carson & Miko_aj Czajkowski, 2014. "The discrete choice experiment approach to environmental contingent valuation," Chapters, in: Stephane Hess & Andrew Daly (ed.), Handbook of Choice Modelling, chapter 9, pages 202-235, Edward Elgar Publishing.
    6. Richard C. Bishop & Kevin J. Boyle, 2019. "Reliability and Validity in Nonmarket Valuation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 72(2), pages 559-582, February.
    7. Frondel, Manuel & Sommer, Stephan & Tomberg, Lukas, 2019. "WTA-WTP disparity: The role of perceived realism of the valuation setting," Ruhr Economic Papers 832, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    8. Lopez-Becerra, E.I. & Alcon, F., 2021. "Social desirability bias in the environmental economic valuation: An inferred valuation approach," Ecological Economics, Elsevier, vol. 184(C).
    9. Milad Haghani & Michiel C. J. Bliemer & John M. Rose & Harmen Oppewal & Emily Lancsar, 2021. "Hypothetical bias in stated choice experiments: Part I. Integrative synthesis of empirical evidence and conceptualisation of external validity," Papers 2102.02940, arXiv.org.
    10. Svensson, Mikael, 2006. "The Value of a Statistical Life in Sweden Estimates from Two Studies using the "Certainty Approach" Calibration," Working Papers 2006:6, Örebro University, School of Business, revised 12 May 2009.
    11. Sawe, Nik, 2017. "Using neuroeconomics to understand environmental valuation," Ecological Economics, Elsevier, vol. 135(C), pages 1-9.
    12. Wainger, Lisa A. & Helcoski, Ryan & Farge, Kevin W. & Espinola, Brandy A. & Green, Gary T., 2018. "Evidence of a Shared Value for Nature," Ecological Economics, Elsevier, vol. 154(C), pages 107-116.
    13. Chavez, Daniel E. & Palma, Marco A. & Nayga, Rodolfo M. & Mjelde, James W., 2020. "Product availability in discrete choice experiments with private goods," Journal of choice modelling, Elsevier, vol. 36(C).
    14. Ivehammar, Pernilla, 2014. "Valuing environmental quality in actual travel time savings – The Haningeleden road project in Stockholm," Research in Transportation Economics, Elsevier, vol. 48(C), pages 349-356.
    15. Tonin, Stefania, 2018. "Citizens’ perspectives on marine protected areas as a governance strategy to effectively preserve marine ecosystem services and biodiversity," Ecosystem Services, Elsevier, vol. 34(PB), pages 189-200.
    16. Kees Vringer & Eline van der Heijden & Daan van Soest & Herman Vollebergh & Frank Dietz, 2017. "Sustainable Consumption Dilemmas," Sustainability, MDPI, Open Access Journal, vol. 9(6), pages 1-21, June.
    17. Bishop, Richard C., 2018. "Warm Glow, Good Feelings, and Contingent Valuation," Journal of Agricultural and Resource Economics, Western Agricultural Economics Association, vol. 43(3), September.
    18. Brynjolfsson, Erik & Collis, Avinash & Diewert, W. Erwin & Eggers, Felix & Fox, Kevin J., 2019. "GDP-B: Accounting for the Value of New and Free Goods in the Digital Economy," OSF Preprints sptfu, Center for Open Science.
    19. Banzhaf, H. Spencer, 2016. "Constructing markets: environmental economics and the contingent valuation controversy," MPRA Paper 78814, University Library of Munich, Germany.
    20. Bhattacharyya, Aditi & Kutlu, Levent & Sickles, Robin C., 2018. "Pricing Inputs and Outputs: Market prices versus shadow prices, market power, and welfare analysis," Working Papers 18-009, Rice University, Department of Economics.

    More about this item

    Keywords

    consumer surplus; digital goods; free goods; GDP choice experiments;
    All these keywords.

    JEL classification:

    • E01 - Macroeconomics and Monetary Economics - - General - - - Measurement and Data on National Income and Product Accounts and Wealth; Environmental Accounts
    • E16 - Macroeconomics and Monetary Economics - - General Aggregative Models - - - Social Accounting Matrix
    • O1 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development
    • O4 - Economic Development, Innovation, Technological Change, and Growth - - Economic Growth and Aggregate Productivity

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nas:journl:v:116:y:2019:p:7250-7255. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.pnas.org/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Kat Rodenhizer (email available below). General contact details of provider: http://www.pnas.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.