IDEAS home Printed from https://ideas.repec.org/p/hbs/wpaper/13-042.html
   My bibliography  Save this paper

Aggregation of Consumer Ratings: An Application to Yelp.com

Author

Listed:
  • Weijia Dai

    (University of Maryland)

  • Ginger Jin

    (University of Maryland)

  • Jungmin Lee

    (Sogang University)

  • Michael Luca

    (Harvard Business School, Negotiation, Organizations & Markets unit)

Abstract

Because consumer reviews leverage the wisdom of the crowd, the way in which they are aggregated is a central decision faced by platforms. We explore this "rating aggregation problem" and offer a structural approach to solving it, allowing for (1) reviewers to vary in stringency and accuracy, (2) reviewers to be influenced by existing reviews, and (3) product quality to change over time. Applying this to restaurant reviews from Yelp.com, we construct an adjusted average rating and show that even a simple algorithm can lead to large information efficiency gains relative to the arithmetic average.

Suggested Citation

  • Weijia Dai & Ginger Jin & Jungmin Lee & Michael Luca, 2012. "Aggregation of Consumer Ratings: An Application to Yelp.com," Harvard Business School Working Papers 13-042, Harvard Business School, revised Nov 2017.
    • Handle: RePEc:hbs:wpaper:13-042
    as

    Download full text from publisher

    File URL: http://www.hbs.edu/faculty/pages/download.aspx?name=13-042.pdf
    File Function: Revised version, 2017
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Chunhua Wu & Hai Che & Tat Y. Chan & Xianghua Lu, 2015. "The Economic Value of Online Reviews," Marketing Science, INFORMS, vol. 34(5), pages 739-754, September.
    2. Dina Mayzlin & Yaniv Dover & Judith Chevalier, 2014. "Promotional Reviews: An Empirical Investigation of Online Review Manipulation," American Economic Review, American Economic Association, vol. 104(8), pages 2421-2455, August.
    3. Dan Ariely & Anat Bracha & Stephan Meier, 2009. "Doing Good or Doing Well? Image Motivation and Monetary Incentives in Behaving Prosocially," American Economic Review, American Economic Association, vol. 99(1), pages 544-555, March.
    4. Jean Tirole & Roland Bénabou, 2006. "Incentives and Prosocial Behavior," American Economic Review, American Economic Association, vol. 96(5), pages 1652-1678, December.
    5. Jonathan E. Alevy & Michael S. Haigh & John A. List, 2007. "Information Cascades: Evidence from a Field Experiment with Financial Market Professionals," Journal of Finance, American Finance Association, vol. 62(1), pages 151-180, February.
    6. Nolan Miller & Paul Resnick & Richard Zeckhauser, 2005. "Eliciting Informative Feedback: The Peer-Prediction Method," Management Science, INFORMS, vol. 51(9), pages 1359-1373, September.
    7. Chrysanthos Dellarocas, 2006. "Strategic Manipulation of Internet Opinion Forums: Implications for Consumers and Firms," Management Science, INFORMS, vol. 52(10), pages 1577-1593, October.
    8. George A. Akerlof, 1980. "A Theory of Social Custom, of which Unemployment may be One Consequence," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 94(4), pages 749-775.
    9. Yan Chen & F. Maxwell Harper & Joseph Konstan & Sherry Xin Li, 2010. "Social Comparisons and Contributions to Online Communities: A Field Experiment on MovieLens," American Economic Review, American Economic Association, vol. 100(4), pages 1358-1398, September.
    10. Nikolay Archak & Anindya Ghose & Panagiotis G. Ipeirotis, 2007. "Deriving the Pricing Power of Product Features by Mining Consumer Reviews," Working Papers 07-36, NET Institute.
    11. David Godes & Dina Mayzlin, 2004. "Using Online Conversations to Study Word-of-Mouth Communication," Marketing Science, INFORMS, vol. 23(4), pages 545-560, June.
    12. Wendy W. Moe & David A. Schweidel, 2012. "Online Product Opinions: Incidence, Evaluation, and Evolution," Marketing Science, INFORMS, vol. 31(3), pages 372-386, May.
    13. David Godes & Dina Mayzlin, 2009. "Firm-Created Word-of-Mouth Communication: Evidence from a Field Test," Marketing Science, INFORMS, vol. 28(4), pages 721-739, 07-08.
    14. Nikolay Archak & Anindya Ghose & Panagiotis G. Ipeirotis, 2011. "Deriving the Pricing Power of Product Features by Mining Consumer Reviews," Management Science, INFORMS, vol. 57(8), pages 1485-1509, August.
    15. Pope, Devin G., 2009. "Reacting to rankings: Evidence from "America's Best Hospitals"," Journal of Health Economics, Elsevier, vol. 28(6), pages 1154-1165, December.
    16. Erik Eyster & Matthew Rabin, 2010. "Naïve Herding in Rich-Information Settings," American Economic Journal: Microeconomics, American Economic Association, vol. 2(4), pages 221-243, November.
    17. Jennifer Brown & Tanjim Hossain & John Morgan, 2010. "Shrouded Attributes and Information Suppression: Evidence from the Field," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(2), pages 859-876.
    18. Bikhchandani, Sushil & Hirshleifer, David & Welch, Ivo, 1992. "A Theory of Fads, Fashion, Custom, and Cultural Change in Informational Cascades," Journal of Political Economy, University of Chicago Press, vol. 100(5), pages 992-1026, October.
    19. Dina Mayzlin, 2006. "Promotional Chat on the Internet," Marketing Science, INFORMS, vol. 25(2), pages 155-163, 03-04.
    20. Anindya Ghose & Panagiotis G. Ipeirotis & Beibei Li, 2012. "Designing Ranking Systems for Hotels on Travel Search Engines by Mining User-Generated and Crowdsourced Content," Marketing Science, INFORMS, vol. 31(3), pages 493-520, May.
    21. Wang Zhongmin, 2010. "Anonymity, Social Image, and the Competition for Volunteers: A Case Study of the Online Market for Reviews," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 10(1), pages 1-35, May.
    22. Michael Luca & Jonathan Smith, 2013. "Salience in Quality Disclosure: Evidence from the U.S. News College Rankings," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 22(1), pages 58-77, March.
    23. Glazer, Jacob & McGuire, Thomas G. & Cao, Zhun & Zaslavsky, Alan, 2008. "Using global ratings of health plans to improve the quality of health care," Journal of Health Economics, Elsevier, vol. 27(5), pages 1182-1195, September.
    24. Chris Nosko & Steven Tadelis, 2015. "The Limits of Reputation in Platform Markets: An Empirical Analysis and Field Experiment," NBER Working Papers 20830, National Bureau of Economic Research, Inc.
    25. Xinxin Li & Lorin M. Hitt, 2008. "Self-Selection and Information Role of Online Product Reviews," Information Systems Research, INFORMS, vol. 19(4), pages 456-474, December.
    26. Abhijit V. Banerjee, 1992. "A Simple Model of Herd Behavior," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 107(3), pages 797-817.
    27. Michael Luca & Georgios Zervas, 2016. "Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud," Management Science, INFORMS, vol. 62(12), pages 3412-3427, December.
    28. David Godes & José C. Silva, 2012. "Sequential and Temporal Dynamics of Online Opinion," Marketing Science, INFORMS, vol. 31(3), pages 448-473, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dominik Gutt & Jürgen Neumann & Steffen Zimmermann & Dennis Kundisch & Jianqing Chen, 2018. "Design of Review Systems - A Strategic Instrument to shape Online Review Behavior and Economic Outcomes," Working Papers Dissertations 42, Paderborn University, Faculty of Business Administration and Economics.
    2. Greiff, Matthias & Paetzel, Fabian, 2020. "Information about average evaluations spurs cooperation: An experiment on noisy reputation systems," Journal of Economic Behavior & Organization, Elsevier, vol. 180(C), pages 334-356.
    3. Tao Lu & May Yuan & Chong (Alex) Wang & Xiaoquan (Michael) Zhang, 2022. "Histogram Distortion Bias in Consumer Choices," Management Science, INFORMS, vol. 68(12), pages 8963-8978, December.
    4. Pei-Yu Chen & Yili Hong & Ying Liu, 2018. "The Value of Multidimensional Rating Systems: Evidence from a Natural Experiment and Randomized Experiments," Management Science, INFORMS, vol. 64(10), pages 4629-4647, October.
    5. Christoph Schneider & Markus Weinmann & Peter N.C. Mohr & Jan vom Brocke, 2021. "When the Stars Shine Too Bright: The Influence of Multidimensional Ratings on Online Consumer Ratings," Management Science, INFORMS, vol. 67(6), pages 3871-3898, June.
    6. Warut Khern-am-nuai & Karthik Kannan & Hossein Ghasemkhani, 2018. "Extrinsic versus Intrinsic Rewards for Contributing Reviews in an Online Platform," Information Systems Research, INFORMS, vol. 29(4), pages 871-892, December.
    7. Aleksei Smirnov & Egor Starkov, 2022. "Bad News Turned Good: Reversal under Censorship," American Economic Journal: Microeconomics, American Economic Association, vol. 14(2), pages 506-560, May.
    8. Foster, Joshua, 2022. "How rating mechanisms shape user search, quality inference and engagement in online platforms: Experimental evidence," Journal of Business Research, Elsevier, vol. 142(C), pages 791-807.
    9. Chong (Alex) Wang & Xiaoquan (Michael) Zhang & Il-Horn Hann, 2018. "Socially Nudged: A Quasi-Experimental Study of Friends’ Social Influence in Online Product Ratings," Information Systems Research, INFORMS, vol. 29(3), pages 641-655, September.
    10. Khim-Yong Goh & Cheng-Suang Heng & Zhijie Lin, 2013. "Social Media Brand Community and Consumer Behavior: Quantifying the Relative Impact of User- and Marketer-Generated Content," Information Systems Research, INFORMS, vol. 24(1), pages 88-107, March.
    11. Michael Luca & Georgios Zervas, 2013. "Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud," Harvard Business School Working Papers 14-006, Harvard Business School, revised May 2015.
    12. Sungsik Park & Woochoel Shin & Jinhong Xie, 2021. "The Fateful First Consumer Review," Marketing Science, INFORMS, vol. 40(3), pages 481-507, May.
    13. Ana Babić Rosario & Kristine Valck & Francesca Sotgiu, 2020. "Conceptualizing the electronic word-of-mouth process: What we know and need to know about eWOM creation, exposure, and evaluation," Journal of the Academy of Marketing Science, Springer, vol. 48(3), pages 422-448, May.
    14. Anna Kerkhof & Johannes Münster, 2021. "Detecting Coverage Bias in User-Generated Content," CESifo Working Paper Series 8844, CESifo.
    15. Young Kwark & Gene Moo Lee & Paul A. Pavlou & Liangfei Qiu, 2021. "On the Spillover Effects of Online Product Reviews on Purchases: Evidence from Clickstream Data," Information Systems Research, INFORMS, vol. 32(3), pages 895-913, September.
    16. Yi Zhao & Sha Yang & Vishal Narayan & Ying Zhao, 2013. "Modeling Consumer Learning from Online Product Reviews," Marketing Science, INFORMS, vol. 32(1), pages 153-169, May.
    17. Mingwen Yang & Zhiqiang (Eric) Zheng & Vijay Mookerjee, 2019. "Prescribing Response Strategies to Manage Customer Opinions: A Stochastic Differential Equation Approach," Information Systems Research, INFORMS, vol. 30(2), pages 351-374, June.
    18. S. Cicognani & P. Figini & M. Magnani, 2016. "Social Influence Bias in Online Ratings: A Field Experiment," Working Papers wp1060, Dipartimento Scienze Economiche, Universita' di Bologna.
    19. David Godes & José C. Silva, 2012. "Sequential and Temporal Dynamics of Online Opinion," Marketing Science, INFORMS, vol. 31(3), pages 448-473, May.
    20. King, Robert Allen & Racherla, Pradeep & Bush, Victoria D., 2014. "What We Know and Don't Know About Online Word-of-Mouth: A Review and Synthesis of the Literature," Journal of Interactive Marketing, Elsevier, vol. 28(3), pages 167-183.

    More about this item

    Keywords

    user generated content; crowdsourcing; e-commerce; learning; Yelp.;
    All these keywords.

    JEL classification:

    • D8 - Microeconomics - - Information, Knowledge, and Uncertainty
    • L15 - Industrial Organization - - Market Structure, Firm Strategy, and Market Performance - - - Information and Product Quality
    • L86 - Industrial Organization - - Industry Studies: Services - - - Information and Internet Services; Computer Software

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hbs:wpaper:13-042. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: HBS (email available below). General contact details of provider: https://edirc.repec.org/data/harbsus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.