IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/18567.html
   My bibliography  Save this paper

Optimal Aggregation of Consumer Ratings: An Application to Yelp.com

Author

Listed:
  • Weijia Dai
  • Ginger Z. Jin
  • Jungmin Lee
  • Michael Luca

Abstract

Consumer review websites leverage the wisdom of the crowd, with each product being reviewed many times (some with more than 1,000 reviews). Because of this, the way in which information is aggregated is a central decision faced by consumer review websites. Given a set of reviews, what is the optimal way to construct an average rating? We offer a structural approach to answering this question, allowing for (1) reviewers to vary in stringency and accuracy, (2) reviewers to be influenced by existing reviews, and (3) product quality to change over time. Applying this approach to restaurant reviews from Yelp.com, we construct optimal ratings for all restaurants and compare them to the arithmetic averages displayed by Yelp. Depending on how we interpret the downward trend of reviews within a restaurant, we find 19.1-41.38% of the simple average ratings are more than 0.15 stars away from optimal ratings, and 5.33-19.1% are more than 0.25 stars away at the end of our sample period. Moreover, the deviation grows significantly as a restaurant accumulates reviews over time. This suggests that large gains could be made by implementing optimal ratings, especially as Yelp grows. Our algorithm can be flexibly applied to many different review settings.

Suggested Citation

  • Weijia Dai & Ginger Z. Jin & Jungmin Lee & Michael Luca, 2012. "Optimal Aggregation of Consumer Ratings: An Application to Yelp.com," NBER Working Papers 18567, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:18567 Note: IO
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w18567.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Dan Ariely & Anat Bracha & Stephan Meier, 2009. "Doing Good or Doing Well? Image Motivation and Monetary Incentives in Behaving Prosocially," American Economic Review, American Economic Association, pages 544-555.
    2. Nolan Miller & Paul Resnick & Richard Zeckhauser, 2005. "Eliciting Informative Feedback: The Peer-Prediction Method," Management Science, INFORMS, pages 1359-1373.
    3. Chrysanthos Dellarocas, 2006. "Strategic Manipulation of Internet Opinion Forums: Implications for Consumers and Firms," Management Science, INFORMS, pages 1577-1593.
    4. Yan Chen & F. Maxwell Harper & Joseph Konstan & Sherry Xin Li, 2010. "Social Comparisons and Contributions to Online Communities: A Field Experiment on MovieLens," American Economic Review, American Economic Association, vol. 100(4), pages 1358-1398, September.
    5. Nikolay Archak & Anindya Ghose & Panagiotis G. Ipeirotis, 2007. "Deriving the Pricing Power of Product Features by Mining Consumer Reviews," Working Papers 07-36, NET Institute.
    6. David Godes & Dina Mayzlin, 2009. "Firm-Created Word-of-Mouth Communication: Evidence from a Field Test," Marketing Science, INFORMS, vol. 28(4), pages 721-739, 07-08.
    7. Nikolay Archak & Anindya Ghose & Panagiotis G. Ipeirotis, 2011. "Deriving the Pricing Power of Product Features by Mining Consumer Reviews," Management Science, INFORMS, pages 1485-1509.
    8. Pope, Devin G., 2009. "Reacting to rankings: Evidence from "America's Best Hospitals"," Journal of Health Economics, Elsevier, vol. 28(6), pages 1154-1165, December.
    9. Glazer, Jacob & McGuire, Thomas G. & Cao, Zhun & Zaslavsky, Alan, 2008. "Using global ratings of health plans to improve the quality of health care," Journal of Health Economics, Elsevier, vol. 27(5), pages 1182-1195, September.
    10. Xinxin Li & Lorin M. Hitt, 2008. "Self-Selection and Information Role of Online Product Reviews," Information Systems Research, INFORMS, vol. 19(4), pages 456-474, December.
    11. Abhijit V. Banerjee, 1992. "A Simple Model of Herd Behavior," The Quarterly Journal of Economics, Oxford University Press, vol. 107(3), pages 797-817.
    12. Jonathan E. Alevy & Michael S. Haigh & John A. List, 2007. "Information Cascades: Evidence from a Field Experiment with Financial Market Professionals," Journal of Finance, American Finance Association, vol. 62(1), pages 151-180, February.
    13. Dina Mayzlin, 2006. "Promotional Chat on the Internet," Marketing Science, INFORMS, vol. 25(2), pages 155-163, 03-04.
    14. Michael Luca & Jonathan Smith, 2013. "Salience in Quality Disclosure: Evidence from the U.S. News College Rankings," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 22(1), pages 58-77, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Liad Wagman & Vincent Conitzer, 2014. "False-name-proof voting with costs over two alternatives," International Journal of Game Theory, Springer;Game Theory Society, vol. 43(3), pages 599-618, August.
    2. Michael Luca & Georgios Zervas, 2013. "Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud," Harvard Business School Working Papers 14-006, Harvard Business School, revised May 2015.
    3. Benjamin Edelman & Micahel Luca, 2014. "Digital Discrimination: The Case of Airbnb.com," Harvard Business School Working Papers 14-054, Harvard Business School.
    4. Amedeo Piolatto, 2015. "Online booking and information: competition and welfare consequences of review aggregators," Working Papers 2015/11, Institut d'Economia de Barcelona (IEB).

    More about this item

    JEL classification:

    • D8 - Microeconomics - - Information, Knowledge, and Uncertainty
    • L15 - Industrial Organization - - Market Structure, Firm Strategy, and Market Performance - - - Information and Product Quality
    • L86 - Industrial Organization - - Industry Studies: Services - - - Information and Internet Services; Computer Software

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:18567. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (). General contact details of provider: http://edirc.repec.org/data/nberrus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.