Advanced Search
MyIDEAS: Login

Optimal Aggregation of Consumer Ratings: An Application to Yelp.com

Contents:

Author Info

  • Weijia Dai
  • Ginger Z. Jin
  • Jungmin Lee
  • Michael Luca

Abstract

Consumer review websites such as Yelp.com leverage the wisdom of the crowd, with each product being reviewed many times (some with more than 1000 reviews). Because of this, the way in which information is aggregated is a central decision faced by consumer review websites. Given a set of reviews, what is the optimal way to construct an average rating? We offer a structural approach to answering this question, allowing for (1) reviewers to vary in stringency (some reviewers tend to leave worse reviews on average) and accuracy (some reviewers are more erratic than others), (2) reviewers to be influenced by existing reviews, and (3) product quality to change over time. We apply this approach to reviews from Yelp.com to derive optimal ratings for each restaurant (in contrast with the arithmetic average displayed by Yelp). Because we have the history of reviews for each restaurant and many reviews left by each reviewer, we are able to identify these factors using variation in ratings within and across reviewers and restaurants. Using our estimated parameters, we construct optimal ratings for all restaurants on Yelp, and compare them to the arithmetic averages displayed by Yelp. As of the end of our sample, a conservative finding is that roughly 25-27% of restaurants are more than 0.15 stars away from the optimal rating, and 8-10% of restaurants are more than 0.25 stars away from the optimal rating. This suggests that large gains could be made by implementing optimal ratings. Much of the gains come from our method responding more quickly to changes in a restaurant’s quality. Our algorithm can be flexibly applied to many different review settings.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://www.nber.org/papers/w18567.pdf
Download Restriction: Access to the full text is generally limited to series subscribers, however if the top level domain of the client browser is in a developing country or transition economy free access is provided. More information about subscriptions and free access is available at http://www.nber.org/wwphelp.html. Free access is also available to older working papers.

As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

Bibliographic Info

Paper provided by National Bureau of Economic Research, Inc in its series NBER Working Papers with number 18567.

as in new window
Length:
Date of creation: Nov 2012
Date of revision:
Handle: RePEc:nbr:nberwo:18567

Note: IO
Contact details of provider:
Postal: National Bureau of Economic Research, 1050 Massachusetts Avenue Cambridge, MA 02138, U.S.A.
Phone: 617-868-3900
Email:
Web page: http://www.nber.org
More information through EDIRC

Related research

Keywords:

Find related papers by JEL classification:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Banerjee, Abhijit V, 1992. "A Simple Model of Herd Behavior," The Quarterly Journal of Economics, MIT Press, vol. 107(3), pages 797-817, August.
  2. Chrysanthos Dellarocas, 2006. "Strategic Manipulation of Internet Opinion Forums: Implications for Consumers and Firms," Management Science, INFORMS, vol. 52(10), pages 1577-1593, October.
  3. Alevy, Jonathan E. & Haigh, Michael S. & List, John A., 2003. "Information Cascades: Evidence From A Field Experiment With Financial Market Professionals," Working Papers 28608, University of Maryland, Department of Agricultural and Resource Economics.
  4. David Godes & Dina Mayzlin, 2009. "Firm-Created Word-of-Mouth Communication: Evidence from a Field Test," Marketing Science, INFORMS, vol. 28(4), pages 721-739, 07-08.
  5. Dan Ariely & Anat Bracha & Stephan Meier, 2007. "Doing good or doing well? Image motivation and monetary incentives in behaving prosocially," Working Papers 07-9, Federal Reserve Bank of Boston.
  6. Nikolay Archak & Anindya Ghose & Panagiotis G. Ipeirotis, 2011. "Deriving the Pricing Power of Product Features by Mining Consumer Reviews," Management Science, INFORMS, vol. 57(8), pages 1485-1509, August.
  7. Yan Chen & F. Maxwell Harper & Joseph Konstan & Sherry Xin Li, 2010. "Social Comparisons and Contributions to Online Communities: A Field Experiment on MovieLens," American Economic Review, American Economic Association, vol. 100(4), pages 1358-98, September.
  8. Glazer, Jacob & McGuire, Thomas G. & Cao, Zhun & Zaslavsky, Alan, 2008. "Using global ratings of health plans to improve the quality of health care," Journal of Health Economics, Elsevier, vol. 27(5), pages 1182-1195, September.
  9. Nolan Miller & Paul Resnick & Richard Zeckhauser, 2005. "Eliciting Informative Feedback: The Peer-Prediction Method," Management Science, INFORMS, vol. 51(9), pages 1359-1373, September.
  10. Pope, Devin G., 2009. "Reacting to rankings: Evidence from "America's Best Hospitals"," Journal of Health Economics, Elsevier, vol. 28(6), pages 1154-1165, December.
  11. Dina Mayzlin, 2006. "Promotional Chat on the Internet," Marketing Science, INFORMS, vol. 25(2), pages 155-163, 03-04.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Benjamin Edelman & Micahel Luca, 2014. "Digital Discrimination: The Case of Airbnb.com," Harvard Business School Working Papers 14-054, Harvard Business School.
  2. Michael Luca & Georgios Zervas, 2013. "Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud," Harvard Business School Working Papers 14-006, Harvard Business School.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:18567. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ().

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.