Optimal Aggregation of Consumer Ratings: An Application to Yelp.com
AbstractConsumer review websites such as Yelp.com leverage the wisdom of the crowd, with each product being reviewed many times (some with more than 1000 reviews). Because of this, the way in which information is aggregated is a central decision faced by consumer review websites. Given a set of reviews, what is the optimal way to construct an average rating? We offer a structural approach to answering this question, allowing for (1) reviewers to vary in stringency (some reviewers tend to leave worse reviews on average) and accuracy (some reviewers are more erratic than others), (2) reviewers to be influenced by existing reviews, and (3) product quality to change over time. We apply this approach to reviews from Yelp.com to derive optimal ratings for each restaurant (in contrast with the arithmetic average displayed by Yelp). Because we have the history of reviews for each restaurant and many reviews left by each reviewer, we are able to identify these factors using variation in ratings within and across reviewers and restaurants. Using our estimated parameters, we construct optimal ratings for all restaurants on Yelp, and compare them to the arithmetic averages displayed by Yelp. As of the end of our sample, a conservative finding is that roughly 25-27% of restaurants are more than 0.15 stars away from the optimal rating, and 8-10% of restaurants are more than 0.25 stars away from the optimal rating. This suggests that large gains could be made by implementing optimal ratings. Much of the gains come from our method responding more quickly to changes in a restaurant’s quality. Our algorithm can be flexibly applied to many different review settings.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.
Bibliographic InfoPaper provided by National Bureau of Economic Research, Inc in its series NBER Working Papers with number 18567.
Date of creation: Nov 2012
Date of revision:
Contact details of provider:
Postal: National Bureau of Economic Research, 1050 Massachusetts Avenue Cambridge, MA 02138, U.S.A.
Web page: http://www.nber.org
More information through EDIRC
Find related papers by JEL classification:
- D8 - Microeconomics - - Information, Knowledge, and Uncertainty
- L15 - Industrial Organization - - Market Structure, Firm Strategy, and Market Performance - - - Information and Product Quality
- L86 - Industrial Organization - - Industry Studies: Services - - - Information and Internet Services; Computer Software
This paper has been announced in the following NEP Reports:
- NEP-ALL-2012-12-15 (All new papers)
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Banerjee, Abhijit V, 1992. "A Simple Model of Herd Behavior," The Quarterly Journal of Economics, MIT Press, vol. 107(3), pages 797-817, August.
- Chrysanthos Dellarocas, 2006. "Strategic Manipulation of Internet Opinion Forums: Implications for Consumers and Firms," Management Science, INFORMS, vol. 52(10), pages 1577-1593, October.
- Alevy, Jonathan E. & Haigh, Michael S. & List, John A., 2003.
"Information Cascades: Evidence From A Field Experiment With Financial Market Professionals,"
28608, University of Maryland, Department of Agricultural and Resource Economics.
- Jonathan E. Alevy & Michael S. Haigh & John A. List, 2007. "Information Cascades: Evidence from a Field Experiment with Financial Market Professionals," Journal of Finance, American Finance Association, vol. 62(1), pages 151-180, 02.
- Jonathan Alevy & Michael Haigh & John List, 2005. "Information cascades: Evidence from a field experiment with financial market professionals," Framed Field Experiments 00116, The Field Experiments Website.
- David Godes & Dina Mayzlin, 2009. "Firm-Created Word-of-Mouth Communication: Evidence from a Field Test," Marketing Science, INFORMS, vol. 28(4), pages 721-739, 07-08.
- Dan Ariely & Anat Bracha & Stephan Meier, 2007.
"Doing good or doing well? Image motivation and monetary incentives in behaving prosocially,"
07-9, Federal Reserve Bank of Boston.
- Dan Ariely & Anat Bracha & Stephan Meier, 2009. "Doing Good or Doing Well? Image Motivation and Monetary Incentives in Behaving Prosocially," American Economic Review, American Economic Association, vol. 99(1), pages 544-55, March.
- Ariely, Dan & Bracha, Anat & Meier, Stephan, 2007. "Doing Good or Doing Well? Image Motivation and Monetary Incentives in Behaving Prosocially," IZA Discussion Papers 2968, Institute for the Study of Labor (IZA).
- Nikolay Archak & Anindya Ghose & Panagiotis G. Ipeirotis, 2011. "Deriving the Pricing Power of Product Features by Mining Consumer Reviews," Management Science, INFORMS, vol. 57(8), pages 1485-1509, August.
- Yan Chen & F. Maxwell Harper & Joseph Konstan & Sherry Xin Li, 2010. "Social Comparisons and Contributions to Online Communities: A Field Experiment on MovieLens," American Economic Review, American Economic Association, vol. 100(4), pages 1358-98, September.
- Glazer, Jacob & McGuire, Thomas G. & Cao, Zhun & Zaslavsky, Alan, 2008. "Using global ratings of health plans to improve the quality of health care," Journal of Health Economics, Elsevier, vol. 27(5), pages 1182-1195, September.
- Nolan Miller & Paul Resnick & Richard Zeckhauser, 2005. "Eliciting Informative Feedback: The Peer-Prediction Method," Management Science, INFORMS, vol. 51(9), pages 1359-1373, September.
- Pope, Devin G., 2009. "Reacting to rankings: Evidence from "America's Best Hospitals"," Journal of Health Economics, Elsevier, vol. 28(6), pages 1154-1165, December.
- Dina Mayzlin, 2006. "Promotional Chat on the Internet," Marketing Science, INFORMS, vol. 25(2), pages 155-163, 03-04.
- Benjamin Edelman & Micahel Luca, 2014. "Digital Discrimination: The Case of Airbnb.com," Harvard Business School Working Papers 14-054, Harvard Business School.
- Michael Luca & Georgios Zervas, 2013. "Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud," Harvard Business School Working Papers 14-006, Harvard Business School.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ().
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If references are entirely missing, you can add them using this form.
If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.
Please note that corrections may take a couple of weeks to filter through the various RePEc services.