IDEAS home Printed from https://ideas.repec.org/a/inm/orisre/v34y2023i2p590-608.html
   My bibliography  Save this article

The Consequences of Rating Inflation on Platforms: Evidence from a Quasi-Experiment

Author

Listed:
  • Arslan Aziz

    (Sauder School of Business, The University of British Columbia, Vancouver, British Columbia V6T 1Z2, Canada)

  • Hui Li

    (HKU Business School, The University of Hong Kong, Hong Kong)

  • Rahul Telang

    (Heinz College of Information Systems and Public Policy, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213)

Abstract

Informative online ratings enable digital platforms to reduce the search cost for buyers to find good sellers. However, rating inflation, a phenomenon in which average rating increases and rating variance across listings decreases, threatens the informativeness of ratings. We empirically identify the consequences of rating inflation by conducting a quasi-experiment with a digital platform that exogenously changed its rating display rule in a treated neighborhood, which resulted in rating inflation. Using a differences-in-differences approach, we find that platforms benefit from one aspect of rating inflation: user purchases and seller sales increase because of the increased average rating. However, they also face negative consequences: rating inflation causes a decrease in user trial and a greater concentration of sales among popular restaurants. Overall, our results illustrate the potential consequences of rating inflation that platforms need to consider when designing and managing their rating system.

Suggested Citation

  • Arslan Aziz & Hui Li & Rahul Telang, 2023. "The Consequences of Rating Inflation on Platforms: Evidence from a Quasi-Experiment," Information Systems Research, INFORMS, vol. 34(2), pages 590-608, June.
  • Handle: RePEc:inm:orisre:v:34:y:2023:i:2:p:590-608
    DOI: 10.1287/isre.2022.1134
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/isre.2022.1134
    Download Restriction: no

    File URL: https://libkey.io/10.1287/isre.2022.1134?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Cameron,A. Colin & Trivedi,Pravin K., 2013. "Regression Analysis of Count Data," Cambridge Books, Cambridge University Press, number 9781107667273, January.
    2. Chunhua Wu & Hai Che & Tat Y. Chan & Xianghua Lu, 2015. "The Economic Value of Online Reviews," Marketing Science, INFORMS, vol. 34(5), pages 739-754, September.
    3. Dina Mayzlin & Yaniv Dover & Judith Chevalier, 2014. "Promotional Reviews: An Empirical Investigation of Online Review Manipulation," American Economic Review, American Economic Association, vol. 104(8), pages 2421-2455, August.
    4. Ronald T. Cenfetelli & Andrew Schwarz, 2011. "Identifying and Testing the Inhibitors of Technology Usage Intentions," Information Systems Research, INFORMS, vol. 22(4), pages 808-823, December.
    5. Weijia (Daisy) Dai & Ginger Jin & Jungmin Lee & Michael Luca, 2018. "Aggregation of consumer ratings: an application to Yelp.com," Quantitative Marketing and Economics (QME), Springer, vol. 16(3), pages 289-339, September.
    6. Yili (Kevin) Hong & Paul A. Pavlou, 2014. "Product Fit Uncertainty in Online Markets: Nature, Effects, and Antecedents," Information Systems Research, INFORMS, vol. 25(2), pages 328-344, June.
    7. Santos Silva, J.M.C. & Tenreyro, Silvana, 2011. "Further simulation evidence on the performance of the Poisson pseudo-maximum likelihood estimator," Economics Letters, Elsevier, vol. 112(2), pages 220-222, August.
    8. Xianghua Lu & Sulin Ba & Lihua Huang & Yue Feng, 2013. "Promotional Marketing or Word-of-Mouth? Evidence from Online Restaurant Reviews," Information Systems Research, INFORMS, vol. 24(3), pages 596-612, September.
    9. Ciani Emanuele & Fisher Paul, 2019. "Dif-in-Dif Estimators of Multiplicative Treatment Effects," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-10, January.
    10. Monic Sun, 2012. "How Does the Variance of Product Ratings Matter?," Management Science, INFORMS, vol. 58(4), pages 696-707, April.
    11. Michael Anderson & Jeremy Magruder, 2012. "Learning from the Crowd: Regression Discontinuity Estimates of the Effects of an Online Review Database," Economic Journal, Royal Economic Society, vol. 122(563), pages 957-989, September.
    12. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, December.
    13. Tingting Song & Jinghua Huang & Yong Tan & Yifan Yu, 2019. "Using User- and Marketer-Generated Content for Box Office Revenue Prediction: Differences Between Microblogging and Third-Party Platforms," Service Science, INFORMS, vol. 30(1), pages 191-203, March.
    14. Panagiotis Adamopoulos & Anindya Ghose & Vilma Todri, 2018. "The Impact of User Personality Traits on Word of Mouth: Text-Mining Social Media Platforms," Information Systems Research, INFORMS, vol. 29(3), pages 612-640, September.
    15. Pierre Azoulay & Joshua S. Graff Zivin & Jialan Wang, 2010. "Superstar Extinction," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(2), pages 549-589.
    16. Puhani, Patrick A., 2012. "The treatment effect, the cross difference, and the interaction term in nonlinear “difference-in-differences” models," Economics Letters, Elsevier, vol. 115(1), pages 85-87.
    17. Sergio Correia & Paulo Guimarães & Tom Zylkin, 2020. "Fast Poisson estimation with high-dimensional fixed effects," Stata Journal, StataCorp LP, vol. 20(1), pages 95-115, March.
    18. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    19. Shengwu Shang & Erik Nesson & Maoyong Fan, 2018. "Interaction Terms In Poisson And Log Linear Regression Models," Bulletin of Economic Research, Wiley Blackwell, vol. 70(1), pages 89-96, January.
    20. Chris Nosko & Steven Tadelis, 2015. "The Limits of Reputation in Platform Markets: An Empirical Analysis and Field Experiment," NBER Working Papers 20830, National Bureau of Economic Research, Inc.
    21. Xinxin Li & Lorin M. Hitt, 2008. "Self-Selection and Information Role of Online Product Reviews," Information Systems Research, INFORMS, vol. 19(4), pages 456-474, December.
    22. Yi Zhao & Sha Yang & Vishal Narayan & Ying Zhao, 2013. "Modeling Consumer Learning from Online Product Reviews," Marketing Science, INFORMS, vol. 32(1), pages 153-169, May.
    23. Aishwarya Deep Shukla & Guodong (Gordon) Gao & Ritu Agarwal, 2021. "How Digital Word-of-Mouth Affects Consumer Decision Making: Evidence from Doctor Appointment Booking," Management Science, INFORMS, vol. 67(3), pages 1546-1568, March.
    24. Davide Proserpio & Wendy Xu & Georgios Zervas, 2018. "You get what you give: theory and evidence of reciprocity in the sharing economy," Quantitative Marketing and Economics (QME), Springer, vol. 16(4), pages 371-407, December.
    25. Gordon Burtch & Yili Hong & Ravi Bapna & Vladas Griskevicius, 2018. "Stimulating Online Reviews by Combining Financial Incentives and Social Norms," Management Science, INFORMS, vol. 64(5), pages 2065-2082, May.
    26. Ai, Chunrong & Norton, Edward C., 2003. "Interaction terms in logit and probit models," Economics Letters, Elsevier, vol. 80(1), pages 123-129, July.
    27. Hongbin Cai & Yuyu Chen & Hanming Fang, 2009. "Observational Learning: Evidence from a Randomized Natural Field Experiment," American Economic Review, American Economic Association, vol. 99(3), pages 864-882, June.
    28. Georgios Zervas & Davide Proserpio & John W. Byers, 2021. "A first look at online reputation on Airbnb, where every stay is above average," Marketing Letters, Springer, vol. 32(1), pages 1-16, March.
    29. Pei-Yu Chen & Yili Hong & Ying Liu, 2018. "The Value of Multidimensional Rating Systems: Evidence from a Natural Experiment and Randomized Experiments," Management Science, INFORMS, vol. 64(10), pages 4629-4647, October.
    30. Luís Cabral & Ali Hortaçsu, 2010. "The Dynamics Of Seller Reputation: Evidence From Ebay," Journal of Industrial Economics, Wiley Blackwell, vol. 58(1), pages 54-78, March.
    31. Thomas Zylkin, 2019. "Verifying the Existence of Maximum Likelihood Estimates in Generalized Linear Models," 2019 Stata Conference 47, Stata Users Group.
    32. Chrysanthos Dellarocas & Charles A. Wood, 2008. "The Sound of Silence in Online Feedback: Estimating Trading Risks in the Presence of Reporting Bias," Management Science, INFORMS, vol. 54(3), pages 460-476, March.
    33. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    34. Yi-Chun (Chad) Ho & Junjie Wu & Yong Tan, 2017. "Disconfirmation Effect on Online Rating Behavior: A Structural Model," Information Systems Research, INFORMS, vol. 28(3), pages 626-642, September.
    35. Cheng Yi & Zhenhui (Jack) Jiang & Izak Benbasat, 2017. "Designing for Diagnosticity and Serendipity: An Investigation of Social Product-Search Mechanisms," Information Systems Research, INFORMS, vol. 28(2), pages 413-429, June.
    36. Chong (Alex) Wang & Xiaoquan (Michael) Zhang & Il-Horn Hann, 2018. "Socially Nudged: A Quasi-Experimental Study of Friends’ Social Influence in Online Product Ratings," Information Systems Research, INFORMS, vol. 29(3), pages 641-655, September.
    37. Steven Tadelis, 2016. "Reputation and Feedback Systems in Online Platform Markets," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 321-340, October.
    38. David Godes & José C. Silva, 2012. "Sequential and Temporal Dynamics of Online Opinion," Marketing Science, INFORMS, vol. 31(3), pages 448-473, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dominik Gutt & Jürgen Neumann & Steffen Zimmermann & Dennis Kundisch & Jianqing Chen, 2018. "Design of Review Systems - A Strategic Instrument to shape Online Review Behavior and Economic Outcomes," Working Papers Dissertations 42, Paderborn University, Faculty of Business Administration and Economics.
    2. Foster, Joshua, 2022. "How rating mechanisms shape user search, quality inference and engagement in online platforms: Experimental evidence," Journal of Business Research, Elsevier, vol. 142(C), pages 791-807.
    3. Xiang Hui & Tobias J. Klein & Konrad Stahl, 2021. "When and Why Do Buyers Rate in Online Markets?," CRC TR 224 Discussion Paper Series crctr224_2021_267v1, University of Bonn and University of Mannheim, Germany.
    4. Cheng Zhao & Chong Alex Wang, 2023. "A cross-site comparison of online review manipulation using Benford’s law," Electronic Commerce Research, Springer, vol. 23(1), pages 365-406, March.
    5. Christoph Schneider & Markus Weinmann & Peter N.C. Mohr & Jan vom Brocke, 2021. "When the Stars Shine Too Bright: The Influence of Multidimensional Ratings on Online Consumer Ratings," Management Science, INFORMS, vol. 67(6), pages 3871-3898, June.
    6. Xue Bai & James R. Marsden & William T. Ross & Gang Wang, 2020. "A Note on the Impact of Daily Deals on Local Retailers’ Online Reputation: Mediation Effects of the Consumer Experience," Information Systems Research, INFORMS, vol. 31(4), pages 1132-1143, December.
    7. Nachiketa Sahoo & Chrysanthos Dellarocas & Shuba Srinivasan, 2018. "The Impact of Online Product Reviews on Product Returns," Information Systems Research, INFORMS, vol. 29(3), pages 723-738, September.
    8. Gesche, Tobias, 2018. "Reference Price Shifts and Customer Antagonism: Evidence from Reviews for Online Auctions," VfS Annual Conference 2018 (Freiburg, Breisgau): Digital Economy 181650, Verein für Socialpolitik / German Economic Association.
    9. Erfan Rezvani & Christian Rojas, 2022. "Firm responsiveness to consumers' reviews: The effect on online reputation," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 31(4), pages 898-922, November.
    10. Andreas J. Steur & Mischa Seiter, 2021. "Properties of feedback mechanisms on digital platforms: an exploratory study," Journal of Business Economics, Springer, vol. 91(4), pages 479-526, May.
    11. Yue Guan & Yong Tan & Qiang Wei & Guoqing Chen, 2023. "When Images Backfire: The Effect of Customer-Generated Images on Product Rating Dynamics," Information Systems Research, INFORMS, vol. 34(4), pages 1641-1663, December.
    12. Marios Kokkodis & Theodoros Lappas, 2020. "Your Hometown Matters: Popularity-Difference Bias in Online Reputation Platforms," Information Systems Research, INFORMS, vol. 31(2), pages 412-430, June.
    13. Georgios Zervas & Davide Proserpio & John W. Byers, 2021. "A first look at online reputation on Airbnb, where every stay is above average," Marketing Letters, Springer, vol. 32(1), pages 1-16, March.
    14. Marios Kokkodis & Theodoros Lappas & Gerald C. Kane, 2022. "Optional purchase verification in e‐commerce platforms: More representative product ratings and higher quality reviews," Production and Operations Management, Production and Operations Management Society, vol. 31(7), pages 2943-2961, July.
    15. Wei Chen & Bin Gu & Qiang Ye & Kevin Xiaoguo Zhu, 2019. "Measuring and Managing the Externality of Managerial Responses to Online Customer Reviews," Service Science, INFORMS, vol. 30(1), pages 81-96, March.
    16. Chong (Alex) Wang & Xiaoquan (Michael) Zhang & Il-Horn Hann, 2018. "Socially Nudged: A Quasi-Experimental Study of Friends’ Social Influence in Online Product Ratings," Information Systems Research, INFORMS, vol. 29(3), pages 641-655, September.
    17. Michael Luca & Georgios Zervas, 2013. "Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud," Harvard Business School Working Papers 14-006, Harvard Business School, revised May 2015.
    18. Sungsik Park & Woochoel Shin & Jinhong Xie, 2021. "The Fateful First Consumer Review," Marketing Science, INFORMS, vol. 40(3), pages 481-507, May.
    19. Pei-Yu Chen & Yili Hong & Ying Liu, 2018. "The Value of Multidimensional Rating Systems: Evidence from a Natural Experiment and Randomized Experiments," Management Science, INFORMS, vol. 64(10), pages 4629-4647, October.
    20. Dominik Gutt & Philipp Herrmann & Mohammad S. Rahman, 2018. "Crowd-Driven Competitive Intelligence: Understanding the Relationship Between Local Market Competition and Online Rating Distributions," Working Papers Dissertations 41, Paderborn University, Faculty of Business Administration and Economics.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orisre:v:34:y:2023:i:2:p:590-608. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.