IDEAS home Printed from https://ideas.repec.org/p/cpr/ceprdp/16147.html
   My bibliography  Save this paper

Bias in expert product reviews

Author

Listed:
  • van Ours, Jan C.
  • Vollaard, Ben

Abstract

Hard evidence on bias in expert reviews and its consequences for ratings is rare. This holds particularly true for conflicts of interest that are thought to be common in non-blind product reviews but are not readily observable: ad hoc relationships between reviewers and producers. We present a textbook case of a long-running expert product review in the food service industry for which we happen to know the reviewer's conflict of interest: being affiliated to one particular producer. As is typical, only insiders were aware of the possible source of bias in the review. The review resembles other non-blind tests of product quality. We obtained detailed data to map the consequences of the conflict of interest. We find evidence of a sizable bias in the reviewers' ratings. Our findings suggest that reviewers' ad hoc relationships with producers, often dismissed as `coming with the job', can be very harmful.

Suggested Citation

  • van Ours, Jan C. & Vollaard, Ben, 2021. "Bias in expert product reviews," CEPR Discussion Papers 16147, C.E.P.R. Discussion Papers.
    • Handle: RePEc:cpr:ceprdp:16147
    as

    Download full text from publisher

    File URL: https://cepr.org/publications/DP16147
    Download Restriction: CEPR Discussion Papers are free to download for our researchers, subscribers and members. If you fall into one of these categories but have trouble downloading our papers, please contact us at subscribers@cepr.org
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Imke Reimers & Joel Waldfogel, 2021. "Digitization and Pre-purchase Information: The Causal and Welfare Impacts of Reviews and Crowd Ratings," American Economic Review, American Economic Association, vol. 111(6), pages 1944-1971, June.
    2. Dina Mayzlin & Yaniv Dover & Judith Chevalier, 2014. "Promotional Reviews: An Empirical Investigation of Online Review Manipulation," American Economic Review, American Economic Association, vol. 104(8), pages 2421-2455, August.
    3. Santosh Anagol & Shawn Cole & Shayak Sarkar, 2017. "Understanding the Advice of Commissions-Motivated Agents: Evidence from the Indian Life Insurance Market," The Review of Economics and Statistics, MIT Press, vol. 99(1), pages 1-15, March.
    4. Goldin, Claudia D. & Rouse, Cecilia, 2000. "Orchestrating Impartiality: The Impact of “Blind†Auditions on Female Musicians," Scholarly Articles 30703974, Harvard University Department of Economics.
    5. Lin, Hsiou-wei & McNichols, Maureen F., 1998. "Underwriting relationships, analysts' earnings forecasts and investment recommendations," Journal of Accounting and Economics, Elsevier, vol. 25(1), pages 101-127, February.
    6. Tom Hamami, 2019. "Network Effects, Bargaining Power, and Product Review Bias: Theory and Evidence," Journal of Industrial Economics, Wiley Blackwell, vol. 67(2), pages 372-407, June.
    7. David A. Reinstein & Christopher M. Snyder, 2005. "The Influence Of Expert Reviews On Consumer Demand For Experience Goods: A Case Study Of Movie Critics," Journal of Industrial Economics, Wiley Blackwell, vol. 53(1), pages 27-51, March.
    8. Yubo Chen & Jinhong Xie, 2005. "Third-Party Product Review and Firm Marketing Strategy," Marketing Science, INFORMS, vol. 24(2), pages 218-240, February.
    9. Dobrescu, Loretti I. & Luca, Michael & Motta, Alberto, 2013. "What makes a critic tick? Connected authors and the determinants of book reviews," Journal of Economic Behavior & Organization, Elsevier, vol. 96(C), pages 85-103.
    10. Victor A. Ginsburgh & Jan C. van Ours, 2003. "Expert Opinion and Compensation: Evidence from a Musical Competition," American Economic Review, American Economic Association, vol. 93(1), pages 289-296, March.
    11. Michael Anderson & Jeremy Magruder, 2012. "Learning from the Crowd: Regression Discontinuity Estimates of the Effects of an Online Review Database," Economic Journal, Royal Economic Society, vol. 122(563), pages 957-989, September.
    12. Amanda Hollenbacher & David Yerger, 2001. "Third party evaluations and resale prices in the US used vehicle market," Applied Economics Letters, Taylor & Francis Journals, vol. 8(6), pages 415-418.
    13. S. Ravid & John Wald & Suman Basuroy, 2006. "Distributors and film critics: does it take two to Tango?," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 30(3), pages 201-218, December.
    14. Tommaso Colussi, 2018. "Social Ties in Academia: A Friend Is a Treasure," The Review of Economics and Statistics, MIT Press, vol. 100(1), pages 45-50, March.
    15. Alessandro Lizzeri, 1999. "Information Revelation and Certification Intermediaries," RAND Journal of Economics, The RAND Corporation, vol. 30(2), pages 214-231, Summer.
    16. Véronique Chossat & Olivier Gergaud, 2003. "Expert Opinion and Gastronomy: The Recipe for Success," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 27(2), pages 127-141, May.
    17. Dewenter Ralf & Heimeshoff Ulrich, 2014. "Media Bias and Advertising: Evidence from a German Car Magazine," Review of Economics, De Gruyter, vol. 65(1), pages 77-94, April.
    18. Danielle Li, 2017. "Expertise versus Bias in Evaluation: Evidence from the NIH," American Economic Journal: Applied Economics, American Economic Association, vol. 9(2), pages 60-92, April.
    19. Colussi, Tommaso, 2015. "Social Ties in Academia: A Friend is a Treasure," IZA Discussion Papers 9414, Institute of Labor Economics (IZA).
    20. Claudia Goldin & Cecilia E. Rouse, 1997. "Orchestrating Impartiality: The Impact of 'Blind' Auditions on Female Musicians," Working Papers 755, Princeton University, Department of Economics, Industrial Relations Section..
    21. Stefano Dellavigna & Johannes Hermle, 2017. "Does Conflict of Interest Lead to Biased Coverage? Evidence from Movie Reviews," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 84(4), pages 1510-1550.
    22. Victor Ginsburgh & Renato Flores Galvao, 1996. "The Queen Elisabeth Musical Competition: how fair is the final ranking," ULB Institutional Repository 2013/1713, ULB -- Universite Libre de Bruxelles.
    23. Cecilia Rouse & Claudia Goldin, 2000. "Orchestrating Impartiality: The Impact of "Blind" Auditions on Female Musicians," American Economic Review, American Economic Association, vol. 90(4), pages 715-741, September.
    24. Reuter, Jonathan, 2009. "Does Advertising Bias Product Reviews? An Analysis of Wine Ratings," Journal of Wine Economics, Cambridge University Press, vol. 4(2), pages 125-151, January.
    25. Michaely, Roni & Womack, Kent L, 1999. "Conflict of Interest and the Credibility of Underwriter Analyst Recommendations," The Review of Financial Studies, Society for Financial Studies, vol. 12(4), pages 653-686.
    26. Tom Hamami & James Bailey, 2021. "Expert product reviews and conflict of interest," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 42(1), pages 170-176, January.
    27. Ashenfelter, Orley & Jones, Gregory V., 2013. "The Demand for Expert Opinion: Bordeaux Wine," Journal of Wine Economics, Cambridge University Press, vol. 8(3), pages 285-293, December.
    28. Michael Luca & Georgios Zervas, 2016. "Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud," Management Science, INFORMS, vol. 62(12), pages 3412-3427, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Asmat, Roberto & Borowiecki, Karol J. & Law, Marc T., 2023. "Do experts and laypersons differ? Some evidence from international classical music competitions," Journal of Economic Behavior & Organization, Elsevier, vol. 214(C), pages 270-290.
    2. Gábor Békés & Endre Borza & Márton Fleck, 2024. "Favoritism under multiple sources of social pressure," Economic Inquiry, Western Economic Association International, vol. 62(4), pages 1748-1769, October.
    3. Fengnan Gao & Richard D. Gill, 2023. "Pitfalls of amateur regression: The Dutch New Herring controversies," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 50(4), pages 1901-1918, December.
    4. Jan C. Van Ours & Ben Vollaard, 2024. "Commentary on “Pitfalls of amateur regression: The Dutch New Herring controversies”," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 51(3), pages 1388-1389, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tom Hamami & James Bailey, 2021. "Expert product reviews and conflict of interest," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 42(1), pages 170-176, January.
    2. Luis Aguiar, 2024. "Bad Apples on Rotten Tomatoes: Critics, Crowds, and Gender Bias in Product Ratings," CESifo Working Paper Series 11422, CESifo.
    3. Luis Aguiar & Imke Reimers & Joel Waldfogel, 2024. "Platforms and the transformation of the content industries," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 33(2), pages 317-326, March.
    4. Emmanuel Paroissien & Michael Visser, 2020. "The Causal Impact of Medals on Wine Producers' Prices and the Gains from Participating in Contests," American Journal of Agricultural Economics, John Wiley & Sons, vol. 102(4), pages 1135-1153, August.
    5. Ginsburgh, Victor & Radermecker, Anne-Sophie & Tommasi, Denni, 2019. "The effect of experts’ opinion on prices of art works: The case of Peter Brueghel the Younger," Journal of Economic Behavior & Organization, Elsevier, vol. 159(C), pages 36-50.
    6. Brett Hollenbeck & Sridhar Moorthy & Davide Proserpio, 2019. "Advertising Strategy in the Presence of Reviews: An Empirical Analysis," Marketing Science, INFORMS, vol. 38(5), pages 793-811, September.
    7. Tom Hamami, 2019. "Network Effects, Bargaining Power, and Product Review Bias: Theory and Evidence," Journal of Industrial Economics, Wiley Blackwell, vol. 67(2), pages 372-407, June.
    8. Stefano Dellavigna & Johannes Hermle, 2017. "Does Conflict of Interest Lead to Biased Coverage? Evidence from Movie Reviews," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 84(4), pages 1510-1550.
    9. Jonathan Reuter & Eric Zitzewitz, 2006. "Do Ads Influence Editors? Advertising and Bias in the Financial Media," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 121(1), pages 197-227.
    10. Ian Ayres & Mahzarin Banaji & Christine Jolls, 2015. "Race effects on eBay," RAND Journal of Economics, RAND Corporation, vol. 46(4), pages 891-917, October.
    11. Florysiak, David & Schandlbauer, Alexander, 2022. "Experts or charlatans? ICO analysts and white paper informativeness," Journal of Banking & Finance, Elsevier, vol. 139(C).
    12. Warut Khern-am-nuai & Karthik Kannan & Hossein Ghasemkhani, 2018. "Extrinsic versus Intrinsic Rewards for Contributing Reviews in an Online Platform," Information Systems Research, INFORMS, vol. 29(4), pages 871-892, December.
    13. Jacobsen, Grant D., 2015. "Consumers, experts, and online product evaluations: Evidence from the brewing industry," Journal of Public Economics, Elsevier, vol. 126(C), pages 114-123.
    14. Lerchenmueller, Marc J. & Sorenson, Olav, 2018. "The gender gap in early career transitions in the life sciences," Research Policy, Elsevier, vol. 47(6), pages 1007-1017.
    15. Sung H. Ham & Ingrid Koch & Noah Lim & Jiabin Wu, 2021. "Conflict of Interest in Third-Party Reviews: An Experimental Study," Management Science, INFORMS, vol. 67(12), pages 7535-7559, December.
    16. Kevin Chung & Keehyung Kim & Noah Lim, 2020. "Social Structures and Reputation in Expert Review Systems," Management Science, INFORMS, vol. 66(7), pages 3249-3276, July.
    17. Hui, Xiang & Klein, Tobias & Stahl, Konrad, 2022. "Learning from Online Ratings," CEPR Discussion Papers 17006, C.E.P.R. Discussion Papers.
    18. Lorenzo Ductor & Sanjeev Goyal & Anja Prummer, 2018. "Gender & Collaboration," Working Papers 856, Queen Mary University of London, School of Economics and Finance.
    19. Committee, Nobel Prize, 2023. "Scientific Background to the Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel 2023," Nobel Prize in Economics documents 2023-2, Nobel Prize Committee.
    20. Meurs, Dominique & Puhani, Patrick A., 2024. "Culture as a Hiring Criterion: Systemic Discrimination in a Procedurally Fair Hiring Process," Labour Economics, Elsevier, vol. 87(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cpr:ceprdp:16147. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://www.cepr.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.