IDEAS home Printed from https://ideas.repec.org/a/inm/ormksc/v42y2023i5p853-865.html
   My bibliography  Save this article

Do Incentives to Review Help the Market? Evidence from a Field Experiment on Airbnb

Author

Listed:
  • Andrey Fradkin

    (Questrom School of Business, Boston University, Boston, Massachusetts 02215)

  • David Holtz

    (Management of Organizations and Entrepreneurship and Innovation, Haas School of Business, University of California, Berkeley, California 94720; MIT Initiative on the Digital Economy, MIT Sloan School of Management, Cambridge, Massachusetts 02142)

Abstract

Many online reputation systems operate by asking volunteers to write reviews for free. As a result, a large share of buyers do not review, and those who do review are self-selected. This can cause the reputation system to miss important information about seller quality. We study the extent to which a platform can improve market outcomes by attempting to increase the amount and quality of information collected by its reputation system. We do so by analyzing a randomized experiment conducted by Airbnb. In the treatment, buyers were offered a coupon to review listings that had no prior reviews. In the control, buyers were not offered any incentive to review. We find that, although the treatment induced additional reviews that were more negative on average, these reviews did not affect the number of nights sold or total revenue. Furthermore, we find that, contrary to the treatment’s intended effect, Airbnb’s incentivized program caused transaction quality for treated sellers to fall. We examine how the quality of the induced reviews, market conditions, and the design of Airbnb’s reputation system can explain our findings.

Suggested Citation

  • Andrey Fradkin & David Holtz, 2023. "Do Incentives to Review Help the Market? Evidence from a Field Experiment on Airbnb," Marketing Science, INFORMS, vol. 42(5), pages 853-865, September.
    • Handle: RePEc:inm:ormksc:v:42:y:2023:i:5:p:853-865
    DOI: 10.1287/mksc.2023.1439
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mksc.2023.1439
    Download Restriction: no
    File URL: https://libkey.io/10.1287/mksc.2023.1439?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Imke Reimers & Joel Waldfogel, 2021. "Digitization and Pre-purchase Information: The Causal and Welfare Impacts of Reviews and Crowd Ratings," American Economic Review, American Economic Association, vol. 111(6), pages 1944-1971, June.
    2. Chunhua Wu & Hai Che & Tat Y. Chan & Xianghua Lu, 2015. "The Economic Value of Online Reviews," Marketing Science, INFORMS, vol. 34(5), pages 739-754, September.
    3. Dina Mayzlin & Yaniv Dover & Judith Chevalier, 2014. "Promotional Reviews: An Empirical Investigation of Online Review Manipulation," American Economic Review, American Economic Association, vol. 104(8), pages 2421-2455, August.
    4. Lingfang (Ivy) Li & Erte Xiao, 2014. "Money Talks: Rebate Mechanisms in Reputation System Design," Management Science, INFORMS, vol. 60(8), pages 2054-2072, August.
    5. Xiang Hui & Tobias J. Klein & Konrad Stahl, 2021. "When and Why Do Buyers Rate in Online Markets?," CRC TR 224 Discussion Paper Series crctr224_2021_267, University of Bonn and University of Mannheim, Germany.
    6. Lingfang (Ivy) Li, 2010. "Reputation, Trust, and Rebates: How Online Auction Markets Can Improve Their Feedback Mechanisms," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 19(2), pages 303-331, June.
    7. Nikhil Garg & Ramesh Johari, 2021. "Designing Informative Rating Systems: Evidence from an Online Labor Market," Manufacturing & Service Operations Management, INFORMS, vol. 23(3), pages 589-605, May.
    8. Luís Cabral & Lingfang (Ivy) Li, 2015. "A Dollar for Your Thoughts: Feedback-Conditional Rebates on eBay," Management Science, INFORMS, vol. 61(9), pages 2052-2063, September.
    9. Paul Resnick & Christopher Avery & Richard Zeckhauser, 1999. "The Market for Evaluations," American Economic Review, American Economic Association, vol. 89(3), pages 564-584, June.
    10. Gordon Burtch & Yili Hong & Ravi Bapna & Vladas Griskevicius, 2018. "Stimulating Online Reviews by Combining Financial Incentives and Social Norms," Management Science, INFORMS, vol. 64(5), pages 2065-2082, May.
    11. Lingfang (Ivy) Li & Steven Tadelis & Xiaolan Zhou, 2020. "Buying reputation as a signal of quality: Evidence from an online marketplace," RAND Journal of Economics, RAND Corporation, vol. 51(4), pages 965-988, December.
    12. Xiang Hui & Tobias J. Klein & Konrad Stahl, 2021. "When and Why Do Buyers Rate in Online Markets?," CRC TR 224 Discussion Paper Series crctr224_2021_267v1, University of Bonn and University of Mannheim, Germany.
    13. Ruomeng Cui & Jun Li & Dennis J. Zhang, 2020. "Reducing Discrimination with Reviews in the Sharing Economy: Evidence from Field Experiments on Airbnb," Management Science, INFORMS, vol. 66(3), pages 1071-1094, March.
    14. Yiangos Papanastasiou & Kostas Bimpikis & Nicos Savva, 2018. "Crowdsourcing Exploration," Management Science, INFORMS, vol. 64(4), pages 1727-1746, April.
    15. Sungsik Park & Woochoel Shin & Jinhong Xie, 2021. "The Fateful First Consumer Review," Decision Analysis, INFORMS, vol. 40(3), pages 481-507, May-June.
    16. Chris Nosko & Steven Tadelis, 2015. "The Limits of Reputation in Platform Markets: An Empirical Analysis and Field Experiment," NBER Working Papers 20830, National Bureau of Economic Research, Inc.
    17. Steven Tadelis, 2016. "Reputation and Feedback Systems in Online Platform Markets," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 321-340, October.
    18. Amanda Pallais, 2014. "Inefficient Hiring in Entry-Level Labor Markets," American Economic Review, American Economic Association, vol. 104(11), pages 3565-3599, November.
    19. Chiara Farronato & Andrey Fradkin & Bradley Larsen & Erik Brynjolfsson, 2020. "Consumer Protection in an Online World: An Analysis of Occupational Licensing," NBER Working Papers 26601, National Bureau of Economic Research, Inc.
    20. Michael Luca & Georgios Zervas, 2016. "Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud," Management Science, INFORMS, vol. 62(12), pages 3412-3427, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yan Chen & Peter Cramton & John A. List & Axel Ockenfels, 2021. "Market Design, Human Behavior, and Management," Management Science, INFORMS, vol. 67(9), pages 5317-5348, September.
    2. Lingfang (Ivy) Li & Steven Tadelis & Xiaolan Zhou, 2020. "Buying reputation as a signal of quality: Evidence from an online marketplace," RAND Journal of Economics, RAND Corporation, vol. 51(4), pages 965-988, December.
    3. Gesche, Tobias, 2018. "Reference Price Shifts and Customer Antagonism: Evidence from Reviews for Online Auctions," VfS Annual Conference 2018 (Freiburg, Breisgau): Digital Economy 181650, Verein für Socialpolitik / German Economic Association.
    4. Foster, Joshua, 2022. "How rating mechanisms shape user search, quality inference and engagement in online platforms: Experimental evidence," Journal of Business Research, Elsevier, vol. 142(C), pages 791-807.
    5. Dominik Gutt & Jürgen Neumann & Steffen Zimmermann & Dennis Kundisch & Jianqing Chen, 2018. "Design of Review Systems - A Strategic Instrument to shape Online Review Behavior and Economic Outcomes," Working Papers Dissertations 42, Paderborn University, Faculty of Business Administration and Economics.
    6. Nikhil Garg & Ramesh Johari, 2021. "Designing Informative Rating Systems: Evidence from an Online Labor Market," Manufacturing & Service Operations Management, INFORMS, vol. 23(3), pages 589-605, May.
    7. Paul Belleflamme & Martin Peitz, 2018. "Inside the Engine Room of Digital Platforms: Reviews, Ratings, and Recommendations," Working Papers halshs-01714549, HAL.
    8. Greiff, Matthias & Paetzel, Fabian, 2020. "Information about average evaluations spurs cooperation: An experiment on noisy reputation systems," Journal of Economic Behavior & Organization, Elsevier, vol. 180(C), pages 334-356.
    9. Kaiwei Zhang & Xi Weng & Xienan Cheng, 2022. "Optimal Pricing Schemes in the Presence of Social Learning and Costly Reporting," Papers 2211.07362, arXiv.org, revised Dec 2023.
    10. Arslan Aziz & Hui Li & Rahul Telang, 2023. "The Consequences of Rating Inflation on Platforms: Evidence from a Quasi-Experiment," Information Systems Research, INFORMS, vol. 34(2), pages 590-608, June.
    11. Gary Bolton & Kevin Breuer & Ben Greiner & Axel Ockenfels, 2023. "Fixing feedback revision rules in online markets," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 32(2), pages 247-256, April.
    12. Weijia (Daisy) Dai & Ginger Jin & Jungmin Lee & Michael Luca, 2018. "Aggregation of consumer ratings: an application to Yelp.com," Quantitative Marketing and Economics (QME), Springer, vol. 16(3), pages 289-339, September.
    13. Christoph Carnehl & Maximilian Schaefer & André Stenzel & Kevin Ducbao Tran, 2022. "Value for Money and Selection: How Pricing Affects Airbnb Ratings," Working Papers 684, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    14. Apostolos Filippas & John J. Horton & Joseph M. Golden, 2022. "Reputation Inflation," Marketing Science, INFORMS, vol. 41(4), pages 733-745, July.
    15. Xiang Hui & Tobias J. Klein & Konrad Stahl, 2021. "When and Why Do Buyers Rate in Online Markets?," CRC TR 224 Discussion Paper Series crctr224_2021_267v1, University of Bonn and University of Mannheim, Germany.
    16. Georgios Zervas & Davide Proserpio & John W. Byers, 2021. "A first look at online reputation on Airbnb, where every stay is above average," Marketing Letters, Springer, vol. 32(1), pages 1-16, March.
    17. Kim, Jong Min & Lee, Eunkyung & Mariani, Marcello M., 2021. "The influence of launching mobile channels on online customer reviews," Journal of Business Research, Elsevier, vol. 137(C), pages 366-378.
    18. Sherry He & Brett Hollenbeck & Davide Proserpio, 2022. "The Market for Fake Reviews," Marketing Science, INFORMS, vol. 41(5), pages 896-921, September.
    19. Michael Luca & Oren Reshef, 2021. "The Effect of Price on Firm Reputation," Management Science, INFORMS, vol. 67(7), pages 4408-4419, July.
    20. Ginger Zhe Jin & Zhentong Lu & Xiaolu Zhou & Chunxiao Li, 2020. "The Effects of Government Licensing on E-commerce: Evidence from Alibaba," NBER Working Papers 27884, National Bureau of Economic Research, Inc.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormksc:v:42:y:2023:i:5:p:853-865. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.