IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2511.09877.html

AI in the Pen: How Real-time AI Writing Guidance Shapes Online Reviews

Author

Listed:
  • Fangyan Wang
  • Sai Liang
  • Zaiyan Wei

Abstract

Recent advancements in artificial intelligence (AI) are reshaping user-generated content (UGC). Online reviews, an important form of UGC, exert significant influence on consumer decisions and business reputation. Yet, the impacts of AI-powered tools employed during review generation remain underexplored. We examine Yelp's AI writing guidance adopted in April 2023, a novel human-AI collaboration in content creation. Drawing on Self-Determination Theory, we evaluate the impacts of this intervention on review outcomes. Our findings reveal that, first, the AI guidance was effective in shaping topical composition, encouraging broader coverage across aspects of the dining experience. Second, the AI writing guidance increases review length while reducing overall review volume, revealing a trade-off between depth and productivity. Third, the expansion of topic coverage comes at the cost of increased textual complexity and reduced readability, ultimately diminishing the number of helpfulness votes. Lastly, effects are heterogeneous: less experienced users expand length and topic coverage more, whereas experienced users exhibit greater complexity and sharper declines in perceived helpfulness. These findings advance our understanding of AI-powered tools during content generation. We also discuss implications for platforms aiming to leverage AI in managing online content and for other stakeholders involved in the production and consumption of online content.

Suggested Citation

  • Fangyan Wang & Sai Liang & Zaiyan Wei, 2025. "AI in the Pen: How Real-time AI Writing Guidance Shapes Online Reviews," Papers 2511.09877, arXiv.org.
    • Handle: RePEc:arx:papers:2511.09877
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2511.09877
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Abhay Aneja & Michael Luca & Oren Reshef, 2025. "The Benefits of Revealing Race: Evidence from Minority-Owned Local Businesses," American Economic Review, American Economic Association, vol. 115(2), pages 660-689, February.
    2. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    3. Samuel G. Goldberg & Garrett A. Johnson & Scott K. Shriver, 2024. "Regulating Privacy Online: An Economic Evaluation of the GDPR," American Economic Journal: Economic Policy, American Economic Association, vol. 16(1), pages 325-358, February.
    4. Yi-Jen (Ian) Ho & Siyuan Liu & Lei Wang, 2023. "Fun Shopping: A Randomized Field Experiment on Gamification," Information Systems Research, INFORMS, vol. 34(2), pages 766-785, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gabriele Ruiu & Giovanna Gonano, 2020. "Religious Barriers to the Diffusion of Same-sex Civil Unions in Italy," Population Research and Policy Review, Springer;Southern Demographic Association (SDA), vol. 39(6), pages 1185-1203, December.
    2. Wright, Austin L. & Sonin, Konstantin & Driscoll, Jesse & Wilson, Jarnickae, 2020. "Poverty and economic dislocation reduce compliance with COVID-19 shelter-in-place protocols," Journal of Economic Behavior & Organization, Elsevier, vol. 180(C), pages 544-554.
    3. Guido de Blasio & Daniela Vuri, 2019. "Effects of the Joint Custody Law in Italy," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 16(3), pages 479-514, September.
    4. Graves Jennifer & McMullen Steven & Rouse Kathryn, 2018. "Teacher Turnover, Composition and Qualifications in the Year-Round School Setting," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 18(3), pages 1-27, July.
    5. Alston Lee J. & Mueller Bernardo, 2018. "Priests, Conflicts and Property Rights: the Impacts on Tenancy and Land Use in Brazil," Man and the Economy, De Gruyter, vol. 5(1), pages 1-26, June.
    6. S Anukriti & Catalina Herrera‐Almanza & Praveen K. Pathak & Mahesh Karra, 2020. "Curse of the Mummy‐ji: The Influence of Mothers‐in‐Law on Women in India†," American Journal of Agricultural Economics, John Wiley & Sons, vol. 102(5), pages 1328-1351, October.
    7. Ellison, Richard B. & Ellison, Adrian B. & Greaves, Stephen P. & Sampaio, Breno, 2017. "Electronic ticketing systems as a mechanism for travel behaviour change? Evidence from Sydney’s Opal card," Transportation Research Part A: Policy and Practice, Elsevier, vol. 99(C), pages 80-93.
    8. Cruzatti C., John & Bjørnskov, Christian & Sáenz de Viteri, Andrea & Cruzatti, Christian, 2024. "Geography, development, and power: Parliament leaders and local clientelism," World Development, Elsevier, vol. 182(C).
    9. Yusuke Matsuki, 2016. "A Distribution-Free Test of Monotonicity with an Application to Auctions," Working Papers e110, Tokyo Center for Economic Research.
    10. Shvartsman, Elena & Beckmann, Michael, 2015. "Stressed by your job: What is the role of personnel policy?," Working papers 2015/15, Faculty of Business and Economics - University of Basel.
    11. MacDonald, Peter, 2013. "Labour substitution and the scope for military outsourcing," MPRA Paper 46688, University Library of Munich, Germany.
    12. Matteo Migheli, 2021. "Green purchasing: the effect of parenthood and gender," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 23(7), pages 10576-10600, July.
    13. Yuyang Li & Jiahui Li & Xinjie Li & Qian Lu, 2024. "Does Participation in Digital Supply and Marketing Promote Smallholder Farmers’ Adoption of Green Agricultural Production Technologies?," Land, MDPI, vol. 14(1), pages 1-24, December.
    14. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    15. Sondang Marsinta Uli Panggabean & Mahjus Ekananda & Beta Yulianita Gitaharie & Leslie Djuranovik, 2025. "Export proceeds repatriation policies: A shield against exchange rate volatility in emerging markets?," Papers 2506.09168, arXiv.org.
    16. Laurent Didier, 2017. "South-South Trade and Geographical Diversification of Intra-SSA Trade: Evidence from BRICs," African Development Review, African Development Bank, vol. 29(2), pages 139-154, June.
    17. Bahar, Dany & Rosenow, Samuel & Stein, Ernesto & Wagner, Rodrigo, 2019. "Export take-offs and acceleration: Unpacking cross-sector linkages in the evolution of comparative advantage," World Development, Elsevier, vol. 117(C), pages 48-60.
    18. Marie Bjørneby & Annette Alstadsæter & Kjetil Telle, 2018. "Collusive tax evasion by employers and employees. Evidence from a randomized fi eld experiment in Norway," Discussion Papers 891, Statistics Norway, Research Department.
    19. Gary Goertz & Tony Hak & Jan Dul, 2013. "Ceilings and Floors," Sociological Methods & Research, , vol. 42(1), pages 3-40, February.
    20. Maida, Agata & Pezone, Vincenzo, 2024. "CEO Pay Disclosure and Within-Firm Wage Inequality," IZA Discussion Papers 17243, Institute of Labor Economics (IZA).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2511.09877. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.