IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2503.16458.html

Users Favor LLM-Generated Content -- Until They Know It's AI

Author

Listed:
  • Petr Parshakov
  • Iuliia Naidenova
  • Sofia Paklina
  • Nikita Matkin
  • Cornel Nesseler

Abstract

In this paper, we investigate how individuals evaluate human and large langue models generated responses to popular questions when the source of the content is either concealed or disclosed. Through a controlled field experiment, participants were presented with a set of questions, each accompanied by a response generated by either a human or an AI. In a randomized design, half of the participants were informed of the response's origin while the other half remained unaware. Our findings indicate that, overall, participants tend to prefer AI-generated responses. However, when the AI origin is revealed, this preference diminishes significantly, suggesting that evaluative judgments are influenced by the disclosure of the response's provenance rather than solely by its quality. These results underscore a bias against AI-generated content, highlighting the societal challenge of improving the perception of AI work in contexts where quality assessments should be paramount.

Suggested Citation

  • Petr Parshakov & Iuliia Naidenova & Sofia Paklina & Nikita Matkin & Cornel Nesseler, 2025. "Users Favor LLM-Generated Content -- Until They Know It's AI," Papers 2503.16458, arXiv.org.
    • Handle: RePEc:arx:papers:2503.16458
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2503.16458
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Chen, Yaqi & Wang, Haizhong & Rao Hill, Sally & Li, Binglian, 2024. "Consumer attitudes toward AI-generated ads: Appeal types, self-efficacy and AI’s social role," Journal of Business Research, Elsevier, vol. 185(C).
    2. Melania Nitu & Mihai Dascalu, 2024. "Beyond Lexical Boundaries: LLM-Generated Text Detection for Romanian Digital Libraries," Future Internet, MDPI, vol. 16(2), pages 1-31, January.
    3. Michelle Vaccaro & Abdullah Almaatouq & Thomas Malone, 2024. "When combinations of humans and AI are useful: A systematic review and meta-analysis," Nature Human Behaviour, Nature, vol. 8(12), pages 2293-2303, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. EuiBeom Jeong & DonHee Lee, 2025. "AI–human collaboration in services: an integrative framework to uncover the key success factors," Service Business, Springer;Pan-Pacific Business Association, vol. 19(3), pages 1-45, September.
    2. Hou, Chenxuan & Li, Tingting & Gu, Yanzhang, 2026. "Artificial intelligence versus human providers for personalized solutions? The influence of expected group size and perceived uniqueness on adoption intentions," Journal of Retailing and Consumer Services, Elsevier, vol. 88(C).
    3. Kumar, Aman & Shankar, Amit & Hollebeek, Linda D. & Behl, Abhishek & Lim, Weng Marc, 2025. "Generative artificial intelligence (GenAI) revolution: A deep dive into GenAI adoption," Journal of Business Research, Elsevier, vol. 189(C).
    4. Kai Feng & Han Hong & Ke Tang & Jingyuan Wang, 2025. "Statistical Tests for Replacing Human Decision Makers with Algorithms," Management Science, INFORMS, vol. 71(11), pages 9145-9170, November.
    5. Chen, Qin & Wang, Hongchuan & Ma, Chengcheng & Ru, Peng, 2025. "How professionals respond to disruptive effects of artificial intelligence on their jurisdiction: The role of interactive governance," Social Science & Medicine, Elsevier, vol. 386(C).
    6. Feiyang Xu & Poonacha K. Medappa & Murat M. Tunc & Martijn Vroegindeweij & Jan C. Fransoo, 2025. "AI-Assisted Programming Decreases the Productivity of Experienced Developers by Increasing the Technical Debt and Maintenance Burden," Papers 2510.10165, arXiv.org, revised Jan 2026.
    7. Yang, Zhen & Tian, Allen Ding, 2026. "Designer-consumer similarity matters: The effect of AI-designed products on purchase intention," Journal of Retailing and Consumer Services, Elsevier, vol. 90(C).
    8. Fabrizio Dell’Acqua & Edward McFowland & Ethan Mollick & Hila Lifshitz & Katherine C. Kellogg & Saran Rajendran & Lisa Krayer & François Candelon & Karim R. Lakhani, 2026. "Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality," Organization Science, INFORMS, vol. 37(2), pages 403-423, March.
    9. Lagziel, David & Tsodikovich, Yevgeny, 2025. "Working with AI: An analysis for rational integration," Games and Economic Behavior, Elsevier, vol. 153(C), pages 254-267.
    10. Jung, Timothy & Koghut, Maksym & Lee, Eunseo & Kwon, Ohbyung, 2025. "Artificial creativity in luxury advertising: How trust and perceived humanness drive consumer response to AI-generated content," Journal of Retailing and Consumer Services, Elsevier, vol. 87(C).
    11. Lei, Shaohui & Xie, Lishan, 2025. "“Servant” versus “Partner”: Investigating the effect of service robot personas on customer misbehavior," Journal of Business Research, Elsevier, vol. 199(C).
    12. Serra-Simón, Jordi & Puntí-Brun, Mònica & Espinosa-Mirabet, Sílvia & de Faria Nogueira, Maria Alice & Martín-Guart, Ramón & Tôrres de Azevedo, Sandro, 2025. "Generative artificial intelligence in advertising. Field applications in Rio de Janeiro and Catalonia," Telecommunications Policy, Elsevier, vol. 49(8).
    13. Mario Passalacqua & Robert Pellerin & Florian Magnani & Laurent Joblot & Frédéric Rosin & Esma Yahia & Pierre-Majorique Léger, 2025. "Safeguarding worker psychosocial well-being in the age of AI: The critical role of decision control," Post-Print hal-05345071, HAL.
    14. Qin, Meilian & Jiang, Kan & Bao, Xiaoning, 2026. "Falsity images also have real benefits? The impact of AI-generated advertisements' visual styles and reach methods on consumer consultation intentions," Journal of Retailing and Consumer Services, Elsevier, vol. 91(C).
    15. L. Elisa Celis & Lingxiao Huang & Nisheeth K. Vishnoi, 2025. "A Mathematical Framework for AI-Human Integration in Work," Papers 2505.23432, arXiv.org, revised May 2025.
    16. Kim, Songmee & Cho, Erin, 2026. "How AI involvement affects curiosity and content engagement in AI-generated advertisements," Journal of Retailing and Consumer Services, Elsevier, vol. 92(C).
    17. Pattaraphon Kenny Wongchamcharoen & Kris Gulati & Min Min Fong & Abhishek Nagaraj, 2026. "CentaurBench: Benchmarking LLM Capabilities on Augmenting vs. Automating Real-World Work Tasks," Papers 2608.18554, arXiv.org.
    18. Cao, Zhongpeng & Zhao, Liben & Su, Tong, 2026. "Does it matter who owns the AI device? Impact of device ownership on customer dissatisfaction in AI service failure," Journal of Retailing and Consumer Services, Elsevier, vol. 90(C).
    19. Liu, Manzhi & Wang, Lijing, 2026. "Beyond the code: How algorithmic creativity enhances tolerance to recommendation bias?," Journal of Retailing and Consumer Services, Elsevier, vol. 91(C).
    20. Foucart, Renaud & Zeng, Fanqi & Wang, Shidong, 2025. "The Social Importance of Being Stubborn When an Organization Meets AI," SocArXiv nfgy3_v1, Center for Open Science.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2503.16458. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.