IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2503.16458.html

Users Favor LLM-Generated Content -- Until They Know It's AI

Author

Listed:
  • Petr Parshakov
  • Iuliia Naidenova
  • Sofia Paklina
  • Nikita Matkin
  • Cornel Nesseler

Abstract

In this paper, we investigate how individuals evaluate human and large langue models generated responses to popular questions when the source of the content is either concealed or disclosed. Through a controlled field experiment, participants were presented with a set of questions, each accompanied by a response generated by either a human or an AI. In a randomized design, half of the participants were informed of the response's origin while the other half remained unaware. Our findings indicate that, overall, participants tend to prefer AI-generated responses. However, when the AI origin is revealed, this preference diminishes significantly, suggesting that evaluative judgments are influenced by the disclosure of the response's provenance rather than solely by its quality. These results underscore a bias against AI-generated content, highlighting the societal challenge of improving the perception of AI work in contexts where quality assessments should be paramount.

Suggested Citation

  • Petr Parshakov & Iuliia Naidenova & Sofia Paklina & Nikita Matkin & Cornel Nesseler, 2025. "Users Favor LLM-Generated Content -- Until They Know It's AI," Papers 2503.16458, arXiv.org.
    • Handle: RePEc:arx:papers:2503.16458
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2503.16458
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Chen, Yaqi & Wang, Haizhong & Rao Hill, Sally & Li, Binglian, 2024. "Consumer attitudes toward AI-generated ads: Appeal types, self-efficacy and AI’s social role," Journal of Business Research, Elsevier, vol. 185(C).
    2. Melania Nitu & Mihai Dascalu, 2024. "Beyond Lexical Boundaries: LLM-Generated Text Detection for Romanian Digital Libraries," Future Internet, MDPI, vol. 16(2), pages 1-31, January.
    3. Michelle Vaccaro & Abdullah Almaatouq & Thomas Malone, 2024. "When combinations of humans and AI are useful: A systematic review and meta-analysis," Nature Human Behaviour, Nature, vol. 8(12), pages 2293-2303, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. EuiBeom Jeong & DonHee Lee, 2025. "AI–human collaboration in services: an integrative framework to uncover the key success factors," Service Business, Springer;Pan-Pacific Business Association, vol. 19(3), pages 1-45, September.
    2. Hou, Chenxuan & Li, Tingting & Gu, Yanzhang, 2026. "Artificial intelligence versus human providers for personalized solutions? The influence of expected group size and perceived uniqueness on adoption intentions," Journal of Retailing and Consumer Services, Elsevier, vol. 88(C).
    3. Kumar, Aman & Shankar, Amit & Hollebeek, Linda D. & Behl, Abhishek & Lim, Weng Marc, 2025. "Generative artificial intelligence (GenAI) revolution: A deep dive into GenAI adoption," Journal of Business Research, Elsevier, vol. 189(C).
    4. Chen, Qin & Wang, Hongchuan & Ma, Chengcheng & Ru, Peng, 2025. "How professionals respond to disruptive effects of artificial intelligence on their jurisdiction: The role of interactive governance," Social Science & Medicine, Elsevier, vol. 386(C).
    5. Feiyang Xu & Poonacha K. Medappa & Murat M. Tunc & Martijn Vroegindeweij & Jan C. Fransoo, 2025. "AI-Assisted Programming Decreases the Productivity of Experienced Developers by Increasing the Technical Debt and Maintenance Burden," Papers 2510.10165, arXiv.org, revised Jan 2026.
    6. Lagziel, David & Tsodikovich, Yevgeny, 2025. "Working with AI: An analysis for rational integration," Games and Economic Behavior, Elsevier, vol. 153(C), pages 254-267.
    7. Jung, Timothy & Koghut, Maksym & Lee, Eunseo & Kwon, Ohbyung, 2025. "Artificial creativity in luxury advertising: How trust and perceived humanness drive consumer response to AI-generated content," Journal of Retailing and Consumer Services, Elsevier, vol. 87(C).
    8. Lei, Shaohui & Xie, Lishan, 2025. "“Servant” versus “Partner”: Investigating the effect of service robot personas on customer misbehavior," Journal of Business Research, Elsevier, vol. 199(C).
    9. Serra-Simón, Jordi & Puntí-Brun, Mònica & Espinosa-Mirabet, Sílvia & de Faria Nogueira, Maria Alice & Martín-Guart, Ramón & Tôrres de Azevedo, Sandro, 2025. "Generative artificial intelligence in advertising. Field applications in Rio de Janeiro and Catalonia," Telecommunications Policy, Elsevier, vol. 49(8).
    10. Mario Passalacqua & Robert Pellerin & Florian Magnani & Laurent Joblot & Frédéric Rosin & Esma Yahia & Pierre-Majorique Léger, 2025. "Safeguarding worker psychosocial well-being in the age of AI: The critical role of decision control," Post-Print hal-05345071, HAL.
    11. L. Elisa Celis & Lingxiao Huang & Nisheeth K. Vishnoi, 2025. "A Mathematical Framework for AI-Human Integration in Work," Papers 2505.23432, arXiv.org, revised May 2025.
    12. Foucart, Renaud & Zeng, Fanqi & Wang, Shidong, 2025. "The Social Importance of Being Stubborn When an Organization Meets AI," SocArXiv nfgy3_v1, Center for Open Science.
    13. Iulia POSTOLACHI & Irina-Constanța THEODORU, 2026. "Embracing Hybrid Intelligence: An Exploratory Study on AI Adoption in Content Creation Among Romanian Marketing Professionals," Journal of Emerging Trends in Marketing and Management, The Bucharest University of Economic Studies, vol. 1(1), pages 7-20, March.
    14. Puchu Zhao & Guohua He & Jian Guan, 2026. "The Ethical Costs of Artificial Intelligence: Investigating How and When Workplace Artificial Intelligence Usage Promotes Employee Unethical Outcomes," Journal of Business Ethics, Springer, vol. 203(3), pages 531-547, January.
    15. Spiegler, Simone & Hoda, Rashina & Pant, Aastha, 2026. "Images of AI: How AI practitioners view the impact of Artificial Intelligence on society, now and in the future," Technology in Society, Elsevier, vol. 84(C).
    16. Carmen Acatrinei & Claudia Maria Miu, 2025. "AI vs. Human-Written Email Marketing: Consumer Insights for Sustainable Communication," The AMFITEATRU ECONOMIC journal, Academy of Economic Studies - Bucharest, Romania, vol. 27(S19), pages 1256-1256, November.
    17. Egana-delSol, Pablo & Bravo-Ortega, Claudio, 2025. "Artificial Intelligence and Labor Market Transformations in Latin America," IZA Discussion Papers 17746, IZA Network @ LISER.
    18. Vu Minh Ngo, 2025. "Human–AI collaboration for marketing capabilities: a meta-analysis," Marketing Letters, Springer, vol. 36(4), pages 839-855, December.
    19. Wang, Yong & Sun, Xiaoxin & Zhang, Xiaochen & Shi, Haijiao, 2025. "Do you feel empowered by AI service robot? An exploration of consumer's social power perception in human-AI interaction," Journal of Retailing and Consumer Services, Elsevier, vol. 87(C).
    20. Jaroslaw Kornowicz & Kirsten Thommes, 2025. "Algorithm, expert, or both? Evaluating the role of feature selection methods on user preferences and reliance," PLOS ONE, Public Library of Science, vol. 20(3), pages 1-22, March.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2503.16458. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.