IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2412.18337.html

The Value of AI-Generated Metadata for UGC Platforms: Evidence from a Large-scale Field Experiment

Author

Listed:
  • Xinyi Zhang
  • Chenshuo Sun
  • Renyu Zhang
  • Khim-Yong Goh

Abstract

AI-generated content (AIGC), such as advertisement copy, product descriptions, and social media posts, is becoming ubiquitous in business practices. However, the value of AI-generated metadata, such as titles, remains unclear on user-generated content (UGC) platforms. To address this gap, we conducted a large-scale field experiment on a leading short-video platform in Asia to provide about 1 million users access to AI-generated titles for their uploaded videos. Our findings show that the provision of AI-generated titles significantly boosted content consumption, increasing valid watches by 1.6% and watch duration by 0.9%. When producers adopted these titles, these increases jumped to 7.1% and 4.1%, respectively. This viewership-boost effect was largely attributed to the use of this generative AI (GAI) tool increasing the likelihood of videos having a title by 41.4%. The effect was more pronounced for groups more affected by metadata sparsity. Mechanism analysis revealed that AI-generated metadata improved user-video matching accuracy in the platform's recommender system. Interestingly, for a video for which the producer would have posted a title anyway, adopting the AI-generated title decreased its viewership on average, implying that AI-generated titles may be of lower quality than human-generated ones. However, when producers chose to co-create with GAI and significantly revised the AI-generated titles, the videos outperformed their counterparts with either fully AI-generated or human-generated titles, showcasing the benefits of human-AI co-creation. This study highlights the value of AI-generated metadata and human-AI metadata co-creation in enhancing user-content matching and content consumption for UGC platforms.

Suggested Citation

  • Xinyi Zhang & Chenshuo Sun & Renyu Zhang & Khim-Yong Goh, 2024. "The Value of AI-Generated Metadata for UGC Platforms: Evidence from a Large-scale Field Experiment," Papers 2412.18337, arXiv.org.
    • Handle: RePEc:arx:papers:2412.18337
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2412.18337
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jiankun Sun & Dennis J. Zhang & Haoyuan Hu & Jan A. Van Mieghem, 2022. "Predicting Human Discretion to Adjust Algorithmic Prescription: A Large-Scale Field Experiment in Warehouse Operations," Management Science, INFORMS, vol. 68(2), pages 846-865, February.
    2. Chiara Longoni & Andrea Bonezzi & Carey K Morewedge, 2019. "Resistance to Medical Artificial Intelligence," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 46(4), pages 629-650.
    3. Mikhail Lysyakov & Siva Viswanathan, 2023. "Threatened by AI: Analyzing Users’ Responses to the Introduction of AI in a Crowd-Sourcing Platform," Information Systems Research, INFORMS, vol. 34(3), pages 1191-1210, September.
    4. Tianshu Sun & Lanfei Shi & Siva Viswanathan & Elena Zheleva, 2019. "Motivating Effective Mobile App Adoptions: Evidence from a Large-Scale Randomized Field Experiment," Information Systems Research, INFORMS, vol. 30(2), pages 523-539, June.
    5. Ni Huang & Gordon Burtch & Bin Gu & Yili Hong & Chen Liang & Kanliang Wang & Dongpu Fu & Bo Yang, 2019. "Motivating User-Generated Content with Performance Feedback: Evidence from Randomized Field Experiments," Management Science, INFORMS, vol. 65(1), pages 327-345, January.
    6. Cole, Rebel A. & Sokolyk, Tatyana, 2018. "Debt financing, survival, and growth of start-up firms," Journal of Corporate Finance, Elsevier, vol. 50(C), pages 609-625.
    7. Maytal Saar-Tsechansky & Prem Melville & Foster Provost, 2009. "Active Feature-Value Acquisition," Management Science, INFORMS, vol. 55(4), pages 664-684, April.
    8. Xingyue (Luna) Zhang & James A. Dearden & Yuliang Yao, 2022. "Let them stay or let them go? Online retailer pricing strategy for managing stockouts," Production and Operations Management, Production and Operations Management Society, vol. 31(11), pages 4173-4190, November.
    9. Zhen Fang & Ming Fan & Apurva Jain, 2023. "Content proliferation and narrowcasting in the age of streaming media," Production and Operations Management, Production and Operations Management Society, vol. 32(10), pages 3295-3310, October.
    10. Ni Huang & Probal Mojumder & Tianshu Sun & Jinchi Lv & Joseph M. Golden, 2021. "Not Registered? Please Sign Up First: A Randomized Field Experiment on the Ex Ante Registration Request," Information Systems Research, INFORMS, vol. 32(3), pages 914-931, September.
    11. Scott C. Ellis & Shashank Rao & Dheeraj Raju & Thomas J. Goldsby, 2018. "RFID Tag Performance: Linking the Laboratory to the Field through Unsupervised Learning," Production and Operations Management, Production and Operations Management Society, vol. 27(10), pages 1834-1848, October.
    12. Gordon Burtch & Qinglai He & Yili Hong & Dokyun Lee, 2022. "How Do Peer Awards Motivate Creative Content? Experimental Evidence from Reddit," Management Science, INFORMS, vol. 68(5), pages 3488-3506, May.
    13. Zikun Ye & Dennis J. Zhang & Heng Zhang & Renyu Zhang & Xin Chen & Zhiwei Xu, 2023. "Cold Start to Improve Market Thickness on Online Advertising Platforms: Data-Driven Algorithms and Field Experiments," Management Science, INFORMS, vol. 69(7), pages 3838-3860, July.
    14. Ruomeng Cui & Jun Li & Dennis J. Zhang, 2020. "Reducing Discrimination with Reviews in the Sharing Economy: Evidence from Field Experiments on Airbnb," Management Science, INFORMS, vol. 66(3), pages 1071-1094, March.
    15. Jiaxu Peng & Jungpil Hahn & Ke-Wei Huang, 2023. "Handling Missing Values in Information Systems Research: A Review of Methods and Assumptions," Information Systems Research, INFORMS, vol. 34(1), pages 5-26, March.
    16. Saravanan Kesavan & Tarun Kushwaha, 2020. "Field Experiment on the Profit Implications of Merchants’ Discretionary Power to Override Data-Driven Decision-Making Tools," Management Science, INFORMS, vol. 66(11), pages 5182-5190, November.
    17. Martin Reisenbichler & Thomas Reutterer & David A. Schweidel & Daniel Dan, 2022. "Frontiers: Supporting Content Marketing with Natural Language Generation," Marketing Science, INFORMS, vol. 41(3), pages 441-452, May.
    18. Dandan Qiao & Shun-Yang Lee & Andrew B. Whinston & Qiang Wei, 2020. "Financial Incentives Dampen Altruism in Online Prosocial Contributions: A Study of Online Reviews," Information Systems Research, INFORMS, vol. 31(4), pages 1361-1375, December.
    19. Sriram Narayanan & Sridhar Balasubramanian & Jayashankar M. Swaminathan, 2009. "A Matter of Balance: Specialization, Task Variety, and Individual Learning in a Software Maintenance Environment," Management Science, INFORMS, vol. 55(11), pages 1861-1876, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Chenshuo Sun & Zijun Shi & Xiao Liu & Anindya Ghose, 2025. "The Effect of Voice AI on Digital Commerce," Information Systems Research, INFORMS, vol. 36(2), pages 1147-1166, June.
    2. Chenshuo Sun, 2025. "How Does Prepopulating Search Bars with Keywords Affect Online Consumer Behavior? A Field Experiment," Marketing Science, INFORMS, vol. 44(6), pages 1217-1231, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xinyu Li & Keongtae Kim, 2025. "Impacts of generative AI on user contributions: evidence from a coding Q &A platform," Marketing Letters, Springer, vol. 36(3), pages 577-591, September.
    2. Felipe Caro & Anna Sáez de Tejada Cuenca, 2023. "Believing in Analytics: Managers’ Adherence to Price Recommendations from a DSS," Manufacturing & Service Operations Management, INFORMS, vol. 25(2), pages 524-542, March.
    3. Siliang Tong & Nan Jia & Xueming Luo & Zheng Fang, 2021. "The Janus face of artificial intelligence feedback: Deployment versus disclosure effects on employee performance," Strategic Management Journal, Wiley Blackwell, vol. 42(9), pages 1600-1631, September.
    4. Blair Flicker, 2026. "Managerial Insight and “Optimal” Algorithms," Management Science, INFORMS, vol. 72(1), pages 128-147, January.
    5. Ruth Beer & Anyan Qi & Ignacio Rios, 2026. "Behavioral Externalities of Process Automation," Management Science, INFORMS, vol. 72(1), pages 575-593, January.
    6. Brüns, Jasper David & Meißner, Martin, 2024. "Do you create your content yourself? Using generative artificial intelligence for social media content creation diminishes perceived brand authenticity," Journal of Retailing and Consumer Services, Elsevier, vol. 79(C).
    7. Anuj Kapoor & Madhav Kumar, 2025. "Frontiers: Generative AI and Personalized Video Advertisements," Marketing Science, INFORMS, vol. 44(4), pages 733-747, July.
    8. Rob Glew & Claire Senot, 2025. "Hold Me Accountable: Anonymity and Prosocial Behavior in Services," Manufacturing & Service Operations Management, INFORMS, vol. 27(4), pages 1068-1086, July.
    9. Clare Snyder & Samantha Keppler & Stephen Leider, 2026. "Algorithm Reliance: Fast and Slow," Management Science, INFORMS, vol. 72(1), pages 368-385, January.
    10. Mehmet U. S. Ayvaci & Zahra Mobini & Özalp Özer, 2026. "It Takes Two to Make It Right: How Nurses’ Response to Sepsis Alerts Impacts Physicians’ Process Compliance," Manufacturing & Service Operations Management, INFORMS, vol. 28(1), pages 20-41, January.
    11. Sulin Ba & Shu He & Shun‐Yang Lee, 2022. "Mobile App Adoption and Its Differential Impact on Consumer Shopping Behavior," Production and Operations Management, Production and Operations Management Society, vol. 31(2), pages 764-780, February.
    12. Maria Marchenko & Hendrik Sonnabend, 2025. "The never-ending book: the role of new material and peer feedback in user-generated content production," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 49(3), pages 435-458, September.
    13. Zhiyu Zeng & Hengchen Dai & Dennis J. Zhang & Heng Zhang & Renyu Zhang & Zhiwei Xu & Zuo-Jun Max Shen, 2023. "The Impact of Social Nudges on User-Generated Content for Social Network Platforms," Management Science, INFORMS, vol. 69(9), pages 5189-5208, September.
    14. Evgeny Kagan & Brett Hathaway & Maqbool Dada, 2025. "Deploying Chatbots in Customer Service: Adoption Hurdles and Simple Remedies," Papers 2504.06145, arXiv.org.
    15. Matthew DosSantos DiSorbo & Kris Johnson Ferreira & Maya Balakrishnan & Jordan Tong, 2025. "Warnings and Endorsements: Improving Human–AI Collaboration in the Presence of Outliers," Manufacturing & Service Operations Management, INFORMS, vol. 27(6), pages 1814-1831, November.
    16. Sebastian Krakowski & Darek Haftor & Johannes Luger & Natallia Pashkevich & Sebastian Raisch, 2026. "Human-Centered Artificial Intelligence: A Field Experiment," Management Science, INFORMS, vol. 72(1), pages 57-72, January.
    17. Zhiqi Zhang & Zhiyu Zeng & Ruohan Zhan & Dennis Zhang, 2026. "Personalized Policy Learning through Discrete Experimentation: Theory and Empirical Evidence," Papers 2602.05099, arXiv.org.
    18. Yipu Deng & Jinyang Zheng & Warut Khern-am-nuai & Karthik Kannan, 2022. "More than the Quantity: The Value of Editorial Reviews for a User-Generated Content Platform," Management Science, INFORMS, vol. 68(9), pages 6865-6888, September.
    19. Gloria Urrea & Eunae Yoo, 2023. "The role of volunteer experience on performance on online volunteering platforms," Production and Operations Management, Production and Operations Management Society, vol. 32(2), pages 416-433, February.
    20. Yingda Lu & Youwei Wang & Yuxin Chen & Yun Xiong, 2023. "The role of data‐based intelligence and experience on time efficiency of taxi drivers: An empirical investigation using large‐scale sensor data," Production and Operations Management, Production and Operations Management Society, vol. 32(11), pages 3665-3682, November.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2412.18337. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.