IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2606.08265.html

Unintended Consequences of Recommender System Interventions: Evidence from a Field Experiment

Author

Listed:
  • Shilei Luo
  • Song Yao
  • Dennis J. Zhang

Abstract

Platform content interventions in recommendation systems are typically evaluated as static "nudges", ignoring that the systems adaptively learn from the resulting user behavior. We investigate this dynamic through a large-scale field experiment on a short-video platform. The experiment involves a "sleep reminder" campaign designed to reduce late-night usage. Paradoxically, the intervention increased late-night engagement by 14.75% and overall platform usage by 2.18%, and the effects persisted for weeks even after the experiment. We explain this through a forced-exploration mechanism, showing that by revealing high latent demand for the promoted content, the intervention triggers a recommendation policy update that routine user behavior would not produce. The data generated by the intervention induced the algorithm to update its post-campaign policy, reinforcing the very engagement loops the campaign aimed to mitigate. Our findings demonstrate that user-facing interventions can effectively retrain the underlying algorithm, triggering durable, system-wide shifts in content distribution that challenge standard evaluation metrics in platform governance and social responsibility initiatives.

Suggested Citation

  • Shilei Luo & Song Yao & Dennis J. Zhang, 2026. "Unintended Consequences of Recommender System Interventions: Evidence from a Field Experiment," Papers 2606.08265, arXiv.org.
  • Handle: RePEc:arx:papers:2606.08265
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2606.08265
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Duncan Simester & Artem Timoshenko & Spyros I. Zoumpoulis, 2020. "Efficiently Evaluating Targeting Policies: Improving on Champion vs. Challenger Experiments," Management Science, INFORMS, vol. 66(8), pages 3412-3424, August.
    2. Yuxin Chen & Song Yao, 2017. "Sequential Search with Refinement: Model and Application with Click-Stream Data," Management Science, INFORMS, vol. 63(12), pages 4345-4365, December.
    3. Anindya Ghose & Panagiotis G. Ipeirotis & Beibei Li, 2014. "Examining the Impact of Ranking on Consumer Behavior and Search Engine Revenue," Management Science, INFORMS, vol. 60(7), pages 1632-1654, July.
    4. Ron Berman & Zsolt Katona, 2020. "Curation Algorithms and Filter Bubbles in Social Networks," Marketing Science, INFORMS, vol. 39(2), pages 296-316, March.
    5. Meng Liu & Xiaocheng Tang & Siyuan Xia & Shuo Zhang & Yuting Zhu & Qianying Meng, 2026. "Algorithm Aversion: Evidence from Ridesharing Drivers," Management Science, INFORMS, vol. 72(1), pages 193-203, January.
    6. Thomas Blake & Chris Nosko & Steven Tadelis, 2015. "Consumer Heterogeneity and Paid Search Effectiveness: A Large‐Scale Field Experiment," Econometrica, Econometric Society, vol. 83, pages 155-174, January.
    7. Dokyun Lee & Kartik Hosanagar, 2019. "How Do Recommender Systems Affect Sales Diversity? A Cross-Category Investigation via Randomized Field Experiment," Service Science, INFORMS, vol. 30(1), pages 239-259, March.
    8. Lu, Shijie & Xie, Ying & Chen, Xingyu, 2023. "Immediate and enduring effects of digital badges on online content consumption and generation," International Journal of Research in Marketing, Elsevier, vol. 40(1), pages 146-163.
    9. Liran Einav & Chiara Farronato & Jonathan Levin, 2016. "Peer-to-Peer Markets," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 615-635, October.
    10. Xitong Li & Jörn Grahl & Oliver Hinz, 2022. "How Do Recommender Systems Lead to Consumer Purchases? A Causal Mediation Analysis of a Field Experiment," Information Systems Research, INFORMS, vol. 33(2), pages 620-637, June.
    11. Zhe Yuan & AJ Yuan Chen & Yitong Wang & Tianshu Sun, 2025. "How Recommendation Affects Customer Search: A Field Experiment," Information Systems Research, INFORMS, vol. 36(1), pages 84-106, March.
    12. Christian Peukert & Ananya Sen & Jörg Claussen, 2024. "The Editor and the Algorithm: Recommendation Technology in Online News," Management Science, INFORMS, vol. 70(9), pages 5816-5831, September.
    13. Andrew Caplin & John Leahy, 1991. "State-Dependent Pricing and the Dynamics of Money and Output," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 106(3), pages 683-708.
    14. Daniel Fleder & Kartik Hosanagar, 2009. "Blockbuster Culture's Next Rise or Fall: The Impact of Recommender Systems on Sales Diversity," Management Science, INFORMS, vol. 55(5), pages 697-712, May.
    15. Brett R. Gordon & Florian Zettelmeyer & Neha Bhargava & Dan Chapsky, 2019. "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science, INFORMS, vol. 38(2), pages 193-225, March.
    16. Robert Donnelly & Ayush Kanodia & Ilya Morozov, 2024. "Welfare Effects of Personalized Rankings," Marketing Science, INFORMS, vol. 43(1), pages 92-113, January.
    17. Z. Eddie Ning & Jiwoong Shin & Jungju Yu, 2025. "Targeted Advertising as Implicit Recommendation: Strategic Mistargeting and Personal Data Opt-out," Marketing Science, INFORMS, vol. 44(2), pages 390-410, March.
    18. Hema Yoganarasimhan, 2020. "Search Personalization Using Machine Learning," Management Science, INFORMS, vol. 66(3), pages 1045-1070, March.
    19. Shunyuan Zhang & Nitin Mehta & Param Vir Singh & Kannan Srinivasan, 2021. "Frontiers: Can an Artificial Intelligence Algorithm Mitigate Racial Economic Inequality? An Analysis in the Context of Airbnb," Marketing Science, INFORMS, vol. 40(5), pages 813-820, September.
    20. Ganesh Iyer & T. Tony Ke, 2024. "Competitive Model Selection in Algorithmic Targeting," Marketing Science, INFORMS, vol. 43(6), pages 1226-1241, November.
    21. Anja Lambrecht & Catherine Tucker, 2019. "Algorithmic Bias? An Empirical Study of Apparent Gender-Based Discrimination in the Display of STEM Career Ads," Management Science, INFORMS, vol. 65(7), pages 2966-2981, July.
    22. Jūra Liaukonytė & Anna Tuchman & Xinrong Zhu, 2023. "Frontiers: Spilling the Beans on Political Consumerism: Do Social Media Boycotts and Buycotts Translate to Real Sales Impact?," Marketing Science, INFORMS, vol. 42(1), pages 11-25, January.
    23. Bryan Bollinger & Kenneth Gillingham, 2012. "Peer Effects in the Diffusion of Solar Photovoltaic Panels," Marketing Science, INFORMS, vol. 31(6), pages 900-912, November.
    24. David Godes & Dina Mayzlin, 2009. "Firm-Created Word-of-Mouth Communication: Evidence from a Field Test," Marketing Science, INFORMS, vol. 28(4), pages 721-739, 07-08.
    25. Bradley T. Shapiro & Günter J. Hitsch & Anna E. Tuchman, 2021. "TV Advertising Effectiveness and Profitability: Generalizable Results From 288 Brands," Econometrica, Econometric Society, vol. 89(4), pages 1855-1879, July.
    26. Ganesh Iyer & Zemin (Zachary) Zhong, 2022. "Pushing Notifications as Dynamic Information Design," Marketing Science, INFORMS, vol. 41(1), pages 51-72, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tianshu Sun & Zhe Yuan & Chunxiao Li & Kaifu Zhang & Jun Xu, 2024. "The Value of Personal Data in Internet Commerce: A High-Stakes Field Experiment on Data Regulation Policy," Management Science, INFORMS, vol. 70(4), pages 2645-2660, April.
    2. Wei Zhou & Mingfeng Lin & Mo Xiao & Lu Fang, 2025. "Higher Precision Is Not Always Better: Search Algorithm and Consumer Engagement," Management Science, INFORMS, vol. 71(7), pages 6204-6226, July.
    3. Kevin Zielnicki & Guy Aridor & Aur'elien Bibaut & Allen Tran & Winston Chou & Nathan Kallus, 2025. "The Value of Personalized Recommendations: Evidence from Netflix," Papers 2511.07280, arXiv.org, revised Jun 2026.
    4. Chenshuo Sun, 2025. "How Does Prepopulating Search Bars with Keywords Affect Online Consumer Behavior? A Field Experiment," Marketing Science, INFORMS, vol. 44(6), pages 1217-1231, November.
    5. Shuang Zheng & Siliang (Jack) Tong & Hyeokkoo Eric Kwon & Gordon Burtch & Xianneng Li, 2025. "Frontiers: Recommending What to Search: Sales Volume and Consumption Diversity Effects of a Query Recommender System," Marketing Science, INFORMS, vol. 44(3), pages 516-524, May.
    6. repec:ces:ceswps:_12257 is not listed on IDEAS
    7. Bleier, Alexander & Fossen, Beth L. & Shapira, Michal, 2024. "On the role of social media platforms in the creator economy," International Journal of Research in Marketing, Elsevier, vol. 41(3), pages 411-426.
    8. Miguel Godinho de Matos & Pedro Ferreira, 2020. "The Effect of Binge-Watching on the Subscription of Video on Demand: Results from Randomized Experiments," Information Systems Research, INFORMS, vol. 31(4), pages 1337-1360, December.
    9. Raluca Mihaela Ursu & Andrey Simonov & Eunkyung An, 2025. "Online Advertising as Passive Search," Management Science, INFORMS, vol. 71(2), pages 1050-1073, February.
    10. Wei Zhou & Zidong Wang, 2020. "Competing for Search Traffic in Query Markets: Entry Strategy, Platform Design, and Entrepreneurship," Working Papers 20-12, NET Institute.
    11. Yuyan Wang & Long Tao & Xian Xing Zhang, 2025. "Recommending for a Multi-Sided Marketplace: A Multi-Objective Hierarchical Approach," Marketing Science, INFORMS, vol. 44(1), pages 1-29, January.
    12. Joan Calzada & Nestor Duch-Brown & Ricard Gil, 2021. "Do search engines increase concentration in media markets?," UB School of Economics Working Papers 2021/415, University of Barcelona School of Economics.
    13. Kai Zhu & Qiaoni Shi & Shrabastee Banerjee, 2026. "Monetizing Platforms: An Empirical Analysis of Supply and Demand Responses to Entry Costs in Two-Sided Markets," Management Science, INFORMS, vol. 72(3), pages 2380-2397, March.
    14. Lanfei Shi & Jin Liu & Yongjun Li & Natasha Zhang Foutz, 2025. "Ephemeral State-Dependent Recommendation for Digital Content," Information Systems Research, INFORMS, vol. 36(4), pages 2344-2357, December.
    15. Weijia Dai & Hyunjin Kim & Michael Luca, 2023. "Frontiers: Which Firms Gain from Digital Advertising? Evidence from a Field Experiment," Marketing Science, INFORMS, vol. 42(3), pages 429-439, May.
    16. Brett R. Gordon & Robert Moakler & Florian Zettelmeyer, 2023. "Close Enough? A Large-Scale Exploration of Non-Experimental Approaches to Advertising Measurement," Marketing Science, INFORMS, vol. 42(4), pages 768-793, July.
    17. Xiang (Shawn) Wan & Anuj Kumar & Xitong Li, 2024. "How Do Product Recommendations Help Consumers Search? Evidence from a Field Experiment," Management Science, INFORMS, vol. 70(9), pages 5776-5794, September.
    18. Hsing Kenneth Cheng & D. Daniel Sokol & Xinyu Zang, 2024. "The rise of empirical online platform research in the new millennium," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 33(2), pages 416-451, March.
    19. Nicolas Padilla & Eva Ascarza & Oded Netzer, 2025. "The customer journey as a source of information," Quantitative Marketing and Economics (QME), Springer, vol. 23(3), pages 379-418, September.
    20. Zhe Yuan & AJ Yuan Chen & Yitong Wang & Tianshu Sun, 2025. "How Recommendation Affects Customer Search: A Field Experiment," Information Systems Research, INFORMS, vol. 36(1), pages 84-106, March.
    21. Lu Fang & Zhe Yuan & Kaifu Zhang & Dante Donati & Miklos Sarvary, 2025. "Generative AI and Sales Productivity: Field Experiments in Online Retail," Papers 2510.12049, arXiv.org, revised Jun 2026.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2606.08265. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.