IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2604.08678.html

Scaffolding Human-AI Collaboration: A Field Experiment on Behavioral Protocols and Cognitive Reframing

Author

Listed:
  • Alex Farach
  • Alexia Cambon
  • Lev Tankelevitch
  • Connie Hsueh
  • Rebecca Janssen

Abstract

Organizations have widely deployed generative AI tools, yet productivity gains remain uneven, suggesting that how people use AI matters as much as whether they have access. We conducted a field experiment with 388 employees at a Fortune 500 retailer to test two scaffolding interventions for human-AI collaboration. All participants had access to the same AI tool; we varied only the structure surrounding its use. A behavioral scaffolding intervention (a structured protocol requiring joint AI use within pairs) was associated with lower document quality relative to unstructured use and substantially lower document production. A cognitive scaffolding intervention (partnership training that reframed AI as a thought partner) was associated with higher individual document quality at the top of the distribution. Treatment participants also showed greater positive belief change across the session, though sensitivity analyses suggest this likely reflects recovery from carry-over effects rather than genuine training-induced shifts. Both findings are subject to design limitations including an AM/PM session confound, differential attrition, and LLM grading sensitivity to document length.

Suggested Citation

  • Alex Farach & Alexia Cambon & Lev Tankelevitch & Connie Hsueh & Rebecca Janssen, 2026. "Scaffolding Human-AI Collaboration: A Field Experiment on Behavioral Protocols and Cognitive Reframing," Papers 2604.08678, arXiv.org, revised Apr 2026.
  • Handle: RePEc:arx:papers:2604.08678
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2604.08678
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769, December.
    2. Ajay Agrawal & Joshua S. Gans & Avi Goldfarb, 2024. "Artificial intelligence adoption and system‐wide change," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 33(2), pages 327-337, March.
    3. Romain Cadario & Chiara Longoni & Carey K. Morewedge, 2021. "Understanding, explaining, and utilizing medical artificial intelligence," Nature Human Behaviour, Nature, vol. 5(12), pages 1636-1642, December.
    4. James E. Pustejovsky & Elizabeth Tipton, 2018. "Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 36(4), pages 672-683, October.
    5. David S. Lee, 2009. "Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(3), pages 1071-1102.
    6. Erik Brynjolfsson & Danielle Li & Lindsey Raymond, 2025. "Generative AI at Work," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 140(2), pages 889-942.
    7. Gerardine DeSanctis & Marshall Scott Poole, 1994. "Capturing the Complexity in Advanced Technology Use: Adaptive Structuration Theory," Organization Science, INFORMS, vol. 5(2), pages 121-147, May.
    8. Wanda J. Orlikowski, 1992. "The Duality of Technology: Rethinking the Concept of Technology in Organizations," Organization Science, INFORMS, vol. 3(3), pages 398-427, August.
    9. Emily Oster, 2019. "Unobservable Selection and Coefficient Stability: Theory and Evidence," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 37(2), pages 187-204, April.
    10. Michelle Vaccaro & Abdullah Almaatouq & Thomas Malone, 2024. "When combinations of humans and AI are useful: A systematic review and meta-analysis," Nature Human Behaviour, Nature, vol. 8(12), pages 2293-2303, December.
    11. Sarah Lebovitz & Hila Lifshitz-Assaf & Natalia Levina, 2022. "To Engage or Not to Engage with AI for Critical Judgments: How Professionals Deal with Opacity When Using AI for Medical Diagnosis," Organization Science, INFORMS, vol. 33(1), pages 126-148, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Feiyang Xu & Poonacha K. Medappa & Murat M. Tunc & Martijn Vroegindeweij & Jan C. Fransoo, 2025. "AI-Assisted Programming Decreases the Productivity of Experienced Developers by Increasing the Technical Debt and Maintenance Burden," Papers 2510.10165, arXiv.org, revised Jan 2026.
    2. Upasak Das & Rupayan Pal & Udayan Rathore & Bibhas Saha, 2023. "Rein in pandemic by pricing vaccine: Does social trust matter?," Indira Gandhi Institute of Development Research, Mumbai Working Papers 2023-008, Indira Gandhi Institute of Development Research, Mumbai, India.
    3. Arne Henningsen & Guy Low & David Wuepper & Tobias Dalhaus & Hugo Storm & Dagim Belay & Stefan Hirsch, 2024. "Estimating Causal Effects with Observational Data: Guidelines for Agricultural and Applied Economists," IFRO Working Paper 2024/03, University of Copenhagen, Department of Food and Resource Economics.
    4. Mariek Vanden Abeele & Ralf De Wolf & Rich Ling, 2018. "Mobile Media and Social Space: How Anytime, Anyplace Connectivity Structures Everyday Life," Media and Communication, Cogitatio Press, vol. 6(2), pages 5-14.
    5. Sansone, Dario, 2019. "Pink work: Same-sex marriage, employment and discrimination," Journal of Public Economics, Elsevier, vol. 180(C).
    6. Emmanuelle Vaast & Geoff Walsham, 2009. "Trans-Situated Learning: Supporting a Network of Practice with an Information Infrastructure," Information Systems Research, INFORMS, vol. 20(4), pages 547-564, December.
    7. Christoph Riedl & Eric Bogert, 2024. "Who Benefits from AI? Self-Selection, Skill Gap, and the Hidden Costs of AI Feedback," Papers 2409.18660, arXiv.org, revised Apr 2026.
    8. Célia Lemaire & Thierry Nobre, 2013. "La Pre-Appropriation D'Un Outil De Gestion A L'Hopital," Post-Print hal-00992968, HAL.
    9. León, Gianmarco, 2017. "Turnout, political preferences and information: Experimental evidence from Peru," Journal of Development Economics, Elsevier, vol. 127(C), pages 56-71.
    10. Pamela J. Hinds & Diane E. Bailey, 2003. "Out of Sight, Out of Sync: Understanding Conflict in Distributed Teams," Organization Science, INFORMS, vol. 14(6), pages 615-632, December.
    11. Wanda J. Orlikowski & C. Suzanne Iacono, 2001. "Research Commentary: Desperately Seeking the “IT” in IT Research—A Call to Theorizing the IT Artifact," Information Systems Research, INFORMS, vol. 12(2), pages 121-134, June.
    12. Siverskog, Jonathan & Henriksson, Martin, 2022. "The health cost of reducing hospital bed capacity," Social Science & Medicine, Elsevier, vol. 313(C).
    13. Alloush, Mo & Bloem, Jeffrey R., 2022. "Neighborhood violence, poverty, and psychological well-being," Journal of Development Economics, Elsevier, vol. 154(C).
    14. Tang, Lianzhou & Xu, Wenli, 2025. "Patronage and pollution," Journal of Environmental Economics and Management, Elsevier, vol. 130(C).
    15. Tang, Can & Zhao, Zhong, 2022. "Informal institution meets child development," MERIT Working Papers 2022-032, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    16. Gerald C. Kane & Maryam Alavi, 2008. "Casting the Net: A Multimodal Network Perspective on User-System Interactions," Information Systems Research, INFORMS, vol. 19(3), pages 253-272, September.
    17. Saraswat, Deepak, 2024. "Gender composition of children and sanitation behavior in India," Journal of Environmental Economics and Management, Elsevier, vol. 125(C).
    18. Mai, Nhat Chi, 2022. "Socioeconomic effects of collectivist and individualist education: A comparison between North and South Vietnam," OSF Preprints n9pyw, Center for Open Science.
    19. Eichengreen, Barry & Aksoy, Cevat Giray & Saka, Orkun, 2021. "Revenge of the experts: Will COVID-19 renew or diminish public trust in science?," Journal of Public Economics, Elsevier, vol. 193(C).
    20. Yang, Xiaoyin & Zhang, Xinmin & Zang, Hong & Chu, Hongcong & Feng, Chao, 2025. "From shadows to spotlight: How big data bureaus unveil corporate R&D gaming?," Finance Research Letters, Elsevier, vol. 86(PG).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2604.08678. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.