IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2506.22704.html
   My bibliography  Save this paper

Beyond Code: The Multidimensional Impacts of Large Language Models in Software Development

Author

Listed:
  • Sardar Bonabi
  • Sarah Bana
  • Vijay Gurbaxani
  • Tingting Nian

Abstract

Large language models (LLMs) are poised to significantly impact software development, especially in the Open-Source Software (OSS) sector. To understand this impact, we first outline the mechanisms through which LLMs may influence OSS through code development, collaborative knowledge transfer, and skill development. We then empirically examine how LLMs affect OSS developers' work in these three key areas. Leveraging a natural experiment from a temporary ChatGPT ban in Italy, we employ a Difference-in-Differences framework with two-way fixed effects to analyze data from all OSS developers on GitHub in three similar countries, Italy, France, and Portugal, totaling 88,022 users. We find that access to ChatGPT increases developer productivity by 6.4%, knowledge sharing by 9.6%, and skill acquisition by 8.4%. These benefits vary significantly by user experience level: novice developers primarily experience productivity gains, whereas more experienced developers benefit more from improved knowledge sharing and accelerated skill acquisition. In addition, we find that LLM-assisted learning is highly context-dependent, with the greatest benefits observed in technically complex, fragmented, or rapidly evolving contexts. We show that the productivity effects of LLMs extend beyond direct code generation to include enhanced collaborative learning and knowledge exchange among developers, dynamics that are essential for gaining a holistic understanding of LLMs' impact in OSS. Our findings offer critical managerial implications: strategically deploying LLMs can accelerate novice developers' onboarding and productivity, empower intermediate developers to foster knowledge sharing and collaboration, and support rapid skill acquisition, together enhancing long-term organizational productivity and agility.

Suggested Citation

  • Sardar Bonabi & Sarah Bana & Vijay Gurbaxani & Tingting Nian, 2025. "Beyond Code: The Multidimensional Impacts of Large Language Models in Software Development," Papers 2506.22704, arXiv.org, revised Jun 2025.
    • Handle: RePEc:arx:papers:2506.22704
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2506.22704
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jiangjie Chen & Ziqing Zhuo & Jiacheng Lin, 2023. "Does ChatGPT Play a Double-Edged Sword Role in the Field of Higher Education? An In-Depth Exploration of the Factors Affecting Student Performance," Sustainability, MDPI, vol. 15(24), pages 1-18, December.
    2. Clément de Chaisemartin & Xavier D'Haultfœuille, 2020. "Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects," American Economic Review, American Economic Association, vol. 110(9), pages 2964-2996, September.
    3. Bitzer, Jürgen & Geishecker, Ingo, 2010. "Who contributes voluntarily to OSS? An investigation among German IT employees," Research Policy, Elsevier, vol. 39(1), pages 165-172, February.
    4. Egami, Naoki & Yamauchi, Soichiro, 2023. "Using Multiple Pretreatment Periods to Improve Difference-in-Differences and Staggered Adoption Designs," Political Analysis, Cambridge University Press, vol. 31(2), pages 195-212, April.
    5. Jeffrey M. Perkel, 2018. "Why Jupyter is data scientists’ computational notebook of choice," Nature, Nature, vol. 563(7729), pages 145-146, November.
    6. Josh Lerner & Jean Tirole, 2002. "Some Simple Economics of Open Source," Journal of Industrial Economics, Wiley Blackwell, vol. 50(2), pages 197-234, June.
    7. Anton Korinek, 2023. "Language Models and Cognitive Automation for Economic Research," NBER Working Papers 30957, National Bureau of Economic Research, Inc.
    8. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    9. Seth M. Freedman & Alex Hollingsworth & Kosali I. Simon & Coady Wing & Madeline Yozwiak, 2023. "Designing Difference in Difference Studies With Staggered Treatment Adoption: Key Concepts and Practical Guidelines," NBER Working Papers 31842, National Bureau of Economic Research, Inc.
    10. Knut Blind & Torben Schubert, 2024. "Estimating the GDP effect of Open Source Software and its complementarities with R&D and patents: evidence and policy implications," The Journal of Technology Transfer, Springer, vol. 49(2), pages 466-491, April.
    11. David H. Autor, 2003. "Outsourcing at Will: The Contribution of Unjust Dismissal Doctrine to the Growth of Employment Outsourcing," Journal of Labor Economics, University of Chicago Press, vol. 21(1), pages 1-42, January.
    12. Andy Extance, 2023. "ChatGPT has entered the classroom: how LLMs could transform education," Nature, Nature, vol. 623(7987), pages 474-477, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cl'ement de Chaisemartin & Xavier D'Haultf{oe}uille, 2020. "Difference-in-Differences Estimators of Intertemporal Treatment Effects," Papers 2007.04267, arXiv.org, revised Dec 2024.
    2. Kurt Schmidheiny & Sebastian Siegloch, 2023. "On event studies and distributed‐lags in two‐way fixed effects models: Identification, equivalence, and generalization," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(5), pages 695-713, August.
    3. Pierre Cahuc & Stéphane Carcillo & Bérengère Patault & Flavien Moreau, 2024. "Judge Bias in Labor Courts and Firm Performance," Journal of the European Economic Association, European Economic Association, vol. 22(3), pages 1319-1366.
    4. Giulia Bettin & Isabella Giorgetti & Stefano Staffolani, 2024. "The impact of Covid-19 lockdown on the gender gap in the Italian labour market," Review of Economics of the Household, Springer, vol. 22(1), pages 1-33, March.
    5. Joop Age Harm Adema & Cevat Giray Aksoy & Panu Poutvaara, 2021. "Mobile Internet Access and the Desire to Emigrate," ifo Working Paper Series 365, ifo Institute - Leibniz Institute for Economic Research at the University of Munich.
    6. Emre Ekinci & David Wehrheim, 2024. "Holdup, Knowledge Transferability, and Productivity: Theory and Evidence from Knowledge Workers," Journal of Industrial Economics, Wiley Blackwell, vol. 72(1), pages 193-252, March.
    7. Crescioli, Tommaso & Martelli, Angelo, 2022. "Beyond the Great Reversal: Superstars, Unions, and the Euro," Single Market Economics Papers WP2022/8, Directorate-General for Internal Market, Industry, Entrepreneurship and SMEs (European Commission), Chief Economist Team.
    8. Simon Hartmann & Rok Spruk, 2023. "The impact of unilateral BIT terminations on FDI: Quasi-experimental evidence from India," The Review of International Organizations, Springer, vol. 18(2), pages 259-296, April.
    9. Karca D. Aral & Erasmo Giambona & Luk N. Van Wassenhove, 2023. "Managerial flexibility, capacity investment, and inventory levels," Production and Operations Management, Production and Operations Management Society, vol. 32(12), pages 3890-3907, December.
    10. Krause, Thomas & Sfrappini, Eleonora & Tonzer, Lena & Zgherea, Cristina, 2025. "How do EU banks’ funding costs respond to the CRD IV? An assessment based on the banking union directives database," Journal of Financial Stability, Elsevier, vol. 78(C).
    11. Dettmann, Eva & Giebler, Alexander & Weyh, Antje, 2020. "Flexpaneldid: A Stata toolbox for causal analysis with varying treatment time and duration," IWH Discussion Papers 3/2020, Halle Institute for Economic Research (IWH).
    12. Pilar Beneito & Maria Garcia-Vega & Oscar Vicente-Chirivella & Guillaume Wilemme, 2024. "Robots and firms’ labour search: The role of temporary work agencies," Discussion Papers 2024-02, University of Nottingham, GEP.
    13. Bajo-Buenestado, Raúl & Borrella-Mas, Miguel Ángel, 2025. "Market competition and the adoption of clean technology: Evidence from the taxi industry," European Economic Review, Elsevier, vol. 174(C).
    14. Casagrande, Dieison & Hidalgo, Álvaro & Feistel, Paulo, 2024. "Exports, productivity and capital intensity: Evidence for Brazilian firms," Structural Change and Economic Dynamics, Elsevier, vol. 70(C), pages 279-301.
    15. Bas Scheer & Wiljan van den Berge & Maarten Goos & Alan Manning & Anna Salomons, 2022. "Alternative Work Arrangements and Worker Outcomes: Evidence from Payrolling," CPB Discussion Paper 435, CPB Netherlands Bureau for Economic Policy Analysis.
    16. Ichev, Riste & Valentinčič, Aljoša, 2025. "The effect of impact investing on performance of private firms," Research in International Business and Finance, Elsevier, vol. 73(PA).
    17. Mac Clay, Pablo & Börner, Jan & Sellare, Jorge, 2023. "Institutional and macroeconomic stability mediate the effect of auctions on renewable energy capacity," Energy Policy, Elsevier, vol. 180(C).
    18. Diegmann, André & Pohlan, Laura & Weber, Andrea, 2024. "Do Politicians Affect Firm Outcomes? Evidence from Connections to the German Federal Parliament," IZA Discussion Papers 17031, Institute of Labor Economics (IZA).
    19. Alex Hollingsworth & Krzysztof Karbownik & Melissa A. Thomasson & Anthony Wray, 2024. "The Gift of a Lifetime: The Hospital, Modern Medicine, and Mortality," American Economic Review, American Economic Association, vol. 114(7), pages 2201-2238, July.
    20. Ricardo Dahis & Christiane Szerman, 2023. "Decentralizing Development: Evidence from Government Splits," Monash Economics Working Papers 2023-18, Monash University, Department of Economics.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2506.22704. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.