IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v19y2025i3s1751157725000434.html

Exploring the change in scientific readability following the release of ChatGPT

Author

Listed:
  • Alsudais, Abdulkareem

Abstract

The rise and growing popularity of accessible large language models have raised questions about their impact on various aspects of life, including how scientists write and publish their research. The primary objective of this paper is to analyze a dataset consisting of all abstracts posted on arXiv.org between 2010 and June 7th, 2024, to assess the evolution of their readability and determine whether significant shifts occurred following the release of ChatGPT in November 2022. Four standard readability formulas are used to calculate individual readability scores for each paper, classifying their level of readability. These scores are then aggregated by year and across the eight primary categories covered by the platform. The results show a steady annual decrease in readability, suggesting that abstracts are likely becoming increasingly complex. Additionally, following the release of ChatGPT, a significant change in readability is observed for 2023 and the analyzed months of 2024. Similar trends are found across categories, with most experiencing a notable change in readability during 2023 and 2024. These findings offer insights into the broader changes in readability and point to the likely influence of AI on scientific writing.

Suggested Citation

  • Alsudais, Abdulkareem, 2025. "Exploring the change in scientific readability following the release of ChatGPT," Journal of Informetrics, Elsevier, vol. 19(3).
    • Handle: RePEc:eee:infome:v:19:y:2025:i:3:s1751157725000434
    DOI: 10.1016/j.joi.2025.101679
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157725000434
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2025.101679?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Dowling, Michael & Hammami, Helmi & Zreik, Ousayna, 2018. "Easy to read, easy to cite?," Economics Letters, Elsevier, vol. 173(C), pages 100-103.
    2. Brady D. Lund & Ting Wang & Nishith Reddy Mannuru & Bing Nie & Somipam Shimray & Ziang Wang, 2023. "ChatGPT and a new academic reality: Artificial Intelligence‐written research papers and the ethics of the large language models in scholarly publishing," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(5), pages 570-581, May.
    3. Sun, Zhuanlan & He, Dongjin & Li, Yiwei, 2024. "How the readability of manuscript before journal submission advantages peer review process: Evidence from biomedical scientific publications," Journal of Informetrics, Elsevier, vol. 18(3).
    4. Horbach, Serge P.J.M. & Schneider, Jesper W. & Sainte-Marie, Maxime, 2022. "Ungendered writing: Writing styles are unlikely to account for gender differences in funding rates in the natural and technical sciences," Journal of Informetrics, Elsevier, vol. 16(4).
    5. McCannon, Bryan C., 2019. "Readability and research impact," Economics Letters, Elsevier, vol. 180(C), pages 76-79.
    6. Porwal, Priya & Devare, Manoj H., 2024. "Scientific impact analysis: Unraveling the link between linguistic properties and citations," Journal of Informetrics, Elsevier, vol. 18(3).
    7. Shan Wang & Xiaojun Liu & Jie Zhou, 2022. "Readability is decreasing in language and linguistics," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4697-4729, August.
    8. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Di Costa, Flavia, 2020. "The role of geographical proximity in knowledge diffusion, measured by citations to scientific literature," Journal of Informetrics, Elsevier, vol. 14(1).
    9. Dolnicar, Sara & Chapple, Alexander, 2015. "The readability of articles in tourism journals," Annals of Tourism Research, Elsevier, vol. 52(C), pages 161-166.
    10. Jialiang Lin & Yao Yu & Yu Zhou & Zhiyang Zhou & Xiaodong Shi, 2020. "How many preprints have actually been printed and why: a case study of computer science preprints on arXiv," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 555-574, July.
    11. Michael Dowling & Helmi Hammami & Ousayna Zreik, 2018. "Easy to read, easy to cite?," Post-Print hal-01958017, HAL.
    12. Ante, Lennart, 2022. "The relationship between readability and scientific impact: Evidence from emerging technology discourses," Journal of Informetrics, Elsevier, vol. 16(1).
    13. Alsudais, Abdulkareem, 2021. "In-code citation practices in open research software libraries," Journal of Informetrics, Elsevier, vol. 15(2).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xi Zhao & Li Li & Wei Xiao, 2023. "The diachronic change of research article abstract difficulty across disciplines: a cognitive information-theoretic approach," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 10(1), pages 1-12, December.
    2. Ante, Lennart, 2022. "The relationship between readability and scientific impact: Evidence from emerging technology discourses," Journal of Informetrics, Elsevier, vol. 16(1).
    3. Sun, Zhuanlan & He, Dongjin & Li, Yiwei, 2024. "How the readability of manuscript before journal submission advantages peer review process: Evidence from biomedical scientific publications," Journal of Informetrics, Elsevier, vol. 18(3).
    4. Ju Wen & Lei Lei, 2022. "Adjectives and adverbs in life sciences across 50 years: implications for emotions and readability in academic texts," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4731-4749, August.
    5. Shan Wang & Xiaojun Liu & Jie Zhou, 2022. "Readability is decreasing in language and linguistics," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4697-4729, August.
    6. Joost Winter, 2024. "Can ChatGPT be used to predict citation counts, readership, and social media interaction? An exploration among 2222 scientific abstracts," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(4), pages 2469-2487, April.
    7. Caroline Fry & Megan MacGarvie, 2024. "Author Country of Origin and Attention on Open Science Platforms: Evidence from COVID-19 Preprints," Management Science, INFORMS, vol. 70(8), pages 5426-5444, August.
    8. Diego Marino Fages, 2020. "Write better, publish better," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(3), pages 1671-1681, March.
    9. Burke, Matt & Fry, John, 2019. "How easy is it to understand consumer finance?," Economics Letters, Elsevier, vol. 177(C), pages 1-4.
    10. Rose, Michael E. & Opolot, Daniel C. & Georg, Co-Pierre, 2022. "Discussants," Research Policy, Elsevier, vol. 51(10).
    11. Kayvan Kousha & Mike Thelwall, 2024. "Factors associating with or predicting more cited or higher quality journal articles: An Annual Review of Information Science and Technology (ARIST) paper," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 75(3), pages 215-244, March.
    12. Qian Wang & Guangwei Hu, 2024. "Expressions of confusion in research articles: a diachronic cross-disciplinary investigation," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 445-471, January.
    13. Chili Li & Xue Wang & Long Qian, 2025. "Exploring Syntactic Complexity and Text Readability in an ELT Textbook Series for Chinese English Majors," SAGE Open, , vol. 15(1), pages 21582440251, February.
    14. McCannon, Bryan C., 2019. "Readability and research impact," Economics Letters, Elsevier, vol. 180(C), pages 76-79.
    15. Peng, Wen & Yue, Mingliang & Sun, Mingyue & Ma, Tingcan, 2024. "Revision and academic impact: A case study of bioRxiv preprint papers," Journal of Informetrics, Elsevier, vol. 18(1).
    16. Yueyue Huang & Keru Li & Dechao Li, 2025. "Writing for non-specialists? Investigating readability and jargon use in successful lay summaries of CRF grant proposals 2006–2024," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(11), pages 6305-6321, November.
    17. Hong, Wilson Cheong Hin & Yu, Joanne & Egger, Roman, 2025. "Decoding linguistic complexity in tourism studies," Annals of Tourism Research, Elsevier, vol. 111(C).
    18. Song, Ningyuan & Chen, Kejun & Zhao, Yuehua, 2023. "Understanding writing styles of scientific papers in the IS-LS domain: Evidence from abstracts over the past three decades," Journal of Informetrics, Elsevier, vol. 17(1).
    19. Leonardo Banh & Gero Strobel, 2023. "Generative artificial intelligence," Electronic Markets, Springer;IIM University of St. Gallen, vol. 33(1), pages 1-17, December.
    20. Yoon, Jisung & Park, Jinseo & Yun, Jinhyuk & Jung, Woo-Sung, 2023. "Quantifying knowledge synchronization with the network-driven approach," Journal of Informetrics, Elsevier, vol. 17(4).

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:19:y:2025:i:3:s1751157725000434. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.