IDEAS home Printed from https://ideas.repec.org/a/spr/jcsosc/v6y2023i2d10.1007_s42001-023-00213-y.html
   My bibliography  Save this article

Detecting science-based health disinformation: a stylometric machine learning approach

Author

Listed:
  • Jason A. Williams

    (Augusta University)

  • Ahmed Aleroud

    (Augusta University)

  • Danielle Zimmerman

    (Augusta University)

Abstract

The COVID-19 pandemic showed that misleading scientific health information has become widespread and is challenging to counteract. Some of this disinformation comes from modification of medical research results. This paper investigates how humans create health disinformation through controlled changes of text from abstracts of peer-reviewed COVID-19 research papers. We also developed a machine learning model that used statement embeddings, readability, and text quality features to create datasets that contain falsified scientific statements. We then created machine learning classification models to identify statements containing disinformation. Our results reveal the importance of readability metrics and information quality features in identifying which statements were falsified. We show that text embeddings and semantic similarity do not yield a high detection rate of true/falsified statements compared to using information quality and readability features.

Suggested Citation

  • Jason A. Williams & Ahmed Aleroud & Danielle Zimmerman, 2023. "Detecting science-based health disinformation: a stylometric machine learning approach," Journal of Computational Social Science, Springer, vol. 6(2), pages 817-843, October.
    • Handle: RePEc:spr:jcsosc:v:6:y:2023:i:2:d:10.1007_s42001-023-00213-y
    DOI: 10.1007/s42001-023-00213-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s42001-023-00213-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s42001-023-00213-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Danielle Caled & Mário J. Silva, 2022. "Digital media and misinformation: An outlook on multidisciplinary strategies against manipulation," Journal of Computational Social Science, Springer, vol. 5(1), pages 123-159, May.
    2. Lina Zhou & Judee K. Burgoon & Jay F. Nunamaker & Doug Twitchell, 2004. "Automating Linguistics-Based Cues for Detecting Deception in Text-Based Asynchronous Computer-Mediated Communications," Group Decision and Negotiation, Springer, vol. 13(1), pages 81-106, January.
    3. Lorenz Graf-Vlachy, 2022. "Is the readability of abstracts decreasing in management research?," Review of Managerial Science, Springer, vol. 16(4), pages 1063-1084, May.
    4. Brian G. Southwell & J. Scott Babwah Brennen & Ryan Paquin & Vanessa Boudewyns & Jing Zeng, 2022. "Defining and Measuring Scientific Misinformation," The ANNALS of the American Academy of Political and Social Science, , vol. 700(1), pages 98-111, March.
    5. Lisa Singh & Leticia Bode & Ceren Budak & Kornraphop Kawintiranon & Colton Padden & Emily Vraga, 2020. "Understanding high- and low-quality URL Sharing on COVID-19 Twitter streams," Journal of Computational Social Science, Springer, vol. 3(2), pages 343-366, November.
    6. Wentao Xu & Kazutoshi Sasahara, 2022. "Correction to: Characterizing the roles of bots on Twitter during the COVID‑19 infodemic," Journal of Computational Social Science, Springer, vol. 5(1), pages 1095-1095, May.
    7. Wentao Xu & Kazutoshi Sasahara, 2022. "Characterizing the roles of bots on Twitter during the COVID-19 infodemic," Journal of Computational Social Science, Springer, vol. 5(1), pages 591-609, May.
    8. Umair Majid & Aghna Wasim & Judy Truong & Simran Bakshi, 2021. "Public trust in governments, health care providers, and the media during pandemics: A systematic review," Journal of Trust Research, Taylor & Francis Journals, vol. 11(2), pages 119-141, July.
    9. Justin Farrell & Kathryn McConnell & Robert Brulle, 2019. "Evidence-based strategies to combat scientific misinformation," Nature Climate Change, Nature, vol. 9(3), pages 191-195, March.
    10. Alexander J. Stewart & Mohsen Mosleh & Marina Diakonova & Antonio A. Arechar & David G. Rand & Joshua B. Plotkin, 2019. "Information gerrymandering and undemocratic decisions," Nature, Nature, vol. 573(7772), pages 117-121, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ali Unlu & Sophie Truong & Nitin Sawhney & Jonas Sivelä & Tuukka Tammi, 2024. "Long-term assessment of social amplification of risk during COVID-19: challenges to public health agencies amid misinformation and vaccine stance," Journal of Computational Social Science, Springer, vol. 7(1), pages 809-836, April.
    2. Josep Perell'o & Miquel Montero & Jaume Masoliver & J. Doyne Farmer & John Geanakoplos, 2019. "Statistical analysis and stochastic interest rate modelling for valuing the future with implications in climate change mitigation," Papers 1910.01928, arXiv.org, revised Feb 2020.
    3. David Klenert & Franziska Funke & Linus Mattauch & Brian O’Callaghan, 2020. "Five Lessons from COVID-19 for Advancing Climate Change Mitigation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 76(4), pages 751-778, August.
    4. Carlos Carrasco-Farré, 2022. "The fingerprints of misinformation: how deceptive content differs from reliable sources in terms of cognitive effort and appeal to emotions," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-18, December.
    5. repec:cup:judgdm:v:16:y:2021:i:6:p:1413-1438 is not listed on IDEAS
    6. Zeng, Jing & Brennen, Scott Babwah, 2023. "Misinformation," Internet Policy Review: Journal on Internet Regulation, Alexander von Humboldt Institute for Internet and Society (HIIG), Berlin, vol. 12(4), pages 1-20.
    7. Jacqueline N. Zadelaar & Joost A. Agelink van Rentergem & Jessica V. Schaaf & Tycho J. Dekkers & Nathalie de Vent & Laura M. S. Dekkers & Maria C. Olthof & Brenda R. J. Jansen & Hilde M. Huizenga, 2021. "Development of decision making based on internal and external information: A hierarchical Bayesian approach," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 16(6), pages 1413-1438, November.
    8. Tsvetkova, Milena & Olsson, Henrik & Galesic, Mirta, 2024. "Social networks affect redistribution decisions and polarization," OSF Preprints bw7ux_v1, Center for Open Science.
    9. Lackner, Teresa & Fierro, Luca E. & Mellacher, Patrick, 2025. "Opinion dynamics meet agent-based climate economics: An integrated analysis of carbon taxation," Journal of Economic Behavior & Organization, Elsevier, vol. 229(C).
    10. John McClure & Ilan Noy & Yoshi Kashima & Taciano L. Milfont, 2022. "Attributions for extreme weather events: science and the people," Climatic Change, Springer, vol. 174(3), pages 1-17, October.
    11. Loukia Taxitari & Thanos Sitistas & Eleni Gavriil, 2025. "“Disinformation Aims to Mislead; Misinformation Thrives in Ignorance”: Insights from Experts and Non-Experts in Greek-Speaking Cyprus," Social Sciences, MDPI, vol. 14(3), pages 1-23, February.
    12. Chen, Yuh-Jen & Wu, Chun-Han & Chen, Yuh-Min & Li, Hsin-Ying & Chen, Huei-Kuen, 2017. "Enhancement of fraud detection for narratives in annual reports," International Journal of Accounting Information Systems, Elsevier, vol. 26(C), pages 32-45.
    13. Lina Zhou & Jie Tao & Dongsong Zhang, 2023. "Does Fake News in Different Languages Tell the Same Story? An Analysis of Multi-level Thematic and Emotional Characteristics of News about COVID-19," Information Systems Frontiers, Springer, vol. 25(2), pages 493-512, April.
    14. Mohsen Mosleh & Alexander J. Stewart & Joshua B. Plotkin & David G. Rand, 2020. "Prosociality in the economic Dictator Game is associated with less parochialism and greater willingness to vote for intergroup compromise," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 15(1), pages 1-6, January.
    15. Philipp Lorenz-Spreen & Stephan Lewandowsky & Cass R. Sunstein & Ralph Hertwig, 2020. "How behavioural sciences can promote truth, autonomy and democratic discourse online," Nature Human Behaviour, Nature, vol. 4(11), pages 1102-1109, November.
    16. Wake Smith & Claire Henly, 2021. "Updated and outdated reservations about research into stratospheric aerosol injection," Climatic Change, Springer, vol. 164(3), pages 1-15, February.
    17. Sugandha Srivastav & Ryan Rafaty, 2023. "Political Strategies to Overcome Climate Policy Obstructionism," Papers 2304.14960, arXiv.org.
    18. Emilio Ferrara & Stefano Cresci & Luca Luceri, 2020. "Misinformation, manipulation, and abuse on social media in the era of COVID-19," Journal of Computational Social Science, Springer, vol. 3(2), pages 271-277, November.
    19. Xi Zhao & Li Li & Wei Xiao, 2023. "The diachronic change of research article abstract difficulty across disciplines: a cognitive information-theoretic approach," Palgrave Communications, Palgrave Macmillan, vol. 10(1), pages 1-12, December.
    20. Siri Frisli, 2025. "Semi-supervised self-training for COVID-19 misinformation detection: analyzing Twitter data and alternative news media on Norwegian Twitter," Journal of Computational Social Science, Springer, vol. 8(2), pages 1-34, May.
    21. Pringle Anna & Robbins David, 2022. "From denial to delay: Climate change discourses in Ireland," Administration, Sciendo, vol. 70(3), pages 59-84, August.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jcsosc:v:6:y:2023:i:2:d:10.1007_s42001-023-00213-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.