IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v10y2022i4p569-d747687.html
   My bibliography  Save this article

MisRoBÆRTa: Transformers versus Misinformation

Author

Listed:
  • Ciprian-Octavian Truică

    (InfoLab, Department of Information Technology, Uppsala University, SE-751 05 Uppsala, Sweden
    Computer Science and Engineering Department, Faculty of Automatic Control and Computers, University Politehnica of Bucharest, RO-060042 Bucharest, Romania
    These authors contributed equally to this work.)

  • Elena-Simona Apostol

    (Computer Science and Engineering Department, Faculty of Automatic Control and Computers, University Politehnica of Bucharest, RO-060042 Bucharest, Romania
    These authors contributed equally to this work.)

Abstract

Misinformation is considered a threat to our democratic values and principles. The spread of such content on social media polarizes society and undermines public discourse by distorting public perceptions and generating social unrest while lacking the rigor of traditional journalism. Transformers and transfer learning proved to be state-of-the-art methods for multiple well-known natural language processing tasks. In this paper, we propose MisRoBÆRTa, a novel transformer-based deep neural ensemble architecture for misinformation detection. MisRoBÆRTa takes advantage of two state-of-the art transformers, i.e., BART and RoBERTa, to improve the performance of discriminating between real news and different types of fake news. We also benchmarked and evaluated the performances of multiple transformers on the task of misinformation detection. For training and testing, we used a large real-world news articles dataset (i.e., 100,000 records) labeled with 10 classes, thus addressing two shortcomings in the current research: ( 1 ) increasing the size of the dataset from small to large, and ( 2 ) moving the focus of fake news detection from binary classification to multi-class classification. For this dataset, we manually verified the content of the news articles to ensure that they were correctly labeled. The experimental results show that the accuracy of transformers on the misinformation detection problem was significantly influenced by the method employed to learn the context, dataset size, and vocabulary dimension. We observe empirically that the best accuracy performance among the classification models that use only one transformer is obtained by BART, while DistilRoBERTa obtains the best accuracy in the least amount of time required for fine-tuning and training. However, the proposed MisRoBÆRTa outperforms the other transformer models in the task of misinformation detection. To arrive at this conclusion, we performed ample ablation and sensitivity testing with MisRoBÆRTa on two datasets.

Suggested Citation

  • Ciprian-Octavian Truică & Elena-Simona Apostol, 2022. "MisRoBÆRTa: Transformers versus Misinformation," Mathematics, MDPI, vol. 10(4), pages 1-25, February.
    • Handle: RePEc:gam:jmathe:v:10:y:2022:i:4:p:569-:d:747687
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/10/4/569/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/10/4/569/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Alexandre Bovet & Hernán A. Makse, 2019. "Influence of fake news in Twitter during the 2016 US presidential election," Nature Communications, Nature, vol. 10(1), pages 1-14, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Carlos Carrasco-Farré, 2022. "The fingerprints of misinformation: how deceptive content differs from reliable sources in terms of cognitive effort and appeal to emotions," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-18, December.
    2. Uğur Baloğlu, 2021. "Trolls, Pressure and Agenda: The discursive fight on Twitter in Turkey," Media and Communication, Cogitatio Press, vol. 9(4), pages 39-51.
    3. Mujtaba Ali Isani, 2021. "Methodological Problems of Using Arabic-Language Twitter as a Gauge for Arab Attitudes Toward Politics and Society," Contemporary Review of the Middle East, , vol. 8(1), pages 22-35, March.
    4. Nwaibeh, E.A. & Chikwendu, C.R., 2023. "A deterministic model of the spread of scam rumor and its numerical simulations," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 207(C), pages 111-129.
    5. Sven Gruener, 2024. "Determinants of Gullibility to Misinformation: A Study of Climate Change, COVID-19 and Artificial Intelligence," Journal of Interdisciplinary Economics, , vol. 36(1), pages 58-78, January.
    6. John Bryden & Eric Silverman, 2019. "Underlying socio-political processes behind the 2016 US election," PLOS ONE, Public Library of Science, vol. 14(4), pages 1-11, April.
    7. Peter D. Lunn & Cameron A. Belton & Ciarán Lavin & Féidhlim P. McGowan & Shane Timmons & Deirdre A. Robertson, 2020. "Using behavioral science to help fight the Coronavirus," Journal of Behavioral Public Administration, Center for Experimental and Behavioral Public Administration, vol. 3(1).
    8. James Flamino & Alessandro Galeazzi & Stuart Feldman & Michael W. Macy & Brendan Cross & Zhenkun Zhou & Matteo Serafino & Alexandre Bovet & Hernán A. Makse & Boleslaw K. Szymanski, 2023. "Political polarization of news media and influencers on Twitter in the 2016 and 2020 US presidential elections," Nature Human Behaviour, Nature, vol. 7(6), pages 904-916, June.
    9. Matthew Spradling & Jeremy Straub, 2022. "Evaluation of the Factors That Impact the Perception of Online Content Trustworthiness by Income, Political Affiliation and Online Usage Time," Future Internet, MDPI, vol. 14(11), pages 1-55, November.
    10. Lodh, Rishab & Dey, Oindrila, 2023. "“Fake news alert!”: A game of misinformation and news consumption behavior," MPRA Paper 118371, University Library of Munich, Germany.
    11. Xu, Shuqi & Mariani, Manuel Sebastian & Lü, Linyuan & Medo, Matúš, 2020. "Unbiased evaluation of ranking metrics reveals consistent performance in science and technology citation data," Journal of Informetrics, Elsevier, vol. 14(1).
    12. Lipić, Tomislav & Štajduhar, Andrija & Medvidović, Luka & Wild, Dorian & Korošak, Dean & Podobnik, Boris, 2022. "Stringency without efficiency is not adequate to combat pandemics," Chaos, Solitons & Fractals, Elsevier, vol. 160(C).
    13. Ho-Chun Herbert Chang & Emilio Ferrara, 2022. "Comparative analysis of social bots and humans during the COVID-19 pandemic," Journal of Computational Social Science, Springer, vol. 5(2), pages 1409-1425, November.
    14. Marius Dragomir & José Rúas-Araújo & Minna Horowitz, 2024. "Beyond online disinformation: assessing national information resilience in four European countries," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-10, December.
    15. Junhui Cai & Dan Yang & Wu Zhu & Haipeng Shen & Linda Zhao, 2021. "Network regression and supervised centrality estimation," Papers 2111.12921, arXiv.org.
    16. Yevgeniy Golovchenko, 2020. "Measuring the scope of pro-Kremlin disinformation on Twitter," Palgrave Communications, Palgrave Macmillan, vol. 7(1), pages 1-11, December.
    17. Kai-Cheng Yang & Emilio Ferrara & Filippo Menczer, 2022. "Botometer 101: social bot practicum for computational social scientists," Journal of Computational Social Science, Springer, vol. 5(2), pages 1511-1528, November.
    18. Mao, Yajun & Rong, Zhihai & Wu, Zhi-Xi, 2021. "Effect of collective influence on the evolution of cooperation in evolutionary prisoner’s dilemma games," Applied Mathematics and Computation, Elsevier, vol. 392(C).
    19. So-Min Cheong & Matthew Babcock, 2021. "Attention to misleading and contentious tweets in the case of Hurricane Harvey," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 105(3), pages 2883-2906, February.
    20. Hyehyun Hong & Hyun Jee Oh, 2020. "Utilizing Bots for Sustainable News Business: Understanding Users’ Perspectives of News Bots in the Age of Social Media," Sustainability, MDPI, vol. 12(16), pages 1-16, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:10:y:2022:i:4:p:569-:d:747687. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.