Large Language Models for Fake News Detection in Business Intelligence Pipelines: A Comparative Study

The COVID-19 pandemic highlighted the profound impact of misinformation on business operations, particularly in supply chain decision-making and crisis response. As organizations increasingly integrate real-time news and social media content into Business Intelligence (BI) systems, the need for automated tools that can filter out unreliable information has become urgent. This paper investigates the integration of Large Language Models (LLMs) into BI pipelines to detect fake news before it influences business decisions. We compare two fine-tuning techniques, (1) a Scoring approach, which instructs LLMs to assess news articles based on linguistic, sentiment, and contextual features, and (2) Retrieval-Augmented Generation (RAG), which equips LLMs with external knowledge retrieval to ground their analysis. Using the COVID19-FNIR dataset of true and fake pandemic-related news, we evaluate both OpenAI’s GPT and Microsoft’s Phi-3 models. Both the Scoring-based fine-tuning approach and the Retrieval-Augmented Generation (RAG) pipeline were implemented using a standardised training configuration. Fine-tuning employed LoRA adapters with a learning rate of 2e-5, batch size of 8, and a three-epoch training schedule, selected based on validation loss convergence. Evaluation metrics included accuracy, F1-score, and calibration error to capture both classification performance and reliability of predictions. Results show that the RAG method significantly improves fake news detection accuracy (over 92%), while the Scoring technique provides better interpretability for enterprise users. We discuss the trade-offs between accuracy and explainability and demonstrate how each method can enhance BI systems by improving data quality and trust in analytic outputs. Furthermore, we explore how these techniques generalize beyond COVID-19 to broader domains such as finance, supply chains, and crisis communication. Building on these insights, the paper introduces a practical, organization-ready framework for integrating LLM-based misinformation detection into existing business intelligence pipelines. The framework outlines steps for data ingestion, operationalization of organizational scoring dimensions, model selection (Scoring vs. RAG), deployment, and continuous monitoring. This provides a clear and actionable blueprint for businesses seeking to strengthen information quality assurance using modern LLM technologies.