Synergistic Joint Model of Knowledge Graph and LLM for Enhancing XAI-Based Clinical Decision Support Systems

Despite the excellent generalization capabilities of large-scale language models (LLMs), their severe limitations, such as illusions, lack of domain-specific knowledge, and ambiguity in the reasoning process, challenge their direct application to clinical decision support systems (CDSSs). To address these challenges, this study proposes a synergistic joint model that integrates knowledge graphs (KGs) and LLMs to enhance domain-specific knowledge and improve explainability in CDSSs. The proposed model leverages KGs to provide structured, domain-specific insights while utilizing LLMs’ generative capabilities to dynamically extract, refine, and expand medical knowledge. This bi-directional interaction ensures that CDSS recommendations remain both clinically accurate and contextually comprehensive. Performance evaluation of the joint model for mental health etiology, stress detection, and emotion recognition tasks of the CDSS showed up to a 12.0% increase in accuracy and an 8.6% increase in F1 score when compared to the standalone LLM model, with additional significant improvements when using the model with medical domain knowledge. Thus, the reliable and up-to-date domain knowledge obtained through the joint model not only improves the task performance of the CDSS, but also provides direct evidence of how such decisions were made. These findings validate the broad applicability and effectiveness of our KG–LLM joint model, highlighting its potential in real-world clinical decision support scenarios.