An Empirical Study of Knowledge Graph–Augmented Retrieval for Hallucination Mitigation in University Academic Assistance Systems

Large Language Models (LLMs) used in university chatbots often create false or misleading answers—a problem known as "hallucination". When students ask about course prerequisites, credit transfers, examination rules, or graduation policies in open and distance learning systems, standard AI systems frequently confuse similar policies or guess missing details. Standard document-search methods (Vector Retrieval-Augmented Generation or Vector-RAG) help by pulling relevant text passages, but they still struggle when answers depend on interconnected rules spread across multiple handbook pages. We built and tested a Knowledge Graph–Augmented Retrieval (KG-RAG) system. This approach connects traditional keyword/vector search with a structured "knowledge map" (a Knowledge Graph) that explicitly links courses, degree tracks, examination regulations, and prerequisite dependencies. We constructed a real-world multi-institutional benchmark dataset containing 2,850 policy passages compiled from 20 leading open, distance, and online universities globally (including The Open University UK, IGNOU, BAOU, University of London, and ASU Online) along with a structured graph of 3,400 entities and 8,200 relations. We evaluated 220 verified student queries across three systems: a base LLM, a standard Vector-RAG system, and our hybrid KG-RAG framework. The proposed KG-RAG system achieved 92.0% factual accuracy, outperforming standard Vector-RAG (84.0%) and the base LLM (61.0%). Hallucination rates dropped from 16.0% in standard Vector-RAG to just 6.0% in KG-RAG (p