Enterprise Conversational AI with Retrieval-Augmented Generation: A CRM Case Study

Background: The integration of Large Language Models (LLMs) into enterprise information systems represents a major challenge in Artificial Intelligence (AI) and digital transformation. Retrieval-Augmented Generation (RAG) has emerged as a promising approach to improve knowledge management and reduce information overload in enterprise environments.Objective: This study aimed to design, implement, and evaluate a conversational assistant that combines RAG with fine-tuned LLMs in a Customer Relationship Management (CRM) environment. The research further sought to identify an optimal balance between performance, operational efficiency, and data governance.Methods: Guided by Task-Technology Fit theory, four open-source models (LLaMA 3.2 3B, Mistral 7B, Qwen2.5 3B, and Gemma 3 4B) were fine-tuned using QLoRA on domain-specific CRM data and compared with the commercial baseline GPT-4o-mini. Evaluation included semantic metrics (BERTScore and embedding cosine similarity), traditional n-gram metrics (BLEU and ROUGE-L), and operational indicators such as inference latency and memory footprint.Results: GPT-4o-mini achieved the highest semantic performance (BERTScore: 0.731; embedding similarity: 0.781) while maintaining low inference latency (1.89 s). Among the open-source alternatives, Mistral 7B achieved the best results (BERTScore: 0.711; similarity: 0.699). The findings demonstrate that open-source models can provide competitive performance while offering advantages in deployment flexibility and governance.Conclusion: The study provides a practical framework for organizations deploying conversational AI in enterprise information systems, supporting decisions related to Total Cost of Ownership, operational efficiency, and data governance. Future research should further validate retrieval-aware metrics against human expert assessments and user-based task evaluations to better assess the business value of enterprise RAG systems.