A Hybrid Model Integrating LDA, BERT, and Clustering for Enhanced Topic Modeling

Topic modeling is a popular machine learning technique in natural language processing for identifying themes within unstructured text. One of the most prominent methods for this purpose is Latent Dirichlet Allocation (LDA), which can automatically uncover topics from large text corpora. However, LDA alone may not always provide the best results. Using Bidirectional Encoder Representations from Transformers (BERT) embeddings in topic modeling, significantly enhances the quality and coherence of discovered topics by leveraging deep contextual representations of words. Clustering is another powerful unsupervised machine learning technique frequently used for topic modeling and information extraction from unstructured text. This study introduces a hybrid approach that combines LDA with BERT for enhanced topic modeling, incorporating dimensionality reduction-based clustering. To manage the increasing complexity and computational load of clustering with many features, Uniform Manifold Approximation and Projection is utilized for dimensionality reduction. Experiments conducted on benchmark datasets, specifically Reuters-21578 and 20newsgroups, illustrate the effectiveness of this cluster-informed topic modeling framework. The empirical results suggest that integrating clustering with BERT-LDA for topic modeling can be highly effective, as dimensionality reduction via clustering helps derive more cohesive topics. The study evaluates coherence scores using the BERT-LDA model on the 20newsgroups and Reuters datasets. For the 20newsgroups dataset, BERT-LDA shows a significant improvement in coherence scores: nearly 59% for 10 topics, 42% for 20 topics, 11% for 50 topics, and 16% for 98 topics. Similarly, for the Reuters dataset, coherence scores improved by about 85% for 10 topics, 63% for 20 topics, 43% for 50 topics, and 41% for 98 topics. These results highlight how BERT-LDA enhances topic coherence compared to traditional models.