Integrating Unsupervised Learning for the Factual Consistency of Generative Models

Text summarization involves analyzing large amounts of text, selecting the salient text features, and arranging them coherently. The graph-based TextRank and statistical topic modeling are unsupervised approaches for generating an extractive synopsis. Deep learning models are supervised, data-driven, and pre-trained on a huge corpus of data, making a significant contribution to automatic text summarization systems. Despite grammatical correctness and coherence, deep learning-based summarization systems are prone to factual inconsistency. This has hindered the applicability of transformer-based summarizers, particularly in critical domains where misleading summarization systems can lead to severe consequences due to their significant social impact. This work proposes an ingenious hybrid hierarchical approach that combines unsupervised approaches, such as the TextRank algorithm and Latent Dirichlet Allocation (LDA)-based summaries, with contemporary transformer-based language models. When validated on three benchmark summarization datasets, empirical results prove that our hybrid hierarchical transformer-based approach mitigates the factual inconsistency problem inherent in abstractive summarization. The improved summary consistency score of the abstractive summaries generated with our multilevel hybrid approach, in comparison to the fine-tuned baseline transformer-based language models, increases trust in transformer-based summarizers.