Comparing representations of a discipline derived through LDA vs. intellectual content analysis: the case of information science

The paper looks at the methodology of empirical analyses of the content and structure of Information Science (IS). The traditional approach in empirical analysis is intellectual content analysis (ICA) of a representative data set. The high labor cost prohibits the analysis of massive data sets. A recent alternative is based on data mining/machine learning. Its strength is the capability of analyzing massive datasets efficiently. However, a significant issue is the quality of content analysis. The paper compares latent Dirichlet allocation/topic modeling (LDA/TM) based statistical analysis to ICA using the same data set, 1514 scholarly articles from the year 2015 volumes of 30 IS journals. The intellectual analysis provides the mirror for reflecting the TM results. LDA/TM is strong in identifying new directions of a discipline and processing masses of text. Its weaknesses include semantic haziness of topics due to bag-of-words article representation, text pre-processing, tuning of parameters, and being unanalytic in composing topics from words belonging to different categories.