Analysis of feature selection measures for text categorisation

The curse of dimensionality has made dimension reduction an essential step in text categorisation. Feature selection is an approach for dimension reduction. In this paper an analysis on feature selection measures for text categorisation is performed. Under the unsupervised approach document frequency and under the supervised approach chi-square, odds ratio, mutual information, and information gain are considered for analysis. They are considered here because they are the widely used and effective measures. Analysis of these measures is performed using the 20 newsgroups dataset. Twenty newsgroups dataset consists of closely related categories as well as highly unrelated categories. Certain categories of 20 newsgroups dataset are selected and organised into three groups of overlapping (highly related) classes, non-overlapping (highly unrelated) classes and combination of overlapping and non-overlapping classes. Feature selection and subsequent classification is applied to the three groups separately and the classification performance is studied based on the feature selection measures. The noticeable behaviour was with odds ratio measure in that it performed well for non-overlapping group and overlapping groups considered separately and was poorer in performance for the group containing both overlapping and non-overlapping categories. Remaining measures showed consistent behaviour for all the three groups. Classification was achieved using support vector machine classifier. The performance comparisons of different measures on different groups are presented in terms of micro-F1 and macro-F1.