Do Prior Information on Performance of Individual Classifiers for Fusion of Probabilistic Classifier Outputs Matter?

In this paper, a class of classifier fusion methods are compared to verify the impact of the use of some prior information about individual classifiers during fusion of probabilistic classifier outputs. In particular, we compare two versions (i.e., uninformed and informed versions) of a performance-agnostic fusion of probabilistic classifier outputs from Masakuna et al. (2020) (called Yayambo). Yayambo is iterative and treated black-box classifiers. For this paper, cases where prior information, i.e., performances of individual classifiers in the form of accuracy is taken into account for fusion of classifier outputs, are considered. Then we discuss the relevance of prior information for combination of probabilistic classifier outputs. The experiments have demonstrated that classifier fusion methods, for both informed and uninformed fusion methods, achieve different performances, i.e., the differences are significant in general (using the p-value and the effect size (Gail & Richard, 2012)). Surprisingly, in some particular cases and under the same experimental conditions, the two versions of Yayambo achieve similar results (using the $$p-$$ p - value). This means that one might not need to carefully, for some situations, select a classifier fusion method. We consider 12 classifier fusion methods (5 uninformed and 7 informed), use 8 data sets and apply different experimental settings to address our research question.