Combining Generalizers Using Partitions of the Learning Set

For any real-world generalization problem, there are always many generalizers which could be applied to the problem. This chapter discusses some algorithmic techniques for dealing with this multiplicity of possible generalizers. All of these techniques rely on partitioning the provided learning set in two, many different times. The first technique discussed is cross-validation, which is a winner-takes-all strategy (based on the behavior of the generalizers on the partitions of the learning set, it picks one single generalizer from amongst the set of candidate generalizers, and tells you to use that generalizer). The second technique discussed, the one this chapter concentrates on, is an extension of cross-validation called stacked generalization. As opposed to cross-validation's winner-takes-all strategy, stacked generalization uses the partitions of the learning set to combine the generalizers, in a non-linear manner, via another generalizer (hence the term ``stacked generalization''). This chapter ends by discussing some possible extensions of stacked generalization.