Classification and Regression Trees

This chapter discusses Classification and Regression Trees, widely used in data mining for predictive analytics. The chapter starts by explaining the two principal types of decision trees: classification trees and regression trees. In a classification tree, the dependent variable is categorical, while in a regression tree, it is continuous. The first section discusses classification trees, using an example of customer targeting in a marketing campaign. The chapter emphasizes that classification trees are “automatic” models, as they select independent variables by searching for optimal splits based on measures of purity or entropy. The second section covers regression trees, illustrating their application in predicting continuous target variables using an example of head acceleration measurements from simulated motorcycle accidents. The chapter explores the development of classification trees, explaining how splitting nodes are continued until they are pure or no further splits are possible. It emphasizes the importance of pruning to avoid overfitting, which can lead to poor generalization with unseen data. The author discusses different pruning techniques, including pre-pruning and post-pruning. Pre-pruning involves setting stopping rules during tree growth, while post-pruning involves trimming the tree after it is fully grown. The strengths and weaknesses of decision trees are highlighted. The interpretability and intuitiveness of decision trees are listed as strengths, while the risk of overfitting and sensitivity to minor data changes are cited as weaknesses. Overall, this chapter provides a comprehensive overview of decision trees, their applications, and essential considerations for creating accurate and robust models using this popular data mining technique.