Bayesian Tree Models for Survey Sample Data

Tree models are a popular and effective nonparametric modeling tool for data that depend on many variables that exhibit complex dependence, including interaction effects. Consequently, there are many potential applications for these models when dealing with survey data, which often contain many variables that are not independent from one another. One drawback of these models is that the specification is not stable, in that a few observations could affect the number of nodes and the variables included in the model. Also, obtaining a measure of uncertainty associated with these models is extremely challenging. Using a Bayesian representation naturally alleviates some of these concerns, as it automatically implies a distribution over tree space given the data as well as a distribution for the estimates produced. Since survey data are usually collected using an informative sample design, it is necessary to have an algorithm for creating tree-based models that account for this design during model estimation. In this article, we propose an algorithm and associated prior distribution assumptions to obtain a Bayesian tree model using data collected under an informative sample design. We demonstrate this proposed method using the Consumer Expenditure Survey and the Academic Performance Index datasets. Using an empirical simulation study, we show that the design-based Bayesian algorithm is an extremely flexible and robust way to construct regression tree models with measures of uncertainty that provide prediction intervals with the correct nominal coverage rates.