Nonlinear Regression Modeling via Machine Learning Techniques with Applications in Business and Economics

The paper demonstrates applications of machine learning techniques to economic data. The techniques include nonlinear regression, generalized additive models (GAM), regression trees, bagging, random forest, boosting, and multivariate adaptive regression splines (MARS). Their relative model fitting and forecasting performance are studied. Common algorithms for implementing these techniques and their relative merits and shortcomings are discussed. Performance comparisons among these techniques are carried out via their application to the current population survey (CPS) data on wages and Boston housing data. Overfitting and post-selection inference issues associated with these techniques are also investigated. Our results suggest that the recently developed adaptive machine learning techniques of random forests, boosting, GAM and MARS outperform nonlinear regression model with Gaussian errors and can be scaled to bigger data sets by fitting a rich class of functions almost automatically.