Nonparametric Seemingly Unrelated Regression

This papers presents a method for simultaneously estimating a system of nonparametric multiple regressions which may seem unrelated, but where the errors are potentially correlated between equations. We show that the prime advantage of estimating such a 'seemingly unrelated' system of nonparametric regressions is that substantially less observations can be required to obtain reliable function estimates than if each of the regression equations was estimated separately and the correlation ignored. This increase in efficiency is investigated empirically using both simulated and real data. The method suggested here develops a Bayesian hierarchical framework where the regression function is represented as a linear combination of a large number of basis terms, the number of which is typically greater than the sample size. All the regression coefficients, and the variance matrix of the errors, are estimated simultaneously using their posterior means. The computation is carried out using a Markov chain Monte Carlo sampling scheme that employs a 'focused sampling' step to combat the high dimensional representation of the function and a Metropolis-Hastings step to correctly account for the distribution of the covariance matrix. The methodology is also easily extended to other nonparametric multivariate regression models.