Bayesian identification, selection and estimation of semiparametric functions in high-dimensional additive models
In this paper we propose an approach to both estimate and select unknown smooth functions in an additive model with potentially many functions. Each function is written as a linear combination of basis terms, with coefficients regularized by a proper linearly constrained Gaussian prior. Given any potentially rank deficient prior precision matrix, we show how to derive linear constraints so that the corresponding effect is identified in the additive model. This allows for the use of a wide range of bases and precision matrices in priors for regularization. By introducing indicator variables, each constrained Gaussian prior is augmented with a point mass at zero, thus allowing for function selection. Posterior inference is calculated using Markov chain Monte Carlo and the smoothness in the functions is both the result of shrinkage through the constrained Gaussian prior and model averaging. We show how using non-degenerate priors on the shrinkage parameters enables the application of substantially more computationally efficient sampling schemes than would otherwise be the case. We show the favourable performance of our approach when compared to two contemporary alternative Bayesian methods. To highlight the potential of our approach in high-dimensional settings we apply it to estimate two large seemingly unrelated regression models for intra-day electricity load. Both models feature a variety of different univariate and bivariate functions which require different levels of smoothing, and where component selection is meaningful. Priors for the error disturbance covariances are selected carefully and the empirical results provide a substantive contribution to the electricity load modelling literature in their own right.
If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Cottet R. & Smith M., 2003. "Bayesian Modeling and Forecasting of Intraday Electricity Load," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 839-849, January.
- Koop, Gary & Poirier, Dale J., 2004.
"Bayesian variants of some classical semiparametric regression techniques,"
Journal of Econometrics,
Elsevier, vol. 123(2), pages 259-282, December.
- Gary Koop & Dale J Poirer, 2001. "Bayesian Variants of Some classical Semiparametric Regression Techniques," ESE Discussion Papers 73, Edinburgh School of Economics, University of Edinburgh.
- Koop, G. & Poirier, D., 2000. "Bayesian Variants of Some Classical Semiparametric Regression Techniques," Papers 00-01-22, California Irvine - School of Social Sciences.
- Smith, M. & Wong, C.M. & Kohn, R., 1996. "Additive Nonparametric Regression with Autocorrelated Errors," Monash Econometrics and Business Statistics Working Papers 19/96, Monash University, Department of Econometrics and Business Statistics.
- Sangjoon Kim & Neil Shephard & Siddhartha Chib, 1998. "Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models," Review of Economic Studies, Oxford University Press, vol. 65(3), pages 361-393.
- Smith, M. & Kohn, R., .
"Nonparametric Regression using Bayesian Variable Selection,"
Statistics Working Paper
_009, Australian Graduate School of Management.
- Smith, Michael & Kohn, Robert, 1996. "Nonparametric regression using Bayesian variable selection," Journal of Econometrics, Elsevier, vol. 75(2), pages 317-343, December.
- Dale J. Poirier & Gary Koop & Justin Tobias, 2005.
"Semiparametric Bayesian inference in multiple equation models,"
Journal of Applied Econometrics,
John Wiley & Sons, Ltd., vol. 20(6), pages 723-747.
- Koop, Gary M & Poirier, Dale J & Tobias, Justin, 2005. "Semiparametric Bayesian Inference in Multiple Equation Models," Staff General Research Papers 12009, Iowa State University, Department of Economics.
- Alexandre Pintore & Paul Speckman & Chris C. Holmes, 2006. "Spatially adaptive smoothing splines," Biometrika, Biometrika Trust, vol. 93(1), pages 113-125, March.
- Smith, M. & Kohn, R., 1998.
"Nonparametric Seemingly Unrelated Regression,"
Monash Econometrics and Business Statistics Working Papers
7/98, Monash University, Department of Econometrics and Business Statistics.
- Geweke, John & Keane, Michael, 2007. "Smoothly mixing regressions," Journal of Econometrics, Elsevier, vol. 138(1), pages 252-290, May.
- Chib, Siddhartha & Jeliazkov, Ivan, 2006. "Inference in Semiparametric Dynamic Models for Binary Longitudinal Data," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 685-700, June.
- Carmen Fernandez & Eduardo Ley & Mark F J Steel, 1998.
"Benchmark priors for Bayesian model averaging,"
ESE Discussion Papers
26, Edinburgh School of Economics, University of Edinburgh.
- Carmen Fernandez & Eduardo Ley & Mark F J Steel, 1998. "Benchmark priors for Bayesian model averaging," ESE Discussion Papers 66, Edinburgh School of Economics, University of Edinburgh.
- Carmen Fernández & Eduardo Ley & Mark F. J. Steel, . "Benchmark priors for Bayesian Model averaging," Working Papers 98-06, FEDEA.
- Carmen Fernandez & Eduardo Ley & Mark F.J. Steel, 1998. "Benchmark Priors for Bayesian Model Averaging," Econometrics 9804001, EconWPA, revised 31 Jul 1999.
- David J. Nott & Robert Kohn, 2005. "Adaptive sampling for Bayesian variable selection," Biometrika, Biometrika Trust, vol. 92(4), pages 747-763, December.
- Ramanathan, Ramu & Engle, Robert & Granger, Clive W. J. & Vahid-Araghi, Farshid & Brace, Casey, 1997. "Shorte-run forecasts of electricity loads and peaks," International Journal of Forecasting, Elsevier, vol. 13(2), pages 161-174, June.
- Smith M. & Kohn R., 2002. "Parsimonious Covariance Matrix Estimation for Longitudinal Data," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1141-1153, December.
- Wong, Chi-ming & Kohn, Robert, 1996. "A Bayesian approach to additive semiparametric regression," Journal of Econometrics, Elsevier, vol. 74(2), pages 209-235, October.
- Smith, M. & Yau, P. & Shively, T. & Kohn, R., 1998. "Estimating Long-Term Trends in Tropospheric Ozone Levels," Monash Econometrics and Business Statistics Working Papers 2/98, Monash University, Department of Econometrics and Business Statistics.
- Patrick J. Wolfe & Simon J. Godsill & Wee-Jing Ng, 2004. "Bayesian variable selection and regularization for time-frequency surface estimation," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(3), pages 575-589.
When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:143:y:2008:i:2:p:291-316. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei)
If references are entirely missing, you can add them using this form.