Bayesian identification, selection and estimation of semiparametric functions in high-dimensional additive models
AbstractIn this paper we propose an approach to both estimate and select unknown smooth functions in an additive model with potentially many functions. Each function is written as a linear combination of basis terms, with coefficients regularized by a proper linearly constrained Gaussian prior. Given any potentially rank deficient prior precision matrix, we show how to derive linear constraints so that the corresponding effect is identified in the additive model. This allows for the use of a wide range of bases and precision matrices in priors for regularization. By introducing indicator variables, each constrained Gaussian prior is augmented with a point mass at zero, thus allowing for function selection. Posterior inference is calculated using Markov chain Monte Carlo and the smoothness in the functions is both the result of shrinkage through the constrained Gaussian prior and model averaging. We show how using non-degenerate priors on the shrinkage parameters enables the application of substantially more computationally efficient sampling schemes than would otherwise be the case. We show the favourable performance of our approach when compared to two contemporary alternative Bayesian methods. To highlight the potential of our approach in high-dimensional settings we apply it to estimate two large seemingly unrelated regression models for intra-day electricity load. Both models feature a variety of different univariate and bivariate functions which require different levels of smoothing, and where component selection is meaningful. Priors for the error disturbance covariances are selected carefully and the empirical results provide a substantive contribution to the electricity load modelling literature in their own right.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.
Bibliographic InfoArticle provided by Elsevier in its journal Journal of Econometrics.
Volume (Year): 143 (2008)
Issue (Month): 2 (April)
Contact details of provider:
Web page: http://www.elsevier.com/locate/jeconom
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Carmen Fernández & Eduardo Ley & Mark F. J. Steel, .
"Benchmark priors for Bayesian Model averaging,"
- Carmen Fernandez & Eduardo Ley & Mark F.J. Steel, 1998. "Benchmark Priors for Bayesian Model Averaging," Econometrics 9804001, EconWPA, revised 31 Jul 1999.
- Carmen Fernandez & E Ley & Mark F J Steel, 2004. "Benchmark priors for Bayesian models averaging," ESE Discussion Papers 66, Edinburgh School of Economics, University of Edinburgh.
- Geweke, John & Keane, Michael, 2007. "Smoothly mixing regressions," Journal of Econometrics, Elsevier, vol. 138(1), pages 252-290, May.
- David J. Nott & Robert Kohn, 2005. "Adaptive sampling for Bayesian variable selection," Biometrika, Biometrika Trust, vol. 92(4), pages 747-763, December.
- Smith, M. & Wong, C.M. & Kohn, R., 1996. "Additive Nonparametric Regression with Autocorrelated Errors," Monash Econometrics and Business Statistics Working Papers 19/96, Monash University, Department of Econometrics and Business Statistics.
- Alexandre Pintore & Paul Speckman & Chris C. Holmes, 2006. "Spatially adaptive smoothing splines," Biometrika, Biometrika Trust, vol. 93(1), pages 113-125, March.
- Sangjoon Kim & Neil Shephard, 1994.
"Stochastic volatility: likelihood inference and comparison with ARCH models,"
3., Economics Group, Nuffield College, University of Oxford.
- Kim, Sangjoon & Shephard, Neil & Chib, Siddhartha, 1998. "Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models," Review of Economic Studies, Wiley Blackwell, vol. 65(3), pages 361-93, July.
- Sangjoon Kim, Neil Shephard & Siddhartha Chib, . "Stochastic volatility: likelihood inference and comparison with ARCH models," Economics Papers W26, revised version of W, Economics Group, Nuffield College, University of Oxford.
- Sangjoon Kim & Neil Shephard & Siddhartha Chib, 1996. "Stochastic Volatility: Likelihood Inference And Comparison With Arch Models," Econometrics 9610002, EconWPA.
- Smith, M. & Kohn, R., 1998.
"Nonparametric Seemingly Unrelated Regression,"
Monash Econometrics and Business Statistics Working Papers
7/98, Monash University, Department of Econometrics and Business Statistics.
- Koop, Gary M & Poirier, Dale J & Tobias, Justin, 2005.
"Semiparametric Bayesian Inference in Multiple Equation Models,"
Staff General Research Papers
12009, Iowa State University, Department of Economics.
- Dale J. Poirier & Gary Koop & Justin Tobias, 2005. "Semiparametric Bayesian inference in multiple equation models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(6), pages 723-747.
- Smith M. & Kohn R., 2002. "Parsimonious Covariance Matrix Estimation for Longitudinal Data," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1141-1153, December.
- Smith, M. & Kohn, R., .
"Nonparametric Regression using Bayesian Variable Selection,"
Statistics Working Paper
_009, Australian Graduate School of Management.
- Smith, Michael & Kohn, Robert, 1996. "Nonparametric regression using Bayesian variable selection," Journal of Econometrics, Elsevier, vol. 75(2), pages 317-343, December.
- Chib, Siddhartha & Jeliazkov, Ivan, 2006. "Inference in Semiparametric Dynamic Models for Binary Longitudinal Data," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 685-700, June.
- Ramanathan, Ramu & Engle, Robert & Granger, Clive W. J. & Vahid-Araghi, Farshid & Brace, Casey, 1997. "Shorte-run forecasts of electricity loads and peaks," International Journal of Forecasting, Elsevier, vol. 13(2), pages 161-174, June.
- Koop, G. & Poirier, D., 2000.
"Bayesian Variants of Some Classical Semiparametric Regression Techniques,"
00-01-22, California Irvine - School of Social Sciences.
- Koop, Gary & Poirier, Dale J., 2004. "Bayesian variants of some classical semiparametric regression techniques," Journal of Econometrics, Elsevier, vol. 123(2), pages 259-282, December.
- Gary Koop & Dale J. Poirer, 2004. "Bayesian Variants of Some classical Semiparametric Regression Techniques," ESE Discussion Papers 73, Edinburgh School of Economics, University of Edinburgh.
- Smith, M. & Yau, P. & Shively, T. & Kohn, R., 1998. "Estimating Long-Term Trends in Tropospheric Ozone Levels," Monash Econometrics and Business Statistics Working Papers 2/98, Monash University, Department of Econometrics and Business Statistics.
- Wong, Chi-ming & Kohn, Robert, 1996. "A Bayesian approach to additive semiparametric regression," Journal of Econometrics, Elsevier, vol. 74(2), pages 209-235, October.
- Cottet R. & Smith M., 2003. "Bayesian Modeling and Forecasting of Intraday Electricity Load," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 839-849, January.
- Patrick J. Wolfe & Simon J. Godsill & Wee-Jing Ng, 2004. "Bayesian variable selection and regularization for time-frequency surface estimation," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(3), pages 575-589.
- Shively, Thomas S. & Walker, Stephen G. & Damien, Paul, 2011. "Nonparametric function estimation subject to monotonicity, convexity and other shape constraints," Journal of Econometrics, Elsevier, vol. 161(2), pages 166-181, April.
- Xin-Yuan Song & Zhao-Hua Lu & Jing-Heng Cai & Edward Ip, 2013. "A Bayesian Modeling Approach for Generalized Semiparametric Structural Equation Models," Psychometrika, Springer, vol. 78(4), pages 624-647, October.
- Smith, Michael S. & Kauermann, Göran, 2011. "Bicycle commuting in Melbourne during the 2000s energy crisis: A semiparametric analysis of intraday volumes," Transportation Research Part B: Methodological, Elsevier, vol. 45(10), pages 1846-1862.
- Mestekemper, Thomas & Kauermann, Göran & Smith, Michael S., 2013. "A comparison of periodic autoregressive and dynamic factor models in intraday energy demand forecasting," International Journal of Forecasting, Elsevier, vol. 29(1), pages 1-12.
- Fabian Scheipl & Thomas Kneib & Ludwig Fahrmeir, 2013. "Penalized likelihood and Bayesian function selection in regression models," AStA Advances in Statistical Analysis, Springer, vol. 97(4), pages 349-385, October.
- Panagiotelis, Anastasios & Smith, Michael, 2010. "Bayesian skew selection for multivariate models," Computational Statistics & Data Analysis, Elsevier, vol. 54(7), pages 1824-1839, July.
- Chen, Xue-Dong & Tang, Nian-Sheng, 2010. "Bayesian analysis of semiparametric reproductive dispersion mixed-effects models," Computational Statistics & Data Analysis, Elsevier, vol. 54(9), pages 2145-2158, September.
- Felix Heinzl & Ludwig Fahrmeir & Thomas Kneib, 2012. "Additive mixed models with Dirichlet process mixture and P-spline priors," AStA Advances in Statistical Analysis, Springer, vol. 96(1), pages 47-68, January.
- Stefan Lang & Nikolaus Umlauf & Peter Wechselberger & Kenneth Harttgen & Thomas Kneib, 2012. "Multilevel structured additive regression," Working Papers 2012-07, Faculty of Economics and Statistics, University of Innsbruck.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei).
If references are entirely missing, you can add them using this form.