Advanced Search
MyIDEAS: Login to save this article or follow this journal

Bayesian identification, selection and estimation of semiparametric functions in high-dimensional additive models


Author Info

  • Panagiotelis, Anastasios
  • Smith, Michael


In this paper we propose an approach to both estimate and select unknown smooth functions in an additive model with potentially many functions. Each function is written as a linear combination of basis terms, with coefficients regularized by a proper linearly constrained Gaussian prior. Given any potentially rank deficient prior precision matrix, we show how to derive linear constraints so that the corresponding effect is identified in the additive model. This allows for the use of a wide range of bases and precision matrices in priors for regularization. By introducing indicator variables, each constrained Gaussian prior is augmented with a point mass at zero, thus allowing for function selection. Posterior inference is calculated using Markov chain Monte Carlo and the smoothness in the functions is both the result of shrinkage through the constrained Gaussian prior and model averaging. We show how using non-degenerate priors on the shrinkage parameters enables the application of substantially more computationally efficient sampling schemes than would otherwise be the case. We show the favourable performance of our approach when compared to two contemporary alternative Bayesian methods. To highlight the potential of our approach in high-dimensional settings we apply it to estimate two large seemingly unrelated regression models for intra-day electricity load. Both models feature a variety of different univariate and bivariate functions which require different levels of smoothing, and where component selection is meaningful. Priors for the error disturbance covariances are selected carefully and the empirical results provide a substantive contribution to the electricity load modelling literature in their own right.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL:
Download Restriction: Full text for ScienceDirect subscribers only

As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

Bibliographic Info

Article provided by Elsevier in its journal Journal of Econometrics.

Volume (Year): 143 (2008)
Issue (Month): 2 (April)
Pages: 291-316

as in new window
Handle: RePEc:eee:econom:v:143:y:2008:i:2:p:291-316

Contact details of provider:
Web page:

Related research



References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Carmen Fernández & Eduardo Ley & Mark F. J. Steel, . "Benchmark priors for Bayesian Model averaging," Working Papers 98-06, FEDEA.
  2. Geweke, John & Keane, Michael, 2007. "Smoothly mixing regressions," Journal of Econometrics, Elsevier, vol. 138(1), pages 252-290, May.
  3. David J. Nott & Robert Kohn, 2005. "Adaptive sampling for Bayesian variable selection," Biometrika, Biometrika Trust, vol. 92(4), pages 747-763, December.
  4. Smith, M. & Wong, C.M. & Kohn, R., 1996. "Additive Nonparametric Regression with Autocorrelated Errors," Monash Econometrics and Business Statistics Working Papers 19/96, Monash University, Department of Econometrics and Business Statistics.
  5. Alexandre Pintore & Paul Speckman & Chris C. Holmes, 2006. "Spatially adaptive smoothing splines," Biometrika, Biometrika Trust, vol. 93(1), pages 113-125, March.
  6. Sangjoon Kim & Neil Shephard, 1994. "Stochastic volatility: likelihood inference and comparison with ARCH models," Economics Papers 3., Economics Group, Nuffield College, University of Oxford.
  7. Smith, M. & Kohn, R., 1998. "Nonparametric Seemingly Unrelated Regression," Monash Econometrics and Business Statistics Working Papers 7/98, Monash University, Department of Econometrics and Business Statistics.
  8. Koop, Gary M & Poirier, Dale J & Tobias, Justin, 2005. "Semiparametric Bayesian Inference in Multiple Equation Models," Staff General Research Papers 12009, Iowa State University, Department of Economics.
  9. Smith M. & Kohn R., 2002. "Parsimonious Covariance Matrix Estimation for Longitudinal Data," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1141-1153, December.
  10. Smith, M. & Kohn, R., . "Nonparametric Regression using Bayesian Variable Selection," Statistics Working Paper _009, Australian Graduate School of Management.
  11. Chib, Siddhartha & Jeliazkov, Ivan, 2006. "Inference in Semiparametric Dynamic Models for Binary Longitudinal Data," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 685-700, June.
  12. Ramanathan, Ramu & Engle, Robert & Granger, Clive W. J. & Vahid-Araghi, Farshid & Brace, Casey, 1997. "Shorte-run forecasts of electricity loads and peaks," International Journal of Forecasting, Elsevier, vol. 13(2), pages 161-174, June.
  13. Koop, G. & Poirier, D., 2000. "Bayesian Variants of Some Classical Semiparametric Regression Techniques," Papers 00-01-22, California Irvine - School of Social Sciences.
  14. Smith, M. & Yau, P. & Shively, T. & Kohn, R., 1998. "Estimating Long-Term Trends in Tropospheric Ozone Levels," Monash Econometrics and Business Statistics Working Papers 2/98, Monash University, Department of Econometrics and Business Statistics.
  15. Wong, Chi-ming & Kohn, Robert, 1996. "A Bayesian approach to additive semiparametric regression," Journal of Econometrics, Elsevier, vol. 74(2), pages 209-235, October.
  16. Cottet R. & Smith M., 2003. "Bayesian Modeling and Forecasting of Intraday Electricity Load," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 839-849, January.
  17. Patrick J. Wolfe & Simon J. Godsill & Wee-Jing Ng, 2004. "Bayesian variable selection and regularization for time-frequency surface estimation," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(3), pages 575-589.
Full references (including those not matched with items on IDEAS)


Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Shively, Thomas S. & Walker, Stephen G. & Damien, Paul, 2011. "Nonparametric function estimation subject to monotonicity, convexity and other shape constraints," Journal of Econometrics, Elsevier, vol. 161(2), pages 166-181, April.
  2. Xin-Yuan Song & Zhao-Hua Lu & Jing-Heng Cai & Edward Ip, 2013. "A Bayesian Modeling Approach for Generalized Semiparametric Structural Equation Models," Psychometrika, Springer, vol. 78(4), pages 624-647, October.
  3. Smith, Michael S. & Kauermann, Göran, 2011. "Bicycle commuting in Melbourne during the 2000s energy crisis: A semiparametric analysis of intraday volumes," Transportation Research Part B: Methodological, Elsevier, vol. 45(10), pages 1846-1862.
  4. Mestekemper, Thomas & Kauermann, Göran & Smith, Michael S., 2013. "A comparison of periodic autoregressive and dynamic factor models in intraday energy demand forecasting," International Journal of Forecasting, Elsevier, vol. 29(1), pages 1-12.
  5. Fabian Scheipl & Thomas Kneib & Ludwig Fahrmeir, 2013. "Penalized likelihood and Bayesian function selection in regression models," AStA Advances in Statistical Analysis, Springer, vol. 97(4), pages 349-385, October.
  6. Panagiotelis, Anastasios & Smith, Michael, 2010. "Bayesian skew selection for multivariate models," Computational Statistics & Data Analysis, Elsevier, vol. 54(7), pages 1824-1839, July.
  7. Chen, Xue-Dong & Tang, Nian-Sheng, 2010. "Bayesian analysis of semiparametric reproductive dispersion mixed-effects models," Computational Statistics & Data Analysis, Elsevier, vol. 54(9), pages 2145-2158, September.
  8. Felix Heinzl & Ludwig Fahrmeir & Thomas Kneib, 2012. "Additive mixed models with Dirichlet process mixture and P-spline priors," AStA Advances in Statistical Analysis, Springer, vol. 96(1), pages 47-68, January.
  9. Stefan Lang & Nikolaus Umlauf & Peter Wechselberger & Kenneth Harttgen & Thomas Kneib, 2012. "Multilevel structured additive regression," Working Papers 2012-07, Faculty of Economics and Statistics, University of Innsbruck.


This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.


Access and download statistics


When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:143:y:2008:i:2:p:291-316. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.