Bayesian identification, selection and estimation of semiparametric functions in high-dimensional additive models
AbstractIn this paper we propose an approach to both estimate and select unknown smooth functions in an additive model with potentially many functions. Each function is written as a linear combination of basis terms, with coefficients regularized by a proper linearly constrained Gaussian prior. Given any potentially rank deficient prior precision matrix, we show how to derive linear constraints so that the corresponding effect is identified in the additive model. This allows for the use of a wide range of bases and precision matrices in priors for regularization. By introducing indicator variables, each constrained Gaussian prior is augmented with a point mass at zero, thus allowing for function selection. Posterior inference is calculated using Markov chain Monte Carlo and the smoothness in the functions is both the result of shrinkage through the constrained Gaussian prior and model averaging. We show how using non-degenerate priors on the shrinkage parameters enables the application of substantially more computationally efficient sampling schemes than would otherwise be the case. We show the favourable performance of our approach when compared to two contemporary alternative Bayesian methods. To highlight the potential of our approach in high-dimensional settings we apply it to estimate two large seemingly unrelated regression models for intra-day electricity load. Both models feature a variety of different univariate and bivariate functions which require different levels of smoothing, and where component selection is meaningful. Priors for the error disturbance covariances are selected carefully and the empirical results provide a substantive contribution to the electricity load modelling literature in their own right.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.
Bibliographic InfoArticle provided by Elsevier in its journal Journal of Econometrics.
Volume (Year): 143 (2008)
Issue (Month): 2 (April)
Contact details of provider:
Web page: http://www.elsevier.com/locate/jeconom
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Cottet R. & Smith M., 2003. "Bayesian Modeling and Forecasting of Intraday Electricity Load," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 839-849, January.
- Wong, Chi-ming & Kohn, Robert, 1996. "A Bayesian approach to additive semiparametric regression," Journal of Econometrics, Elsevier, vol. 74(2), pages 209-235, October.
- Koop, Gary & Poirier, Dale J., 2004.
"Bayesian variants of some classical semiparametric regression techniques,"
Journal of Econometrics,
Elsevier, vol. 123(2), pages 259-282, December.
- Gary Koop & Dale J. Poirer, 2004. "Bayesian Variants of Some classical Semiparametric Regression Techniques," ESE Discussion Papers 73, Edinburgh School of Economics, University of Edinburgh.
- Koop, G. & Poirier, D., 2000. "Bayesian Variants of Some Classical Semiparametric Regression Techniques," Papers 00-01-22, California Irvine - School of Social Sciences.
- Smith, M. & Yau, P. & Shively, T. & Kohn, R., 1998. "Estimating Long-Term Trends in Tropospheric Ozone Levels," Monash Econometrics and Business Statistics Working Papers 2/98, Monash University, Department of Econometrics and Business Statistics.
- Sangjoon Kim & Neil Shephard, 1994.
"Stochastic volatility: likelihood inference and comparison with ARCH models,"
3., Economics Group, Nuffield College, University of Oxford.
- Kim, Sangjoon & Shephard, Neil & Chib, Siddhartha, 1998. "Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models," Review of Economic Studies, Wiley Blackwell, vol. 65(3), pages 361-93, July.
- Sangjoon Kim, Neil Shephard & Siddhartha Chib, . "Stochastic volatility: likelihood inference and comparison with ARCH models," Economics Papers W26, revised version of W, Economics Group, Nuffield College, University of Oxford.
- Sangjoon Kim & Neil Shephard & Siddhartha Chib, 1996. "Stochastic Volatility: Likelihood Inference And Comparison With Arch Models," Econometrics 9610002, EconWPA.
- Smith, Michael & Kohn, Robert, 1996.
"Nonparametric regression using Bayesian variable selection,"
Journal of Econometrics,
Elsevier, vol. 75(2), pages 317-343, December.
- Smith, M. & Kohn, R., . "Nonparametric Regression using Bayesian Variable Selection," Statistics Working Paper _009, Australian Graduate School of Management.
- Smith, Michael & Kohn, Robert, 2000.
"Nonparametric seemingly unrelated regression,"
Journal of Econometrics,
Elsevier, vol. 98(2), pages 257-281, October.
- Smith, M. & Kohn, R., 1998. "Nonparametric Seemingly Unrelated Regression," Monash Econometrics and Business Statistics Working Papers 7/98, Monash University, Department of Econometrics and Business Statistics.
- Koop, Gary M & Poirier, Dale J & Tobias, Justin, 2005.
"Semiparametric Bayesian Inference in Multiple Equation Models,"
Staff General Research Papers
12009, Iowa State University, Department of Economics.
- Dale J. Poirier & Gary Koop & Justin Tobias, 2005. "Semiparametric Bayesian inference in multiple equation models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(6), pages 723-747.
- Carmen Fernandez & Eduardo Ley & Mark F.J. Steel, 1998.
"Benchmark Priors for Bayesian Model Averaging,"
9804001, EconWPA, revised 31 Jul 1999.
- Smith, M. & Wong, C.M. & Kohn, R., 1996. "Additive Nonparametric Regression with Autocorrelated Errors," Monash Econometrics and Business Statistics Working Papers 19/96, Monash University, Department of Econometrics and Business Statistics.
- David J. Nott & Robert Kohn, 2005. "Adaptive sampling for Bayesian variable selection," Biometrika, Biometrika Trust, vol. 92(4), pages 747-763, December.
- Smith M. & Kohn R., 2002. "Parsimonious Covariance Matrix Estimation for Longitudinal Data," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1141-1153, December.
- Ramanathan, Ramu & Engle, Robert & Granger, Clive W. J. & Vahid-Araghi, Farshid & Brace, Casey, 1997. "Shorte-run forecasts of electricity loads and peaks," International Journal of Forecasting, Elsevier, vol. 13(2), pages 161-174, June.
- Geweke, John & Keane, Michael, 2007. "Smoothly mixing regressions," Journal of Econometrics, Elsevier, vol. 138(1), pages 252-290, May.
- Alexandre Pintore & Paul Speckman & Chris C. Holmes, 2006. "Spatially adaptive smoothing splines," Biometrika, Biometrika Trust, vol. 93(1), pages 113-125, March.
- Chib, Siddhartha & Jeliazkov, Ivan, 2006. "Inference in Semiparametric Dynamic Models for Binary Longitudinal Data," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 685-700, June.
- Chen, Xue-Dong & Tang, Nian-Sheng, 2010. "Bayesian analysis of semiparametric reproductive dispersion mixed-effects models," Computational Statistics & Data Analysis, Elsevier, vol. 54(9), pages 2145-2158, September.
- Shively, Thomas S. & Walker, Stephen G. & Damien, Paul, 2011. "Nonparametric function estimation subject to monotonicity, convexity and other shape constraints," Journal of Econometrics, Elsevier, vol. 161(2), pages 166-181, April.
- Mestekemper, Thomas & Kauermann, Göran & Smith, Michael S., 2013. "A comparison of periodic autoregressive and dynamic factor models in intraday energy demand forecasting," International Journal of Forecasting, Elsevier, vol. 29(1), pages 1-12.
- Panagiotelis, Anastasios & Smith, Michael, 2010. "Bayesian skew selection for multivariate models," Computational Statistics & Data Analysis, Elsevier, vol. 54(7), pages 1824-1839, July.
- Smith, Michael S. & Kauermann, Göran, 2011. "Bicycle commuting in Melbourne during the 2000s energy crisis: A semiparametric analysis of intraday volumes," Transportation Research Part B: Methodological, Elsevier, vol. 45(10), pages 1846-1862.
- Felix Heinzl & Ludwig Fahrmeir & Thomas Kneib, 2012. "Additive mixed models with Dirichlet process mixture and P-spline priors," AStA Advances in Statistical Analysis, Springer, vol. 96(1), pages 47-68, January.
- Stefan Lang & Nikolaus Umlauf & Peter Wechselberger & Kenneth Harttgen & Thomas Kneib, 2012. "Multilevel structured additive regression," Working Papers 2012-07, Faculty of Economics and Statistics, University of Innsbruck.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Wendy Shamier).
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If references are entirely missing, you can add them using this form.
If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.
Please note that corrections may take a couple of weeks to filter through the various RePEc services.