IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v143y2008i2p291-316.html
   My bibliography  Save this article

Bayesian identification, selection and estimation of semiparametric functions in high-dimensional additive models

Author

Listed:
  • Panagiotelis, Anastasios
  • Smith, Michael

Abstract

In this paper we propose an approach to both estimate and select unknown smooth functions in an additive model with potentially many functions. Each function is written as a linear combination of basis terms, with coefficients regularized by a proper linearly constrained Gaussian prior. Given any potentially rank deficient prior precision matrix, we show how to derive linear constraints so that the corresponding effect is identified in the additive model. This allows for the use of a wide range of bases and precision matrices in priors for regularization. By introducing indicator variables, each constrained Gaussian prior is augmented with a point mass at zero, thus allowing for function selection. Posterior inference is calculated using Markov chain Monte Carlo and the smoothness in the functions is both the result of shrinkage through the constrained Gaussian prior and model averaging. We show how using non-degenerate priors on the shrinkage parameters enables the application of substantially more computationally efficient sampling schemes than would otherwise be the case. We show the favourable performance of our approach when compared to two contemporary alternative Bayesian methods. To highlight the potential of our approach in high-dimensional settings we apply it to estimate two large seemingly unrelated regression models for intra-day electricity load. Both models feature a variety of different univariate and bivariate functions which require different levels of smoothing, and where component selection is meaningful. Priors for the error disturbance covariances are selected carefully and the empirical results provide a substantive contribution to the electricity load modelling literature in their own right.

Suggested Citation

  • Panagiotelis, Anastasios & Smith, Michael, 2008. "Bayesian identification, selection and estimation of semiparametric functions in high-dimensional additive models," Journal of Econometrics, Elsevier, vol. 143(2), pages 291-316, April.
  • Handle: RePEc:eee:econom:v:143:y:2008:i:2:p:291-316
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304-4076(07)00214-X
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Smith, Michael & Kohn, Robert, 2000. "Nonparametric seemingly unrelated regression," Journal of Econometrics, Elsevier, vol. 98(2), pages 257-281, October.
    2. Koop, Gary & Poirier, Dale J., 2004. "Bayesian variants of some classical semiparametric regression techniques," Journal of Econometrics, Elsevier, vol. 123(2), pages 259-282, December.
    3. Fernandez, Carmen & Ley, Eduardo & Steel, Mark F. J., 2001. "Benchmark priors for Bayesian model averaging," Journal of Econometrics, Elsevier, vol. 100(2), pages 381-427, February.
    4. David J. Nott & Robert Kohn, 2005. "Adaptive sampling for Bayesian variable selection," Biometrika, Biometrika Trust, vol. 92(4), pages 747-763, December.
    5. Smith, M. & Wong, C.M. & Kohn, R., 1996. "Additive Nonparametric Regression with Autocorrelated Errors," Monash Econometrics and Business Statistics Working Papers 19/96, Monash University, Department of Econometrics and Business Statistics.
    6. Sangjoon Kim & Neil Shephard & Siddhartha Chib, 1998. "Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models," Review of Economic Studies, Oxford University Press, vol. 65(3), pages 361-393.
    7. Alexandre Pintore & Paul Speckman & Chris C. Holmes, 2006. "Spatially adaptive smoothing splines," Biometrika, Biometrika Trust, vol. 93(1), pages 113-125, March.
    8. Ramanathan, Ramu & Engle, Robert & Granger, Clive W. J. & Vahid-Araghi, Farshid & Brace, Casey, 1997. "Shorte-run forecasts of electricity loads and peaks," International Journal of Forecasting, Elsevier, vol. 13(2), pages 161-174, June.
    9. Chib, Siddhartha & Jeliazkov, Ivan, 2006. "Inference in Semiparametric Dynamic Models for Binary Longitudinal Data," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 685-700, June.
    10. Smith, M. & Yau, P. & Shively, T. & Kohn, R., 1998. "Estimating Long-Term Trends in Tropospheric Ozone Levels," Monash Econometrics and Business Statistics Working Papers 2/98, Monash University, Department of Econometrics and Business Statistics.
    11. Smith, Michael & Kohn, Robert, 1996. "Nonparametric regression using Bayesian variable selection," Journal of Econometrics, Elsevier, vol. 75(2), pages 317-343, December.
    12. Cottet R. & Smith M., 2003. "Bayesian Modeling and Forecasting of Intraday Electricity Load," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 839-849, January.
    13. Geweke, John & Keane, Michael, 2007. "Smoothly mixing regressions," Journal of Econometrics, Elsevier, vol. 138(1), pages 252-290, May.
    14. Dale J. Poirier & Gary Koop & Justin Tobias, 2005. "Semiparametric Bayesian inference in multiple equation models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(6), pages 723-747.
    15. Smith M. & Kohn R., 2002. "Parsimonious Covariance Matrix Estimation for Longitudinal Data," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1141-1153, December.
    16. Patrick J. Wolfe & Simon J. Godsill & Wee-Jing Ng, 2004. "Bayesian variable selection and regularization for time-frequency surface estimation," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(3), pages 575-589.
    17. Wong, Chi-ming & Kohn, Robert, 1996. "A Bayesian approach to additive semiparametric regression," Journal of Econometrics, Elsevier, vol. 74(2), pages 209-235, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shively, Thomas S. & Walker, Stephen G. & Damien, Paul, 2011. "Nonparametric function estimation subject to monotonicity, convexity and other shape constraints," Journal of Econometrics, Elsevier, vol. 161(2), pages 166-181, April.
    2. Zhao, Kaifeng & Lian, Heng, 2016. "The Expectation–Maximization approach for Bayesian quantile regression," Computational Statistics & Data Analysis, Elsevier, vol. 96(C), pages 1-11.
    3. Duchwan Ryu & Erning Li & Bani K. Mallick, 2011. "Bayesian Nonparametric Regression Analysis of Data with Random Effects Covariates from Longitudinal Measurements," Biometrics, The International Biometric Society, vol. 67(2), pages 454-466, June.
    4. Aijun Yang & Xuejun Jiang & Lianjie Shu & Jinguan Lin, 2017. "Bayesian variable selection with sparse and correlation priors for high-dimensional data analysis," Computational Statistics, Springer, vol. 32(1), pages 127-143, March.
    5. Shao, Zhen & Gao, Fei & Zhang, Qiang & Yang, Shan-Lin, 2015. "Multivariate statistical and similarity measure based semiparametric modeling of the probability distribution: A novel approach to the case study of mid-long term electricity consumption forecasting i," Applied Energy, Elsevier, vol. 156(C), pages 502-518.
    6. Min Wang & Xiaoqian Sun & Tao Lu, 2015. "Bayesian structured variable selection in linear regression models," Computational Statistics, Springer, vol. 30(1), pages 205-229, March.
    7. Xin-Yuan Song & Zhao-Hua Lu & Jing-Heng Cai & Edward Ip, 2013. "A Bayesian Modeling Approach for Generalized Semiparametric Structural Equation Models," Psychometrika, Springer;The Psychometric Society, vol. 78(4), pages 624-647, October.
    8. Stefan Lang & Nikolaus Umlauf & Peter Wechselberger & Kenneth Harttgen & Thomas Kneib, 2012. "Multilevel structured additive regression," Working Papers 2012-07, Faculty of Economics and Statistics, University of Innsbruck.
    9. Felix Heinzl & Ludwig Fahrmeir & Thomas Kneib, 2012. "Additive mixed models with Dirichlet process mixture and P-spline priors," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 96(1), pages 47-68, January.
    10. Panagiotelis, Anastasios & Smith, Michael, 2010. "Bayesian skew selection for multivariate models," Computational Statistics & Data Analysis, Elsevier, vol. 54(7), pages 1824-1839, July.
    11. Chen, Xue-Dong & Tang, Nian-Sheng, 2010. "Bayesian analysis of semiparametric reproductive dispersion mixed-effects models," Computational Statistics & Data Analysis, Elsevier, vol. 54(9), pages 2145-2158, September.
    12. Peter J. Danaher & Michael S. Smith, 2011. "Modeling Multivariate Distributions Using Copulas: Applications in Marketing," Marketing Science, INFORMS, vol. 30(1), pages 4-21, 01-02.
    13. repec:eee:rensus:v:75:y:2017:i:c:p:123-136 is not listed on IDEAS
    14. Mestekemper, Thomas & Kauermann, Göran & Smith, Michael S., 2013. "A comparison of periodic autoregressive and dynamic factor models in intraday energy demand forecasting," International Journal of Forecasting, Elsevier, vol. 29(1), pages 1-12.
    15. Fabian Scheipl & Thomas Kneib & Ludwig Fahrmeir, 2013. "Penalized likelihood and Bayesian function selection in regression models," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 97(4), pages 349-385, October.
    16. Smith, Michael S. & Kauermann, Göran, 2011. "Bicycle commuting in Melbourne during the 2000s energy crisis: A semiparametric analysis of intraday volumes," Transportation Research Part B: Methodological, Elsevier, vol. 45(10), pages 1846-1862.
    17. Aijun Yang & Yunxian Li & Niansheng Tang & Jinguan Lin, 2015. "Bayesian variable selection in multinomial probit model for classifying high-dimensional data," Computational Statistics, Springer, vol. 30(2), pages 399-418, June.
    18. Zhao, Kaifeng & Lian, Heng, 2014. "Variational inferences for partially linear additive models with variable selection," Computational Statistics & Data Analysis, Elsevier, vol. 80(C), pages 223-239.
    19. Bin Jiang & Anastasios Panagiotelis & George Athanasopoulos & Rob Hyndman & Farshid Vahid, 2016. "Bayesian Rank Selection in Multivariate Regression," Monash Econometrics and Business Statistics Working Papers 6/16, Monash University, Department of Econometrics and Business Statistics.
    20. repec:kap:compec:v:51:y:2018:i:2:d:10.1007_s10614-017-9741-1 is not listed on IDEAS

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:143:y:2008:i:2:p:291-316. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Dana Niculescu). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.