IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v53y2008i1p61-81.html
   My bibliography  Save this article

Simultaneous selection of variables and smoothing parameters in structured additive regression models

Author

Listed:
  • Belitz, Christiane
  • Lang, Stefan

Abstract

In recent years, considerable research has been devoted to developing complex regression models that can deal simultaneously with nonlinear covariate effects and time trends, unit- or cluster specific heterogeneity, spatial heterogeneity and complex interactions between covariates of different types. Much less effort, however, has been devoted to model and variable selection. The paper develops a methodology for the simultaneous selection of variables and the degree of smoothness in regression models with a structured additive predictor. These models are quite general, containing additive (mixed) models, geoadditive models and varying coefficient models as special cases. This approach allows one to decide whether a particular covariate enters the model linearly or nonlinearly or is removed from the model. Moreover, it is possible to decide whether a spatial or cluster specific effect should be incorporated into the model to cope with spatial or cluster specific heterogeneity. Particular emphasis is also placed on selecting complex interactions between covariates and effects of different types. A new penalty for two-dimensional smoothing is proposed, that allows for ANOVA-type decompositions into main effects and an interaction effect without explicitly specifying the main effects. The penalty is an additive combination of other penalties. Fast algorithms and software are developed that allow one to even handle situations with many covariate effects and observations. The algorithms are related to backfitting and Markov chain Monte Carlo techniques, which divide the problem in a divide and conquer strategy into smaller pieces. Confidence intervals taking model uncertainty into account are based on the bootstrap in combination with MCMC techniques.

Suggested Citation

  • Belitz, Christiane & Lang, Stefan, 2008. "Simultaneous selection of variables and smoothing parameters in structured additive regression models," Computational Statistics & Data Analysis, Elsevier, vol. 53(1), pages 61-81, September.
  • Handle: RePEc:eee:csdana:v:53:y:2008:i:1:p:61-81
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-9473(08)00300-9
    Download Restriction: Full text for ScienceDirect subscribers only.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ruppert,David & Wand,M. P. & Carroll,R. J., 2003. "Semiparametric Regression," Cambridge Books, Cambridge University Press, number 9780521785167.
    2. Ludwig Fahrmeir & Stefan Lang, 2001. "Bayesian inference for generalized additive mixed models based on Markov random field priors," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 50(2), pages 201-220.
    3. Brezger, Andreas & Kneib, Thomas & Lang, Stefan, 2005. "BayesX: Analyzing Bayesian Structural Additive Regression Models," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 14(i11).
    4. Marx, Brian D. & Eilers, Paul H. C., 1998. "Direct generalized additive modeling with penalized likelihood," Computational Statistics & Data Analysis, Elsevier, vol. 28(2), pages 193-209, August.
    5. Jullion, Astrid & Lambert, Philippe, 2007. "Robust specification of the roughness penalty prior distribution in spatially adaptive Bayesian P-splines models," Computational Statistics & Data Analysis, Elsevier, vol. 51(5), pages 2542-2558, February.
    6. S. N. Wood, 2000. "Modelling and smoothing parameter estimation with multiple quadratic penalties," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 62(2), pages 413-428.
    7. Gamerman, Dani & Moreira, Ajax R. B. & Rue, Havard, 2003. "Space-varying regression models: specifications and simulation," Computational Statistics & Data Analysis, Elsevier, vol. 42(3), pages 513-533, March.
    8. Tutz, Gerhard & Binder, Harald, 2007. "Boosting ridge regression," Computational Statistics & Data Analysis, Elsevier, vol. 51(12), pages 6044-6059, August.
    9. Brezger, Andreas & Lang, Stefan, 2006. "Generalized structured additive regression based on Bayesian P-splines," Computational Statistics & Data Analysis, Elsevier, vol. 50(4), pages 967-991, February.
    10. Casella, George & Moreno, Elias, 2006. "Objective Bayesian Variable Selection," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 157-167, March.
    11. R. A. Rigby & D. M. Stasinopoulos, 2005. "Generalized additive models for location, scale and shape," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 54(3), pages 507-554, June.
    12. Ruppert,David & Wand,M. P. & Carroll,R. J., 2003. "Semiparametric Regression," Cambridge Books, Cambridge University Press, number 9780521780506.
    13. E. E. Kammann & M. P. Wand, 2003. "Geoadditive models," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 52(1), pages 1-18, January.
    14. X. Lin & D. Zhang, 1999. "Inference in generalized additive mixed modelsby using smoothing splines," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 61(2), pages 381-400, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Chakraborty, Sounak, 2009. "Simultaneous cancer classification and gene selection with Bayesian nearest neighbor method: An integrated approach," Computational Statistics & Data Analysis, Elsevier, vol. 53(4), pages 1462-1474, February.
    2. de Uña Álvarez, Jacobo & Roca Pardiñas, Javier, 2009. "Additive models in censored regression," Computational Statistics & Data Analysis, Elsevier, vol. 53(9), pages 3490-3501, July.
    3. Nadja Klein & Thomas Kneib & Stefan Lang, 2015. "Bayesian Generalized Additive Models for Location, Scale, and Shape for Zero-Inflated and Overdispersed Count Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(509), pages 405-419, March.
    4. Stefan Lang & Nikolaus Umlauf & Peter Wechselberger & Kenneth Harttgen & Thomas Kneib, 2012. "Multilevel structured additive regression," Working Papers 2012-07, Faculty of Economics and Statistics, University of Innsbruck.
    5. Umlauf, Nikolaus & Adler, Daniel & Kneib, Thomas & Lang, Stefan & Zeileis, Achim, 2015. "Structured Additive Regression Models: An R Interface to BayesX," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 63(i21).
    6. Chakraborty, Sounak, 2009. "Bayesian binary kernel probit model for microarray based cancer classification and gene selection," Computational Statistics & Data Analysis, Elsevier, vol. 53(12), pages 4198-4209, October.
    7. Fabian Scheipl & Thomas Kneib & Ludwig Fahrmeir, 2013. "Penalized likelihood and Bayesian function selection in regression models," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 97(4), pages 349-385, October.
    8. Marra, Giampiero & Wood, Simon N., 2011. "Practical variable selection for generalized additive models," Computational Statistics & Data Analysis, Elsevier, vol. 55(7), pages 2372-2387, July.
    9. Strasak, Alexander M. & Umlauf, Nikolaus & Pfeiffer, Ruth M. & Lang, Stefan, 2011. "Comparing penalized splines and fractional polynomials for flexible modelling of the effects of continuous predictor variables," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1540-1551, April.
    10. Lee, Dae-Jin & Durbán, María & Eilers, Paul, 2013. "Efficient two-dimensional smoothing with P-spline ANOVA mixed models and nested bases," Computational Statistics & Data Analysis, Elsevier, vol. 61(C), pages 22-37.
    11. McKay Curtis, S. & Banerjee, Sayantan & Ghosal, Subhashis, 2014. "Fast Bayesian model assessment for nonparametric additive regression," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 347-358.
    12. Nikolaus Umlauf & Nadja Klein & Achim Zeileis, 2017. "BAMLSS: Bayesian Additive Models for Location, Scale and Shape (and Beyond)," Working Papers 2017-05, Faculty of Economics and Statistics, University of Innsbruck.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:53:y:2008:i:1:p:61-81. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Haili He). General contact details of provider: http://www.elsevier.com/locate/csda .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.