IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v53y2008i2p298-311.html
   My bibliography  Save this article

Boosting additive models using component-wise P-Splines

Author

Listed:
  • Schmid, Matthias
  • Hothorn, Torsten

Abstract

An efficient approximation of L2 Boosting with component-wise smoothing splines is considered. Smoothing spline base-learners are replaced by P-spline base-learners, which yield similar prediction errors but are more advantageous from a computational point of view. A detailed analysis of the effect of various P-spline hyper-parameters on the boosting fit is given. In addition, a new theoretical result on the relationship between the boosting stopping iteration and the step length factor used for shrinking the boosting estimates is derived.

Suggested Citation

  • Schmid, Matthias & Hothorn, Torsten, 2008. "Boosting additive models using component-wise P-Splines," Computational Statistics & Data Analysis, Elsevier, vol. 53(2), pages 298-311, December.
    • Handle: RePEc:eee:csdana:v:53:y:2008:i:2:p:298-311
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-9473(08)00441-6
    Download Restriction: Full text for ScienceDirect subscribers only.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ruppert,David & Wand,M. P. & Carroll,R. J., 2003. "Semiparametric Regression," Cambridge Books, Cambridge University Press, number 9780521785167, October.
    2. Marx, Brian D. & Eilers, Paul H. C., 1998. "Direct generalized additive modeling with penalized likelihood," Computational Statistics & Data Analysis, Elsevier, vol. 28(2), pages 193-209, August.
    3. Gerhard Tutz & Harald Binder, 2006. "Generalized Additive Modeling with Implicit Variable Selection by Likelihood-Based Boosting," Biometrics, The International Biometric Society, vol. 62(4), pages 961-971, December.
    4. Buhlmann P. & Yu B., 2003. "Boosting With the L2 Loss: Regression and Classification," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 324-339, January.
    5. Leitenstorfer, Florian & Tutz, Gerhard, 2007. "Knot selection by boosting techniques," Computational Statistics & Data Analysis, Elsevier, vol. 51(9), pages 4605-4621, May.
    6. Ruppert,David & Wand,M. P. & Carroll,R. J., 2003. "Semiparametric Regression," Cambridge Books, Cambridge University Press, number 9780521780506, October.
    7. Clifford M. Hurvich & Jeffrey S. Simonoff & Chih‐Ling Tsai, 1998. "Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 60(2), pages 271-293.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wang Zhu, 2011. "HingeBoost: ROC-Based Boost for Classification and Variable Selection," The International Journal of Biostatistics, De Gruyter, vol. 7(1), pages 1-30, February.
    2. Guilherme Schultz Lindenmeyer & Hudson Silva Torrent, 2024. "Boosting and Predictability of Macroeconomic Variables: Evidence from Brazil," Computational Economics, Springer;Society for Computational Economics, vol. 64(1), pages 377-409, July.
    3. Christmann, Andreas & Hable, Robert, 2012. "Consistency of support vector machines using additive kernels for additive models," Computational Statistics & Data Analysis, Elsevier, vol. 56(4), pages 854-873.
    4. Carlos A. Manzanares & Ying Jiang & Patrick Bajari, 2015. "Improving Policy Functions in High-Dimensional Dynamic Games," NBER Working Papers 21124, National Bureau of Economic Research, Inc.
    5. Souhaib Ben Taieb & Rob J Hyndman, 2014. "Boosting multi-step autoregressive forecasts," Monash Econometrics and Business Statistics Working Papers 13/14, Monash University, Department of Econometrics and Business Statistics.
    6. Hansen, Bjørn Gunnar, 2019. "CO2 – Emission, costs and capacity of different manure management practices- results from an advisory project," Agricultural Systems, Elsevier, vol. 173(C), pages 325-334.
    7. Sobotka, Fabian & Kneib, Thomas, 2012. "Geoadditive expectile regression," Computational Statistics & Data Analysis, Elsevier, vol. 56(4), pages 755-767.
    8. Hendrik van der Wurp & Andreas Groll, 2023. "Introducing LASSO-type penalisation to generalised joint regression modelling for count data," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 107(1), pages 127-151, March.
    9. Xin Fang & Bo Fang & Chunfang Wang & Tian Xia & Matteo Bottai & Fang Fang & Yang Cao, 2019. "Comparison of Frequentist and Bayesian Generalized Additive Models for Assessing the Association between Daily Exposure to Fine Particles and Respiratory Mortality: A Simulation Study," IJERPH, MDPI, vol. 16(5), pages 1-20, March.
    10. Kauppi, Heikki & Virtanen, Timo, 2021. "Boosting nonlinear predictability of macroeconomic time series," International Journal of Forecasting, Elsevier, vol. 37(1), pages 151-170.
    11. Nikolay Robinzonov & Gerhard Tutz & Torsten Hothorn, 2012. "Boosting techniques for nonlinear time series models," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 96(1), pages 99-122, January.
    12. Guilherme Lindenmeyer & Pedro Pablo Skorin & Hudson da Silva Torrent, 2021. "Using boosting for forecasting electric energy consumption during a recession: a case study for the Brazilian State Rio Grande do Sul," Letters in Spatial and Resource Sciences, Springer, vol. 14(2), pages 111-128, August.
    13. Groll Andreas & Kneib Thomas & Mayr Andreas & Schauberger Gunther, 2018. "On the dependency of soccer scores – a sparse bivariate Poisson model for the UEFA European football championship 2016," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 14(2), pages 65-79, June.
    14. Simon N. Wood, 2020. "Inference and computation with generalized additive models and their extensions," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 29(2), pages 307-339, June.
    15. Holger Reulen & Thomas Kneib, 2016. "Boosting multi-state models," Lifetime Data Analysis: An International Journal Devoted to Statistical Methods and Applications for Time-to-Event Data, Springer, vol. 22(2), pages 241-262, April.
    16. Benjamin Hofner & Torsten Hothorn & Thomas Kneib, 2013. "Variable selection and model choice in structured survival models," Computational Statistics, Springer, vol. 28(3), pages 1079-1101, June.
    17. Möst Lisa & Hothorn Torsten, 2015. "Conditional Transformation Models for Survivor Function Estimation," The International Journal of Biostatistics, De Gruyter, vol. 11(1), pages 23-50, May.
    18. Souhaib Ben Taieb & Raphael Huser & Rob J. Hyndman & Marc G. Genton, 2015. "Probabilistic time series forecasting with boosted additive models: an application to smart meter data," Monash Econometrics and Business Statistics Working Papers 12/15, Monash University, Department of Econometrics and Business Statistics.
    19. Heikki Kauppi & Timo Virtanen, 2018. "Boosting Non-linear Predictabilityof Macroeconomic Time Series," Discussion Papers 124, Aboa Centre for Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Leitenstorfer, Florian & Tutz, Gerhard, 2007. "Knot selection by boosting techniques," Computational Statistics & Data Analysis, Elsevier, vol. 51(9), pages 4605-4621, May.
    2. Belitz, Christiane & Lang, Stefan, 2008. "Simultaneous selection of variables and smoothing parameters in structured additive regression models," Computational Statistics & Data Analysis, Elsevier, vol. 53(1), pages 61-81, September.
    3. Baccini, Michela & Biggeri, Annibale & Lagazio, Corrado & Lertxundi, Aitana & Saez, Marc, 2007. "Parametric and semi-parametric approaches in the analysis of short-term effects of air pollution on health," Computational Statistics & Data Analysis, Elsevier, vol. 51(9), pages 4324-4336, May.
    4. Simon N. Wood, 2011. "Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 73(1), pages 3-36, January.
    5. Bethany Everett & David Rehkopf & Richard Rogers, 2013. "The Nonlinear Relationship Between Education and Mortality: An Examination of Cohort, Race/Ethnic, and Gender Differences," Population Research and Policy Review, Springer;Southern Demographic Association (SDA), vol. 32(6), pages 893-917, December.
    6. Simon N. Wood & Natalya Pya & Benjamin Säfken, 2016. "Smoothing Parameter and Model Selection for General Smooth Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1548-1563, October.
    7. Strasak, Alexander M. & Umlauf, Nikolaus & Pfeiffer, Ruth M. & Lang, Stefan, 2011. "Comparing penalized splines and fractional polynomials for flexible modelling of the effects of continuous predictor variables," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1540-1551, April.
    8. Krisztin, Tamás, 2018. "Semi-parametric spatial autoregressive models in freight generation modeling," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 114(C), pages 121-143.
    9. Nagler Thomas & Schellhase Christian & Czado Claudia, 2017. "Nonparametric estimation of simplified vine copula models: comparison of methods," Dependence Modeling, De Gruyter, vol. 5(1), pages 99-120, January.
    10. Morteza Amini & Mahdi Roozbeh & Nur Anisah Mohamed, 2024. "Separation of the Linear and Nonlinear Covariates in the Sparse Semi-Parametric Regression Model in the Presence of Outliers," Mathematics, MDPI, vol. 12(2), pages 1-17, January.
    11. Christophe Croux & Irène Gijbels & Ilaria Prosdocimi, 2012. "Robust Estimation of Mean and Dispersion Functions in Extended Generalized Additive Models," Biometrics, The International Biometric Society, vol. 68(1), pages 31-44, March.
    12. I. Gijbels & I. Prosdocimi & G. Claeskens, 2010. "Nonparametric estimation of mean and dispersion functions in extended generalized linear models," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 19(3), pages 580-608, November.
    13. I. Gijbels & I. Prosdocimi, 2011. "Smooth estimation of mean and dispersion function in extended generalized additive models with application to Italian induced abortion data," Journal of Applied Statistics, Taylor & Francis Journals, vol. 38(11), pages 2391-2411, December.
    14. Takuma Yoshida & Kanta Naito, 2014. "Asymptotics for penalised splines in generalised additive models," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 26(2), pages 269-289, June.
    15. David O'Donnell & Alastair Rushworth & Adrian W. Bowman & E. Marian Scott & Mark Hallard, 2014. "Flexible regression models over river networks," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 63(1), pages 47-63, January.
    16. Takuma Yoshida, 2019. "Two stage smoothing in additive models with missing covariates," Statistical Papers, Springer, vol. 60(6), pages 1803-1826, December.
    17. Hazelton, Martin L. & Turlach, Berwin A., 2011. "Semiparametric regression with shape-constrained penalized splines," Computational Statistics & Data Analysis, Elsevier, vol. 55(10), pages 2871-2879, October.
    18. Beck, Jonathan, 2008. "Diderot´s rule," Discussion Papers, Research Unit: Competition and Innovation SP II 2008-13, WZB Berlin Social Science Center.
    19. Roca-Pardinas, Javier & Cadarso-Suarez, Carmen & Tahoces, Pablo G. & Lado, Maria J., 2008. "Assessing continuous bivariate effects among different groups through nonparametric regression models: An application to breast cancer detection," Computational Statistics & Data Analysis, Elsevier, vol. 52(4), pages 1958-1970, January.
    20. Hübler, Olaf, 2017. "Health and Body Mass Index: No Simple Relationship," IZA Discussion Papers 10620, Institute of Labor Economics (IZA).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:53:y:2008:i:2:p:298-311. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.