IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v54y2010i6p1535-1546.html
   My bibliography  Save this article

Ensemble classification based on generalized additive models

Author

Listed:
  • De Bock, Koen W.
  • Coussement, Kristof
  • Van den Poel, Dirk

Abstract

Generalized additive models (GAMs) are a generalization of generalized linear models (GLMs) and constitute a powerful technique which has successfully proven its ability to capture nonlinear relationships between explanatory variables and a response variable in many domains. In this paper, GAMs are proposed as base classifiers for ensemble learning. Three alternative ensemble strategies for binary classification using GAMs as base classifiers are proposed: (i) GAMbag based on Bagging, (ii) GAMrsm based on the Random Subspace Method (RSM), and (iii) GAMens as a combination of both. In an experimental validation performed on 12 data sets from the UCI repository, the proposed algorithms are benchmarked to a single GAM and to decision tree based ensemble classifiers (i.e. RSM, Bagging, Random Forest, and the recently proposed Rotation Forest). From the results a number of conclusions can be drawn. Firstly, the use of an ensemble of GAMs instead of a single GAM always leads to improved prediction performance. Secondly, GAMrsm and GAMens perform comparably, while both versions outperform GAMbag. Finally, the value of using GAMs as base classifiers in an ensemble instead of standard decision trees is demonstrated. GAMbag demonstrates performance comparable to ordinary Bagging. Moreover, GAMrsm and GAMens outperform RSM and Bagging, while these two GAM ensemble variations perform comparably to Random Forest and Rotation Forest. Sensitivity analyses are included for the number of member classifiers in the ensemble, the number of variables included in a random feature subspace and the number of degrees of freedom for GAM spline estimation.

Suggested Citation

  • De Bock, Koen W. & Coussement, Kristof & Van den Poel, Dirk, 2010. "Ensemble classification based on generalized additive models," Computational Statistics & Data Analysis, Elsevier, vol. 54(6), pages 1535-1546, June.
  • Handle: RePEc:eee:csdana:v:54:y:2010:i:6:p:1535-1546
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-9473(09)00465-4
    Download Restriction: Full text for ScienceDirect subscribers only.

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Zwane, E. N. & van der Heijden, P. G. M., 2004. "Semiparametric models for capture-recapture studies with covariates," Computational Statistics & Data Analysis, Elsevier, vol. 47(4), pages 729-743, November.
    2. Croux, Christophe & Joossens, Kristel & Lemmens, Aurelie, 2007. "Trimmed bagging," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 362-368, September.
    3. Hothorn, Torsten & Lausen, Berthold, 2005. "Bundling classifiers by bagging trees," Computational Statistics & Data Analysis, Elsevier, vol. 49(4), pages 1068-1078, June.
    4. Archer, Kellie J. & Kimes, Ryan V., 2008. "Empirical characterization of random forest variable importance measures," Computational Statistics & Data Analysis, Elsevier, vol. 52(4), pages 2249-2260, January.
    5. Marx, Brian D. & Eilers, Paul H. C., 1998. "Direct generalized additive modeling with penalized likelihood," Computational Statistics & Data Analysis, Elsevier, vol. 28(2), pages 193-209, August.
    6. Borra, Simone & Di Ciaccio, Agostino, 2002. "Improving nonparametric regression methods by bagging and boosting," Computational Statistics & Data Analysis, Elsevier, vol. 38(4), pages 407-420, February.
    7. Baccini, Michela & Biggeri, Annibale & Lagazio, Corrado & Lertxundi, Aitana & Saez, Marc, 2007. "Parametric and semi-parametric approaches in the analysis of short-term effects of air pollution on health," Computational Statistics & Data Analysis, Elsevier, vol. 51(9), pages 4324-4336, May.
    8. A. Prinzie & D. Van Den Poel, 2007. "Random Forrests for Multiclass classification: Random Multinomial Logit," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 07/435, Ghent University, Faculty of Economics and Business Administration.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. K. W. De Bock & D. Van Den Poel, 2012. "Reconciling Performance and Interpretability in Customer Churn Prediction using Ensemble Learning based on Generalized Additive Models," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 12/805, Ghent University, Faculty of Economics and Business Administration.
    2. Christmann, Andreas & Hable, Robert, 2012. "Consistency of support vector machines using additive kernels for additive models," Computational Statistics & Data Analysis, Elsevier, vol. 56(4), pages 854-873.
    3. Narayanaswamy Balakrishnan & Majid Mojirsheibani, 2015. "A simple method for combining estimates to improve the overall error rates in classification," Computational Statistics, Springer, vol. 30(4), pages 1033-1049, December.
    4. K. W. De Bock & D. Van Den Poel, 2011. "An empirical evaluation of rotation-based ensemble classifiers for customer churn prediction," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 11/717, Ghent University, Faculty of Economics and Business Administration.
    5. Hugo Proença & João C. Neves, 2017. "Fusing Vantage Point Trees and Linear Discriminants for Fast Feature Classification," Journal of Classification, Springer;The Classification Society, vol. 34(1), pages 85-107, April.
    6. Coussement, Kristof & De Bock, Koen W., 2013. "Customer churn prediction in the online gambling industry: The beneficial effect of ensemble learning," Journal of Business Research, Elsevier, vol. 66(9), pages 1629-1636.
    7. Jasmit Shah & Somnath Datta & Susmita Datta, 2014. "A multi-loss super regression learner (MSRL) with application to survival prediction using proteomics," Computational Statistics, Springer, vol. 29(6), pages 1749-1767, December.
    8. Mojirsheibani, Majid & Kong, Jiajie, 2016. "An asymptotically optimal kernel combined classifier," Statistics & Probability Letters, Elsevier, vol. 119(C), pages 91-100.
    9. Fitzpatrick, Trevor & Mues, Christophe, 2016. "An empirical comparison of classification algorithms for mortgage default prediction: evidence from a distressed mortgage market," European Journal of Operational Research, Elsevier, vol. 249(2), pages 427-439.
    10. Adler, Werner & Brenning, Alexander & Potapov, Sergej & Schmid, Matthias & Lausen, Berthold, 2011. "Ensemble classification of paired data," Computational Statistics & Data Analysis, Elsevier, vol. 55(5), pages 1933-1941, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:54:y:2010:i:6:p:1535-1546. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Dana Niculescu). General contact details of provider: http://www.elsevier.com/locate/csda .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.