IDEAS home Printed from https://ideas.repec.org/a/jss/jstsof/v054i02.html
   My bibliography  Save this article

adabag: An R Package for Classification with Boosting and Bagging

Author

Listed:
  • Alfaro, Esteban
  • Gamez, Matias
  • García, Noelia

Abstract

Boosting and bagging are two widely used ensemble methods for classification. Their common goal is to improve the accuracy of a classifier combining single classifiers which are slightly better than random guessing. Among the family of boosting algorithms, AdaBoost (adaptive boosting) is the best known, although it is suitable only for dichotomous tasks. AdaBoost.M1 and SAMME (stagewise additive modeling using a multi-class exponential loss function) are two easy and natural extensions to the general case of two or more classes. In this paper, the adabag R package is introduced. This version implements AdaBoost.M1, SAMME and bagging algorithms with classification trees as base classifiers. Once the ensembles have been trained, they can be used to predict the class of new samples. The accuracy of these classifiers can be estimated in a separated data set or through cross validation. Moreover, the evolution of the error as the ensemble grows can be analysed and the ensemble can be pruned. In addition, the margin in the class prediction and the probability of each class for the observations can be calculated. Finally, several classic examples in classification literature are shown to illustrate the use of this package.

Suggested Citation

  • Alfaro, Esteban & Gamez, Matias & García, Noelia, 2013. "adabag: An R Package for Classification with Boosting and Bagging," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 54(i02).
    • Handle: RePEc:jss:jstsof:v:054:i02
    DOI: http://hdl.handle.net/10.18637/jss.v054.i02
    as

    Download full text from publisher

    File URL: https://www.jstatsoft.org/index.php/jss/article/view/v054i02/adabag_An_R_Package_for_Classification_with_Boosting_and_Bagging.pdf
    Download Restriction: no
    File URL: https://www.jstatsoft.org/index.php/jss/article/downloadSuppFile/v054i02/adabag_3.2.tar.gz
    Download Restriction: no
    File URL: https://www.jstatsoft.org/index.php/jss/article/downloadSuppFile/v054i02/v54i02.R
    Download Restriction: no
    File URL: https://libkey.io/http://hdl.handle.net/10.18637/jss.v054.i02?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Kuhn, Max, 2008. "Building Predictive Models in R Using the caret Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 28(i05).
    2. Culp, Mark & Johnson, Kjell & Michailides, George, 2006. "ada: An R Package for Stochastic Boosting," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 17(i02).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Agustín J. Sánchez-Medina & Félix Blázquez-Santana & Jesús B. Alonso, 2019. "Do Auditors Reflect the True Image of the Company Contrary to the Clients’ Interests? An Artificial Intelligence Approach," Journal of Business Ethics, Springer, vol. 155(2), pages 529-545, March.
    2. Antonella Plaia & Simona Buscemi & Johannes Fürnkranz & Eneldo Loza Mencía, 2022. "Comparing Boosting and Bagging for Decision Trees of Rankings," Journal of Classification, Springer;The Classification Society, vol. 39(1), pages 78-99, March.
    3. Romero Martínez, Mariano & Carmona Ibáñez, Pedro & Pozuelo Campillo, José, 2021. "Utilidad del Deep Learning en la predicción del fracaso empresarial en el ámbito europeo || The usefulness of Deep Learning in the prediction of business failure at the European level," Revista de Métodos Cuantitativos para la Economía y la Empresa = Journal of Quantitative Methods for Economics and Business Administration, Universidad Pablo de Olavide, Department of Quantitative Methods for Economics and Business Administration, vol. 32(1), pages 392-414, December.
    4. Gong, Joonho & Kim, Hyunjoong, 2017. "RHSBoost: Improving classification performance in imbalance data," Computational Statistics & Data Analysis, Elsevier, vol. 111(C), pages 1-13.
    5. Frédéric Kosmowski & Tigist Worku, 2018. "Evaluation of a miniaturized NIR spectrometer for cultivar identification: The case of barley, chickpea and sorghum in Ethiopia," PLOS ONE, Public Library of Science, vol. 13(3), pages 1-17, March.
    6. Erwin Cornelius & Olcay Akman & Dan Hrozencik, 2021. "COVID-19 Mortality Prediction Using Machine Learning-Integrated Random Forest Algorithm under Varying Patient Frailty," Mathematics, MDPI, vol. 9(17), pages 1-22, August.
    7. Junseok Lee & Ji-Ho Kang & Sunghae Jun & Hyunwoong Lim & Dongsik Jang & Sangsung Park, 2018. "Ensemble Modeling for Sustainable Technology Transfer," Sustainability, MDPI, vol. 10(7), pages 1-15, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Prabal Das & D. A. Sachindra & Kironmala Chanda, 2022. "Machine Learning-Based Rainfall Forecasting with Multiple Non-Linear Feature Selection Algorithms," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 36(15), pages 6043-6071, December.
    2. Jie Zhao & Ji Chen & Damien Beillouin & Hans Lambers & Yadong Yang & Pete Smith & Zhaohai Zeng & Jørgen E. Olesen & Huadong Zang, 2022. "Global systematic review with meta-analysis reveals yield advantage of legume-based rotations and its drivers," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    3. Piaopiao Chen & Agnès H. Michel & Jianzhi Zhang, 2022. "Transposon insertional mutagenesis of diverse yeast strains suggests coordinated gene essentiality polymorphisms," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    4. Paulo Infante & Gonçalo Jacinto & Anabela Afonso & Leonor Rego & Pedro Nogueira & Marcelo Silva & Vitor Nogueira & José Saias & Paulo Quaresma & Daniel Santos & Patrícia Góis & Paulo Rebelo Manuel, 2023. "Factors That Influence the Type of Road Traffic Accidents: A Case Study in a District of Portugal," Sustainability, MDPI, vol. 15(3), pages 1-16, January.
    5. Ephrem Habyarimana & Faheem S Baloch, 2021. "Machine learning models based on remote and proximal sensing as potential methods for in-season biomass yields prediction in commercial sorghum fields," PLOS ONE, Public Library of Science, vol. 16(3), pages 1-23, March.
    6. Banks, Jonathan & Rabbani, Arif & Nadkarni, Kabir & Renaud, Evan, 2020. "Estimating parasitic loads related to brine production from a hot sedimentary aquifer geothermal project: A case study from the Clarke Lake gas field, British Columbia," Renewable Energy, Elsevier, vol. 153(C), pages 539-552.
    7. Alexander Wettstein & Gabriel Jenni & Ida Schneider & Fabienne Kühne & Martin grosse Holtforth & Roberto La Marca, 2023. "Predictors of Psychological Strain and Allostatic Load in Teachers: Examining the Long-Term Effects of Biopsychosocial Risk and Protective Factors Using a LASSO Regression Approach," IJERPH, MDPI, vol. 20(10), pages 1-20, May.
    8. Tang, Kayu & Parsons, David J. & Jude, Simon, 2019. "Comparison of automatic and guided learning for Bayesian networks to analyse pipe failures in the water distribution system," Reliability Engineering and System Safety, Elsevier, vol. 186(C), pages 24-36.
    9. Daifeng Xiang & Gangsheng Wang & Jing Tian & Wanyu Li, 2023. "Global patterns and edaphic-climatic controls of soil carbon decomposition kinetics predicted from incubation experiments," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    10. Bellotti, Anthony & Brigo, Damiano & Gambetti, Paolo & Vrins, Frédéric, 2021. "Forecasting recovery rates on non-performing loans with machine learning," International Journal of Forecasting, Elsevier, vol. 37(1), pages 428-444.
    11. Tranos, Emmanouil & Incera, Andre Carrascal & Willis, George, 2022. "Using the web to predict regional trade flows: data extraction, modelling, and validation," OSF Preprints 9bu5z, Center for Open Science.
    12. Štefan Lyócsa & Petra Vašaničová & Branka Hadji Misheva & Marko Dávid Vateha, 2022. "Default or profit scoring credit systems? Evidence from European and US peer-to-peer lending markets," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 8(1), pages 1-21, December.
    13. Arjan S. Gosal & Janine A. McMahon & Katharine M. Bowgen & Catherine H. Hoppe & Guy Ziv, 2021. "Identifying and Mapping Groups of Protected Area Visitors by Environmental Awareness," Land, MDPI, vol. 10(6), pages 1-14, May.
    14. Marcos Rodrigues & Fermín Alcasena & Pere Gelabert & Cristina Vega‐García, 2020. "Geospatial Modeling of Containment Probability for Escaped Wildfires in a Mediterranean Region," Risk Analysis, John Wiley & Sons, vol. 40(9), pages 1762-1779, September.
    15. Giovanny Pillajo-Quijia & Blanca Arenas-Ramírez & Camino González-Fernández & Francisco Aparicio-Izquierdo, 2020. "Influential Factors on Injury Severity for Drivers of Light Trucks and Vans with Machine Learning Methods," Sustainability, MDPI, vol. 12(4), pages 1-28, February.
    16. Francesco Sartor & Jonathan P. Moore & Hans-Peter Kubis, 2021. "Plasma Interleukin-10 and Cholesterol Levels May Inform about Interdependences between Fitness and Fatness in Healthy Individuals," IJERPH, MDPI, vol. 18(4), pages 1-19, February.
    17. Zander S. Venter & Adam Sadilek & Charlotte Stanton & David N. Barton & Kristin Aunan & Sourangsu Chowdhury & Aaron Schneider & Stefano Maria Iacus, 2021. "Mobility in Blue-Green Spaces Does Not Predict COVID-19 Transmission: A Global Analysis," IJERPH, MDPI, vol. 18(23), pages 1-12, November.
    18. G. Brooke Anderson & Keith W. Oleson & Bryan Jones & Roger D. Peng, 2018. "Classifying heatwaves: developing health-based models to predict high-mortality versus moderate United States heatwaves," Climatic Change, Springer, vol. 146(3), pages 439-453, February.
    19. Van Belle, Jente & Guns, Tias & Verbeke, Wouter, 2021. "Using shared sell-through data to forecast wholesaler demand in multi-echelon supply chains," European Journal of Operational Research, Elsevier, vol. 288(2), pages 466-479.
    20. Michael J. Geuenich & Dae-won Gong & Kieran R. Campbell, 2024. "The impacts of active and self-supervised learning on efficient annotation of single-cell expression data," Nature Communications, Nature, vol. 15(1), pages 1-13, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:jss:jstsof:v:054:i02. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christopher F. Baum (email available below). General contact details of provider: http://www.jstatsoft.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.