IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v346y2025i3d10.1007_s10479-025-06516-9.html
   My bibliography  Save this article

Investigating the impact of undersampling and bagging: an empirical investigation for customer attrition modeling

Author

Listed:
  • Arno Caigny

    (Univ. Lille, CNRS, UMR 9221 - LEM - Lille Economie Management)

  • Kristof Coussement

    (Univ. Lille, CNRS, UMR 9221 - LEM - Lille Economie Management)

  • Matthijs Meire

    (Univ. Lille, CNRS, UMR 9221 - LEM - Lille Economie Management)

  • Steven Hoornaert

    (Univ. Lille, CNRS, UMR 9221 - LEM - Lille Economie Management)

Abstract

Given the growing interest in using AI and analytics to support CRM decision making, we discuss why undersampling and bagging are popular prediction techniques in customer churn prediction (CCP). The former helps in tackling the class imbalance problem and the latter improves model stability. However, extant CCP literature is unclear on the impact of undersampling on model stability and predictive performance, while bagging has difficulties in handling the class imbalance problem. Therefore, we extend existing CCP research to benchmark underbagging, which combines undersampling and bagging. Having both prediction techniques combined we recuperate customer data that would have been lost in undersampling by using them in multiple bags and passing an undersampled, more balanced training set to the classifier. In an extensive experiment including 11 real-life CCP datasets, underbagging is benchmarked against its constituents and other popular CCP classifiers in terms of predictive performance, profit and operational efficiency. Our results indicate that underbagging is a valid and reliable alternative framework for CCP prediction.

Suggested Citation

  • Arno Caigny & Kristof Coussement & Matthijs Meire & Steven Hoornaert, 2025. "Investigating the impact of undersampling and bagging: an empirical investigation for customer attrition modeling," Annals of Operations Research, Springer, vol. 346(3), pages 2401-2421, March.
    • Handle: RePEc:spr:annopr:v:346:y:2025:i:3:d:10.1007_s10479-025-06516-9
    DOI: 10.1007/s10479-025-06516-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10479-025-06516-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10479-025-06516-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Risselada, Hans & Verhoef, Peter C. & Bijmolt, Tammo H.A., 2010. "Staying Power of Churn Prediction Models," Journal of Interactive Marketing, Elsevier, vol. 24(3), pages 198-208.
    2. Shen-Ming Lee & Chin-Shang Li & Shu-Hui Hsieh & Li-Hui Huang, 2012. "Semiparametric estimation of logistic regression model with missing covariates and outcome," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 75(5), pages 621-653, July.
    3. Aurélie Lemmens & Sunil Gupta, 2020. "Managing Churn to Maximize Profits," Marketing Science, INFORMS, vol. 39(5), pages 956-973, September.
    4. Libai, Barak & Bart, Yakov & Gensler, Sonja & Hofacker, Charles F. & Kaplan, Andreas & Kötterheinrich, Kim & Kroll, Eike Benjamin, 2020. "Brave New World? On AI and the Management of Customer Relationships," Journal of Interactive Marketing, Elsevier, vol. 51(C), pages 44-56.
    5. Coussement, Kristof & De Bock, Koen W., 2013. "Customer churn prediction in the online gambling industry: The beneficial effect of ensemble learning," Journal of Business Research, Elsevier, vol. 66(9), pages 1629-1636.
    6. Jiayin Qi & Li Zhang & Yanping Liu & Ling Li & Yongpin Zhou & Yao Shen & Liang Liang & Huaizu Li, 2009. "ADTreesLogit model for customer churn prediction," Annals of Operations Research, Springer, vol. 168(1), pages 247-265, April.
    7. Li, Yixin & Hou, Bingzhang & Wu, Yue & Zhao, Donglai & Xie, Aoran & Zou, Peng, 2021. "Giant fight: Customer churn prediction in traditional broadcast industry," Journal of Business Research, Elsevier, vol. 131(C), pages 630-639.
    8. Trapero, Juan R. & Pedregal, Diego J. & Fildes, R. & Kourentzes, N., 2013. "Analysis of judgmental adjustments in the presence of promotions," International Journal of Forecasting, Elsevier, vol. 29(2), pages 234-243.
    9. Buckinx, Wouter & Van den Poel, Dirk, 2005. "Customer base analysis: partial defection of behaviourally loyal clients in a non-contractual FMCG retail setting," European Journal of Operational Research, Elsevier, vol. 164(1), pages 252-268, July.
    10. De Bruyn, Arnaud & Viswanathan, Vijay & Beh, Yean Shan & Brock, Jürgen Kai-Uwe & von Wangenheim, Florian, 2020. "Artificial Intelligence and Marketing: Pitfalls and Opportunities," Journal of Interactive Marketing, Elsevier, vol. 51(C), pages 91-105.
    11. De Caigny, Arno & Coussement, Kristof & De Bock, Koen W. & Lessmann, Stefan, 2020. "Incorporating textual information in customer churn prediction models based on a convolutional neural network," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1563-1578.
    12. K. W. De Bock & D. Van Den Poel, 2011. "An empirical evaluation of rotation-based ensemble classifiers for customer churn prediction," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 11/717, Ghent University, Faculty of Economics and Business Administration.
    13. Arno de Caigny & Kristof Coussement & Koen W. de Bock, 2018. "A new hybrid classification algorithm for customer churn prediction based on logistic regression and decision trees," Post-Print hal-01741661, HAL.
    14. Lemmens, A. & Croux, C., 2006. "Bagging and boosting classification trees to predict churn," Other publications TiSEM d5cb664d-5859-44db-a621-e, Tilburg University, School of Economics and Management.
    15. Maldonado, Sebastián & López, Julio & Vairetti, Carla, 2020. "Profit-based churn prediction based on Minimax Probability Machines," European Journal of Operational Research, Elsevier, vol. 284(1), pages 273-284.
    16. Mitrović, Sandra & Baesens, Bart & Lemahieu, Wilfried & De Weerdt, Jochen, 2018. "On the operational efficiency of different feature types for telco Churn prediction," European Journal of Operational Research, Elsevier, vol. 267(3), pages 1141-1155.
    17. K. Coussement & D. Van Den Poel, 2008. "Improving Customer Attrition Prediction by Integrating Emotions from Client/Company Interaction Emails and Evaluating Multiple Classifiers," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 08/527, Ghent University, Faculty of Economics and Business Administration.
    18. G. Mena & K. Coussement & K. de Bock & A. de Caigny & S. Lessmann, 2024. "Exploiting Time-Varying RFM Measures for Customer Churn Prediction with Deep Neural Networks," Post-Print hal-04680677, HAL.
    19. K. Coussement & D. Van Den Poel, 2008. "Integrating the Voice of Customers through Call Center Emails into a Decision Support System for Churn Prediction," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 08/502, Ghent University, Faculty of Economics and Business Administration.
    20. Gary Mena & Kristof Coussement & Koen W. Bock & Arno Caigny & Stefan Lessmann, 2024. "Exploiting time-varying RFM measures for customer churn prediction with deep neural networks," Annals of Operations Research, Springer, vol. 339(1), pages 765-787, August.
    21. Mahesh Kumar & Nitin Patel, 2010. "Using clustering to improve sales forecasts in retail merchandising," Annals of Operations Research, Springer, vol. 174(1), pages 33-46, February.
    22. K. Coussement & D. Van Den Poel, 2006. "Churn Prediction in Subscription Services: an Application of Support Vector Machines While Comparing Two Parameter-Selection Techniques," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 06/412, Ghent University, Faculty of Economics and Business Administration.
    23. Kristof Coussement & Stefan Lessmann & Geert Verstraeten, 2017. "A comparative analysis of data preparation algorithms for customer churn prediction: A case study in the telecommunication industry," Post-Print hal-01745261, HAL.
    24. Konstantin Kogan & Avi Herbon & Beatrice Venturi, 2020. "Direct marketing of an event under hazards of customer saturation and forgetting," Annals of Operations Research, Springer, vol. 295(1), pages 207-227, December.
    25. Sarkar, Mainak & De Bruyn, Arnaud, 2021. "LSTM Response Models for Direct Marketing Analytics: Replacing Feature Engineering with Deep Learning," Journal of Interactive Marketing, Elsevier, vol. 53(C), pages 80-95.
    26. Villarroel Ordenes, Francisco & Silipo, Rosaria, 2021. "Machine learning for marketing on the KNIME Hub: The development of a live repository for marketing applications," Journal of Business Research, Elsevier, vol. 137(C), pages 393-410.
    27. E Lima & C Mues & B Baesens, 2009. "Domain knowledge integration in data mining using decision tables: case studies in churn prediction," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 60(8), pages 1096-1106, August.
    28. De Caigny, Arno & Coussement, Kristof & De Bock, Koen W., 2018. "A new hybrid classification algorithm for customer churn prediction based on logistic regression and decision trees," European Journal of Operational Research, Elsevier, vol. 269(2), pages 760-772.
    29. Riebe, Erica & Wright, Malcolm & Stern, Philip & Sharp, Byron, 2014. "How to grow a brand: Retain or acquire customers?," Journal of Business Research, Elsevier, vol. 67(5), pages 990-997.
    30. Verbeke, Wouter & Dejaeger, Karel & Martens, David & Hur, Joon & Baesens, Bart, 2012. "New insights into churn prediction in the telecommunication sector: A profit driven data mining approach," European Journal of Operational Research, Elsevier, vol. 218(1), pages 211-229.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Matthias Bogaert & Lex Delaere, 2023. "Ensemble Methods in Customer Churn Prediction: A Comparative Analysis of the State-of-the-Art," Mathematics, MDPI, vol. 11(5), pages 1-28, February.
    2. De Caigny, Arno & Coussement, Kristof & De Bock, Koen W. & Lessmann, Stefan, 2020. "Incorporating textual information in customer churn prediction models based on a convolutional neural network," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1563-1578.
    3. Arno de Caigny & Kristof Coussement & Koen W. de Bock & Stefan Lessmann, 2019. "Incorporating textual information in customer churn prediction models based on a convolutional neural network," Post-Print hal-02275958, HAL.
    4. Gattermann-Itschert, Theresa & Thonemann, Ulrich W., 2021. "How training on multiple time slices improves performance in churn prediction," European Journal of Operational Research, Elsevier, vol. 295(2), pages 664-674.
    5. Chou, Ping & Chuang, Howard Hao-Chun & Chou, Yen-Chun & Liang, Ting-Peng, 2022. "Predictive analytics for customer repurchase: Interdisciplinary integration of buy till you die modeling and machine learning," European Journal of Operational Research, Elsevier, vol. 296(2), pages 635-651.
    6. Liu, Zhenkun & Jiang, Ping & De Bock, Koen W. & Wang, Jianzhou & Zhang, Lifang & Niu, Xinsong, 2024. "Extreme gradient boosting trees with efficient Bayesian optimization for profit-driven customer churn prediction," Technological Forecasting and Social Change, Elsevier, vol. 198(C).
    7. Chandrasekhar Valluri & Sudhakar Raju & Vivek H. Patil, 2022. "Customer determinants of used auto loan churn: comparing predictive performance using machine learning techniques," Journal of Marketing Analytics, Palgrave Macmillan, vol. 10(3), pages 279-296, September.
    8. Koen W. de Bock & Arno de Caigny, 2021. "Spline-rule ensemble classifiers with structured sparsity regularization for interpretable customer churn modeling," Post-Print hal-03391564, HAL.
    9. Arno de Caigny & Kristof Coussement & Koen de Bock, 2020. "Leveraging fine-grained transaction data for customer life event predictions," Post-Print hal-02507998, HAL.
    10. Liu, Zhenkun & Zhang, Ying & Abedin, Mohammad Zoynul & Wang, Jianzhou & Yang, Hufang & Gao, Yuyang & Chen, Yinghao, 2024. "Profit-driven fusion framework based on bagging and boosting classifiers for potential purchaser prediction," Journal of Retailing and Consumer Services, Elsevier, vol. 79(C).
    11. Chen, Yan & Zhang, Lei & Zhao, Yulu & Xu, Bing, 2022. "Implementation of penalized survival models in churn prediction of vehicle insurance," Journal of Business Research, Elsevier, vol. 153(C), pages 162-171.
    12. Bram Janssens & Matthias Bogaert & Astrid Bagué & Dirk Van den Poel, 2024. "B2Boost: instance-dependent profit-driven modelling of B2B churn," Annals of Operations Research, Springer, vol. 341(1), pages 267-293, October.
    13. Gary Mena & Kristof Coussement & Koen W. Bock & Arno Caigny & Stefan Lessmann, 2024. "Exploiting time-varying RFM measures for customer churn prediction with deep neural networks," Annals of Operations Research, Springer, vol. 339(1), pages 765-787, August.
    14. Ballings, Michel & Van den Poel, Dirk, 2015. "CRM in social media: Predicting increases in Facebook usage frequency," European Journal of Operational Research, Elsevier, vol. 244(1), pages 248-260.
    15. Schaeffer, Satu Elisa & Rodriguez Sanchez, Sara Veronica, 2020. "Forecasting client retention — A machine-learning approach," Journal of Retailing and Consumer Services, Elsevier, vol. 52(C).
    16. Johannes Habel & Sascha Alavi & Nicolas Heinitz, 2023. "A theory of predictive sales analytics adoption," AMS Review, Springer;Academy of Marketing Science, vol. 13(1), pages 34-54, June.
    17. K. W. De Bock & D. Van Den Poel, 2012. "Reconciling Performance and Interpretability in Customer Churn Prediction using Ensemble Learning based on Generalized Additive Models," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 12/805, Ghent University, Faculty of Economics and Business Administration.
    18. Lewlisa Saha & Hrudaya Kumar Tripathy & Tarek Gaber & Hatem El-Gohary & El-Sayed M. El-kenawy, 2023. "Deep Churn Prediction Method for Telecommunication Industry," Sustainability, MDPI, vol. 15(5), pages 1-21, March.
    19. Seema & Gaurav Gupta, 2024. "Development of fading channel patch based convolutional neural network models for customer churn prediction," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 15(1), pages 391-411, January.
    20. Feng, Yi & Yin, Yunqiang & Wang, Dujuan & Ignatius, Joshua & Cheng, T.C.E. & Marra, Marianna & Guo, Yihan, 2024. "Enhancing e-commerce customer churn management with a profit- and AUC-focused prescriptive analytics approach," Journal of Business Research, Elsevier, vol. 184(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:346:y:2025:i:3:d:10.1007_s10479-025-06516-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.