IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0276116.html
   My bibliography  Save this article

Machine learning outperformed logistic regression classification even with limit sample size: A model to predict pediatric HIV mortality and clinical progression to AIDS

Author

Listed:
  • Sara Domínguez-Rodríguez
  • Miquel Serna-Pascual
  • Andrea Oletto
  • Shaun Barnabas
  • Peter Zuidewind
  • Els Dobbels
  • Siva Danaviah
  • Osee Behuhuma
  • Maria Grazia Lain
  • Paula Vaz
  • Sheila Fernández-Luis
  • Tacilta Nhampossa
  • Elisa Lopez-Varela
  • Kennedy Otwombe
  • Afaaf Liberty
  • Avy Violari
  • Almoustapha Issiaka Maiga
  • Paolo Rossi
  • Carlo Giaquinto
  • Louise Kuhn
  • Pablo Rojo
  • Alfredo Tagarro
  • on behalf of EPIICAL Consortium

Abstract

Logistic regression (LR) is the most common prediction model in medicine. In recent years, supervised machine learning (ML) methods have gained popularity. However, there are many concerns about ML utility for small sample sizes. In this study, we aim to compare the performance of 7 algorithms in the prediction of 1-year mortality and clinical progression to AIDS in a small cohort of infants living with HIV from South Africa and Mozambique. The data set (n = 100) was randomly split into 70% training and 30% validation set. Seven algorithms (LR, Random Forest (RF), Support Vector Machine (SVM), K-Nearest Neighbor (KNN), Naïve Bayes (NB), Artificial Neural Network (ANN), and Elastic Net) were compared. The variables included as predictors were the same across the models including sociodemographic, virologic, immunologic, and maternal status features. For each of the models, a parameter tuning was performed to select the best-performing hyperparameters using 5 times repeated 10-fold cross-validation. A confusion-matrix was built to assess their accuracy, sensitivity, and specificity. RF ranked as the best algorithm in terms of accuracy (82,8%), sensitivity (78%), and AUC (0,73). Regarding specificity and sensitivity, RF showed better performance than the other algorithms in the external validation and the highest AUC. LR showed lower performance compared with RF, SVM, or KNN. The outcome of children living with perinatally acquired HIV can be predicted with considerable accuracy using ML algorithms. Better models would benefit less specialized staff in limited resources countries to improve prompt referral in case of high-risk clinical progression.

Suggested Citation

  • Sara Domínguez-Rodríguez & Miquel Serna-Pascual & Andrea Oletto & Shaun Barnabas & Peter Zuidewind & Els Dobbels & Siva Danaviah & Osee Behuhuma & Maria Grazia Lain & Paula Vaz & Sheila Fernández-Luis, 2022. "Machine learning outperformed logistic regression classification even with limit sample size: A model to predict pediatric HIV mortality and clinical progression to AIDS," PLOS ONE, Public Library of Science, vol. 17(10), pages 1-13, October.
  • Handle: RePEc:plo:pone00:0276116
    DOI: 10.1371/journal.pone.0276116
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0276116
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0276116&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0276116?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Kuhn, Max, 2008. "Building Predictive Models in R Using the caret Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 28(i05).
    2. Sendhil Mullainathan & Jann Spiess, 2017. "Machine Learning: An Applied Econometric Approach," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 87-106, Spring.
    3. Karatzoglou, Alexandros & Smola, Alexandros & Hornik, Kurt & Zeileis, Achim, 2004. "kernlab - An S4 Package for Kernel Methods in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 11(i09).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Crespo, Cristian, 2020. "Two become one: improving the targeting of conditional cash transfers with a predictive model of school dropout," LSE Research Online Documents on Economics 123139, London School of Economics and Political Science, LSE Library.
    2. Tranos, Emmanouil & Incera, Andre Carrascal & Willis, George, 2022. "Using the web to predict regional trade flows: data extraction, modelling, and validation," OSF Preprints 9bu5z, Center for Open Science.
    3. Madadkhani, Shiva & Ikonnikova, Svetlana, 2024. "Toward high-resolution projection of electricity prices: A machine learning approach to quantifying the effects of high fuel and CO2 prices," Energy Economics, Elsevier, vol. 129(C).
    4. Tobias Rentschler & Philipp Gries & Thorsten Behrens & Helge Bruelheide & Peter Kühn & Steffen Seitz & Xuezheng Shi & Stefan Trogisch & Thomas Scholten & Karsten Schmidt, 2019. "Comparison of catchment scale 3D and 2.5D modelling of soil organic carbon stocks in Jiangxi Province, PR China," PLOS ONE, Public Library of Science, vol. 14(8), pages 1-23, August.
    5. Rachel Sippy & Daniel F Farrell & Daniel A Lichtenstein & Ryan Nightingale & Megan A Harris & Joseph Toth & Paris Hantztidiamantis & Nicholas Usher & Cinthya Cueva Aponte & Julio Barzallo Aguilar & An, 2020. "Severity Index for Suspected Arbovirus (SISA): Machine learning for accurate prediction of hospitalization in subjects suspected of arboviral infection," PLOS Neglected Tropical Diseases, Public Library of Science, vol. 14(2), pages 1-20, February.
    6. Bellotti, Anthony & Brigo, Damiano & Gambetti, Paolo & Vrins, Frédéric, 2021. "Forecasting recovery rates on non-performing loans with machine learning," International Journal of Forecasting, Elsevier, vol. 37(1), pages 428-444.
    7. repec:osf:osfxxx:9bu5z_v1 is not listed on IDEAS
    8. Daoud, Adel & Kim, Rockli & Subramanian, S.V., 2019. "Predicting women's height from their socioeconomic status: A machine learning approach," Social Science & Medicine, Elsevier, vol. 238(C), pages 1-1.
    9. Matthew N Ahmadi & Alok Chowdhury & Toby Pavey & Stewart G Trost, 2020. "Laboratory-based and free-living algorithms for energy expenditure estimation in preschool children: A free-living evaluation," PLOS ONE, Public Library of Science, vol. 15(5), pages 1-14, May.
    10. Mehmet Güney Celbiş, 2021. "A machine learning approach to rural entrepreneurship," Papers in Regional Science, Wiley Blackwell, vol. 100(4), pages 1079-1104, August.
    11. Hirche, Martin & Farris, Paul W. & Greenacre, Luke & Quan, Yiran & Wei, Susan, 2021. "Predicting Under- and Overperforming SKUs within the Distribution–Market Share Relationship," Journal of Retailing, Elsevier, vol. 97(4), pages 697-714.
    12. Tsukioka, Yasutomo & Yanagi, Junya & Takada, Teruko, 2018. "Investor sentiment extracted from internet stock message boards and IPO puzzles," International Review of Economics & Finance, Elsevier, vol. 56(C), pages 205-217.
    13. Prabal Das & D. A. Sachindra & Kironmala Chanda, 2022. "Machine Learning-Based Rainfall Forecasting with Multiple Non-Linear Feature Selection Algorithms," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 36(15), pages 6043-6071, December.
    14. Paulo Infante & Gonçalo Jacinto & Anabela Afonso & Leonor Rego & Pedro Nogueira & Marcelo Silva & Vitor Nogueira & José Saias & Paulo Quaresma & Daniel Santos & Patrícia Góis & Paulo Rebelo Manuel, 2023. "Factors That Influence the Type of Road Traffic Accidents: A Case Study in a District of Portugal," Sustainability, MDPI, vol. 15(3), pages 1-16, January.
    15. Juergen Deppner & Marcelo Cajias, 2024. "Accounting for Spatial Autocorrelation in Algorithm-Driven Hedonic Models: A Spatial Cross-Validation Approach," The Journal of Real Estate Finance and Economics, Springer, vol. 68(2), pages 235-273, February.
    16. Ephrem Habyarimana & Faheem S Baloch, 2021. "Machine learning models based on remote and proximal sensing as potential methods for in-season biomass yields prediction in commercial sorghum fields," PLOS ONE, Public Library of Science, vol. 16(3), pages 1-23, March.
    17. Naguib, Costanza, 2019. "Estimating the Heterogeneous Impact of the Free Movement of Persons on Relative Wage Mobility," Economics Working Paper Series 1903, University of St. Gallen, School of Economics and Political Science.
    18. Philippe Goulet Coulombe & Maxime Leroux & Dalibor Stevanovic & Stéphane Surprenant, 2022. "How is machine learning useful for macroeconomic forecasting?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 920-964, August.
    19. Thiemo Fetzer & Stephan Kyburz, 2024. "Cohesive Institutions and Political Violence," The Review of Economics and Statistics, MIT Press, vol. 106(1), pages 133-150, January.
    20. Dang, Hai-Anh & Carletto, Calogero & Gourlay, Sydney & Abanokova, Kseniya, 2024. "Addressing Soil Quality Data Gaps with Imputation: Evidence from Ethiopia and Uganda," GLO Discussion Paper Series 1445, Global Labor Organization (GLO).
    21. Tobias Götze & Marc Gürtler & Eileen Witowski, 2020. "Improving CAT bond pricing models via machine learning," Journal of Asset Management, Palgrave Macmillan, vol. 21(5), pages 428-446, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0276116. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.