IDEAS home Printed from https://ideas.repec.org/a/spr/compst/v40y2025i3d10.1007_s00180-024-01539-5.html
   My bibliography  Save this article

Performance of evaluation metrics for classification in imbalanced data

Author

Listed:
  • Alex Cruz Huayanay

    (Pontificia Universidad Católica del Perú
    USP/UFSCar)

  • Jorge L. Bazán

    (University of São Paulo)

  • Cibele M. Russo

    (University of São Paulo)

Abstract

This paper investigates the effectiveness of various metrics for selecting the adequate model for binary classification when data is imbalanced. Through an extensive simulation study involving 12 commonly used metrics of classification, our findings indicate that the Matthews Correlation Coefficient, G-Mean, and Cohen’s kappa consistently yield favorable performance. Conversely, the area under the curve and Accuracy metrics demonstrate poor performance across all studied scenarios, while other seven metrics exhibit varying degrees of effectiveness in specific scenarios. Furthermore, we discuss a practical application in the financial area, which confirms the robust performance of these metrics in facilitating model selection among alternative link functions.

Suggested Citation

  • Alex Cruz Huayanay & Jorge L. Bazán & Cibele M. Russo, 2025. "Performance of evaluation metrics for classification in imbalanced data," Computational Statistics, Springer, vol. 40(3), pages 1447-1473, March.
    • Handle: RePEc:spr:compst:v:40:y:2025:i:3:d:10.1007_s00180-024-01539-5
    DOI: 10.1007/s00180-024-01539-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00180-024-01539-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00180-024-01539-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Rameshwar Gupta & Ramesh Gupta, 2008. "Analyzing skewed data by power normal model," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 17(1), pages 197-210, May.
    2. J.L. Bazán & F. Torres‐Avilés & A.K. Suzuki & F. Louzada, 2017. "Power and reversal power links for binary regressions: An application for motor insurance policyholders," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 33(1), pages 22-34, January.
    3. Artur J. Lemonte & Jorge L. Bazán, 2018. "New links for binary regression: an application to coca cultivation in Peru," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 27(3), pages 597-617, September.
    4. Dan Li & Xia Wang & Lizhen Lin & Dipak K. Dey, 2016. "Flexible link functions in nonparametric binary regression with Gaussian process priors," Biometrics, The International Biometric Society, vol. 72(3), pages 707-719, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guillermo Martínez-Flórez & Roger Tovar-Falón & Héctor W. Gómez, 2024. "Mathematical Formalization and Applications to Data with Excess of Zeros and Ones of the Unit-Proportional Hazard Inflated Models," Mathematics, MDPI, vol. 12(22), pages 1-23, November.
    2. García, V.J. & Gómez-Déniz, E. & Vázquez-Polo, F.J., 2010. "A new skew generalization of the normal distribution: Properties and applications," Computational Statistics & Data Analysis, Elsevier, vol. 54(8), pages 2021-2034, August.
    3. Arthur Pewsey & Héctor Gómez & Heleno Bolfarine, 2012. "Likelihood-based inference for power distributions," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 21(4), pages 775-789, December.
    4. Henry R. Scharf & Xinyi Lu & Perry J. Williams & Mevin B. Hooten, 2022. "Constructing Flexible, Identifiable and Interpretable Statistical Models for Binary Data," International Statistical Review, International Statistical Institute, vol. 90(2), pages 328-345, August.
    5. Guillermo Martínez-Flórez & Carlos Barrera-Causil & Artur J. Lemonte, 2022. "Power Families of Bivariate Proportional Hazard Models," Mathematics, MDPI, vol. 10(23), pages 1-18, November.
    6. Roger Tovar-Falón & Guillermo Martínez-Flórez & Heleno Bolfarine, 2022. "Modelling Asymmetric Data by Using the Log-Gamma-Normal Regression Model," Mathematics, MDPI, vol. 10(7), pages 1-16, April.
    7. Weiji Su & Xia Wang & Rhonda D. Szczesniak, 2021. "Flexible link functions in a joint hierarchical Gaussian process model," Biometrics, The International Biometric Society, vol. 77(2), pages 754-764, June.
    8. Wesley Bertoli & Katiane S. Conceição & Marinho G. Andrade & Francisco Louzada, 2018. "On the zero-modified Poisson–Shanker regression model and its application to fetal deaths notification data," Computational Statistics, Springer, vol. 33(2), pages 807-836, June.
    9. Difang Huang & Jiti Gao & Tatsushi Oka, 2022. "Semiparametric Single-Index Estimation for Average Treatment Effects," Papers 2206.08503, arXiv.org, revised Jan 2025.
    10. Guillermo Martínez-Flórez & Diego I. Gallardo & Osvaldo Venegas & Heleno Bolfarine & Héctor W. Gómez, 2021. "Flexible Power-Normal Models with Applications," Mathematics, MDPI, vol. 9(24), pages 1-15, December.
    11. Guillermo Martínez-Flórez & Roger Tovar-Falón & Heleno Bolfarine, 2023. "The Log-Bimodal Asymmetric Generalized Gaussian Model with Application to Positive Data," Mathematics, MDPI, vol. 11(16), pages 1-14, August.
    12. R. N. Rattihalli, 2023. "A Class of Multivariate Power Skew Symmetric Distributions: Properties and Inference for the Power-Parameter," Sankhya A: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 85(2), pages 1356-1393, August.
    13. Xiaoyue Zhao & Lin Zhang & Dipankar Bandyopadhyay, 2021. "A Shared Spatial Model for Multivariate Extreme-Valued Binary Data with Non-Random Missingness," Sankhya B: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 83(2), pages 374-396, November.
    14. Guillermo Martínez-Flórez & Heleno Bolfarine & Héctor Gómez, 2015. "Doubly censored power-normal regression models with inflation," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 24(2), pages 265-286, June.
    15. Artur J. Lemonte & Jorge L. Bazán, 2018. "New links for binary regression: an application to coca cultivation in Peru," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 27(3), pages 597-617, September.
    16. Jianxuan Liu & Yanyuan Ma & Lan Wang, 2018. "An alternative robust estimator of average treatment effect in causal inference," Biometrics, The International Biometric Society, vol. 74(3), pages 910-923, September.
    17. Lemonte, Artur J., 2013. "A new exponential-type distribution with constant, decreasing, increasing, upside-down bathtub and bathtub-shaped failure rate function," Computational Statistics & Data Analysis, Elsevier, vol. 62(C), pages 149-170.
    18. Guillermo Martínez-Flórez & Roger Tovar-Falón & Víctor Leiva & Cecilia Castro, 2024. "Skew-Normal Inflated Models: Mathematical Characterization and Applications to Medical Data with Excess of Zeros and Ones," Mathematics, MDPI, vol. 12(16), pages 1-23, August.
    19. Emilio Gómez-Déniz & Barry C. Arnold & José M. Sarabia & Héctor W. Gómez, 2021. "Properties and Applications of a New Family of Skew Distributions," Mathematics, MDPI, vol. 9(1), pages 1-18, January.
    20. Chénangnon Frédéric Tovissodé & Aliou Diop & Romain Glèlè Kakaï, 2021. "Inference in skew generalized t-link models for clustered binary outcome via a parameter-expanded EM algorithm," PLOS ONE, Public Library of Science, vol. 16(4), pages 1-31, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:compst:v:40:y:2025:i:3:d:10.1007_s00180-024-01539-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.