Imbalanced class distribution and performance evaluation metrics: A systematic review of prediction accuracy for determining model performance in healthcare systems

Focus on predictive algorithm and its performance evaluation is extensively covered in most research studies to determine best or appropriate predictive model with Optimum prediction solution indicated by prediction accuracy score, precision, recall, f1score etc. Prediction accuracy score from performance evaluation has been used extensively as the main determining metric for performance recommendation. It is one of the most widely used metric for identifying optimal prediction solution irrespective of dataset class distribution context or nature of dataset and output class distribution between the minority and majority variables. The key research question however is the impact of class inequality on prediction accuracy score in such datasets with output class distribution imbalance as compared to balanced accuracy score in the determination of model performance in healthcare and other real-world application systems. Answering this question requires an appraisal of current state of knowledge in both prediction accuracy score and balanced accuracy score use in real-world applications where there is unequal class distribution. Review of related works that highlight the use of imbalanced class distribution datasets with evaluation metrics will assist in contextualizing this systematic review.Author summary: The incidence of unequal class distribution in real-world applications such as healthcare and other non-medical settings continue to receive attention due to machine learning technique challenges with minority class contribution in datasets with imbalanced class distribution. Challenges such as discounting minority class contribution which may be the subject of interest. Predictive modeling evaluation of such datasets with prediction accuracy score which does not take into account dataset class distribution variation could create an erroneous impression of a supposedly high performing machine learning technique as it discounts minority class contribution. Estimating predictive model performance with balanced accuracy score that incorporates other important metrics such as true positives, true positive rates, true negatives, true negative rates, false positive, false positive rates, false negatives and false negative rates could help assess machine learning model performance more adequately and accurately to determine appropriate model performance.