Comparative assessment of machine learning models for landslide susceptibility mapping: a focus on validation and accuracy

Accurate landslide susceptibility mapping (LSM) is critical to risk management, especially in areas with significant development. Although the receiver operating characteristic–area under the curve (ROC–AUC) performance metrics are commonly used to measure model effectiveness, showed that these are not enough to check the reliability of the generated maps. In this study, the effectiveness of three machine learning models—logistic regression (LR), random forest (RF), and support vector machine (SVM)—were evaluated and compared in predicting landslide risk in a hilly region east of Cairo, Egypt. A comprehensive dataset was gathered to achieve that, including 183 landslide and 183 non-landslide locations, which were detected through fieldwork and high-resolution satellite imagery. Fourteen conditioning factors from different categories; topographical, geological, hydrological, anthropological, and trigger-related variables, were used as independent factors during the generation of the different LSM. All three models achieved high ROC–AUC values, with RF scoring 0.95, SVM 0.90, and LR 0.88, indicating strong performance. However, further assessment with additional performance metrics like accuracy (ACC), recall, precision, F1 score, and check rationality of the maps revealed key differences. Among the models, only the RF model appeared as the most reliable, with superior across all performance metrics, and fewer misclassifications in critical areas. In contrast, SVM and LR exhibited higher misclassification rates for both landslide-prone and safe locations. These findings show that high ROC–AUC values do not always equate to practical reliability.