Bias, Calibration and Stability in Logistic Regression Models: A Comparative Simulation Study of MLE, Firth and Ridge Methods

Logistic regression is widely used to model binary outcomes but the traditional maximum likelihood estimation performs poorly in small samples, rare events and in cases of correlated predictors. This simulation study compared the MLE, the Firth’s bias-reduced and the ridge-penalized logistic regression across diverse sample sizes (n = 20, 100, 1000), event rates (5%, 20%, 50%) and predictor correlations (Ï = 0.1, 0.5, 0.8). The key performance metrics were estimation bias, calibration slope and bootstrap-based coefficient variability. Results show that MLE suffers extreme bias and instability in small samples with rare events but recovers well at n = 1000 thus achieving nearly perfect calibration with slope approximately 1. The Firth’s method mitigates bias and complete-separation issues in small samples though it also introduces severe calibration distortion (slopes >50). Ridge regression on the other hand provide the most stable coefficient estimates from the bootstrap SDs significantly lower than those of MLE but shows inconsistent calibration especially under sparse conditions. Overall, the Firth is recommended for inference in sparse data, the ridge for prediction in high-dimensional and multicollinear settings and the MLE for large and well-powered datasets. This study demonstrates the significance of aligning estimation methods with those of data characteristics to ensure accuracy and robustness of logistic regression modeling.