Temporal and Cost-Sensitive Evaluation Framework for Credit Risk Modeling Under Distributional Shifts

Machine learning-based credit risk models are commonly assessed using discrimination metrics alone. Such evaluation, however, does not fully capture economic consequences, temporal deployment conditions, or changes in the underlying risk environment. This study develops a decision-aligned, temporally consistent evaluation framework for real-world deployment. Using loan-level data, model performance is examined under a rolling forward validation scheme. A coverage-based alert policy is implemented to reflect operational resource constraints. Predictive discrimination is measured using PR-AUC, while economic performance is evaluated through a cost-sensitive saving function. The false-negative cost parameter (λ) is varied between 5 and 25 to assess sensitivity. Performance is also compared across high- and low-default regimes, and alternative alert budgets of 5%, 10%, and 20% are considered to examine policy stability. The results indicate no systematic decline in PR-AUC over time. Changes in λ do not alter predictive ranking, although economic returns scale proportionally with the cost parameter. Economic gains are higher in high-default regimes, yet no structural deterioration is observed in low-default periods. Increasing coverage improves recall but reduces economic benefit due to higher false-positive costs. To consolidate these stability dimensions, the Unified Policy Stability Index (UPSI) is proposed as a composite measure integrating predictive variability, economic volatility, and regime differences. The index indicates sustained overall stability during the study period. The findings suggest that credit risk model evaluation should extend beyond accuracy-centered metrics and incorporate decision consistency, temporal robustness, and policy stability within a deployment-oriented framework.