Evaluating credit risk models

Over the past decade, commercial banks have devoted many resources to developing internal models to better quantify their financial risks and assign economic capital. These efforts have been recognized and encouraged by bank regulators; for example, the 1997 Market Risk Amendment (MRA) formally incorporates banks' internal, value-at-risk models into regulatory capital calculations. A key component in the implementation of the MRA was the development of standards, such as for model validation, that must be satisfied in order for banks' models to be used for regulatory capital purposes. Recent proposals, such as by the IIF (1998) and ISDA (1998), argue that credit risk models should also be used to determine risk-adjusted capital requirements. ; However, an important question for both users of such models and their regulators is evaluating the accuracy of the model's forecasts of credit losses. A serious impediment to such model validation is the small number of forecasts available due to the long planning horizons typical of credit risk models. Using a panel data approach, we propose evaluation methods for credit risk models based on cross-sectional simulation. Specifically, models are evaluated not only on their forecasts over time, but also on their forecasts at a given point in time for simulated credit portfolios. Once the forecasts corresponding to these portfolios are generated, they can be evaluated using various methods, such as the binomial method commonly used for evaluating value-at-risk models and currently embodied in the MRA.