Performances of Model Selection Criteria When Variables are Ill Conditioned

Model selection criteria are often used to find a “proper” model for the data under investigation when building models in cases in which the dependent or explained variables are assumed to be functions of several independent or explanatory variables. For this purpose, researchers have suggested using a large number of such criteria. These criteria have been shown to act differently, under the same or different conditions, when trying to select the “correct” number of explanatory variables to be included in a given model; this, unfortunately, leads to severe problems and confusion for researchers. In this paper, using Monte Carlo methods, we investigate the properties of four of the most common criteria under a number of realistic situations. These criteria are the adjusted coefficient of determination ( $$\hbox {R}^{2}$$ R 2 -adj), Akaike’s information criterion (AIC), the Hannan–Quinn information criterion (HQC) and the Bayesian information criterion (BIC). The results from this investigation indicate that the HQC outperforms the BIC, the AIC and the $$\hbox {R}^{2}$$ R 2 -adj under specific circumstances. None of them perform satisfactorily, however, when the degree of multicollinearity is high, the sample sizes are small or when the fit of the model is poor (i.e., there is a low $$\hbox {R}^{2})$$ R 2 ) . In the presence of all these factors, the criteria perform very badly and are not very useful. In these cases, the criteria are often not able to select the true model.