A Flexible Tool for Model Building: the Relevant Transformation of the Inputs Network Approach (RETINA)

A new method, called relevant transformation of the inputs network approach (RETINA) is proposed as a tool for model building and selection. It is designed to improve on some of the shortcomings of neural networks. RETINA has the flexibility of neural network models, the concavity of the likelihood in the weights of the usual linear models, and the ability to identify a parsimonious set of attributes that are likely to be relevant for predicting out of sample outcomes. It achieves flexibility by considering transformations of the original inputs; it splits the sample into three disjoint subsamples, sorts the candidate regressors by a saliency feature, chooses the models in subsample 1, uses subsample 2 for parameter estimation, and uses subsample 3 for cross-validation. It is modular, can be used as a data exploratory tool, and is computationally feasible in personal computers. In tests on simulated data, it achieves high rates of successes when the sample size or the R2 are large enough. As our experiments show, it is superior to alternative procedures such as the non-negative garrote and backward stepwise regression.