On approximate validation of models: a Kolmogorov–Smirnov-based approach

Classical tests of fit typically reject a model for large enough real data samples. In contrast, often in statistical practice, a model offers a good description of the data even though it is not the ‘true’ random generator. We consider a more flexible approach based on contamination neighbourhoods: using trimming methods and the Kolmogorov metric, we introduce a functional statistic measuring departures from a contaminated model. We show how the plug-in estimator allows testing of fit for the (slightly) contaminated model vs sensible deviations from it, with uniformly exponentially small type I and type II error probabilities. We also address the asymptotic behaviour of the estimator showing that, under suitable regularity conditions, it asymptotically behaves as the supremum of a Gaussian process. As an application, we explore methods of comparison between descriptive models based on the paradigm of model falseness. We also include some connections of our approach with the false discovery rate setting, showing competitive behaviour when estimating the contamination level, and being applicable in a wider framework.