Analysis of Thurstone-Motivated Models in the Case of Non-Evaluable Data: Methods for Extracting Information

We study Thurstone-motivated paired comparison models from the perspective of data evaluability, focusing on cases where datasets cannot be directly evaluated. Despite this limitation, such datasets may still contain extractable information. Three main strategies are known in the literature: increasing the number of options, inserting artificial data through perturbation methods, and requesting new real comparisons. We propose a new approach closely related to the latter two. We analyze the structure of the data and introduce the concept of the optimal limit point, related to the supremum of the log-likelihood function. We prove a theorem for determining optimal limit points based on the data structure, which characterizes the information content of the available dataset. We also prove a theorem linking optimal limit points to the limiting behavior of evaluation results obtained via perturbations, thereby explaining why different perturbation methods may yield different outcomes. In addition, we propose a new perturbation method that adds the minimum possible amount of artificial data. Furthermore, the method identifies the most informative object pairs for new real comparisons, enabling a full evaluation of the dataset.