A Robustness Analysis of Newspaper-based Indices

In this paper, we subject the methodology for newspaper-based indices to several tests of robustness, to address the potential problems of this proposed approach. Firstly, we examine the strong dependency between the selected keywords and the entered query. We do this using state-of-the-art language models, such as BERT, to automatically select relevant articles to build the index. Secondly, we propose that the weighting of articles partly allows for the control of the context of the articles and potential errors in the incorrect identification of articles, which leads to more stable index results. Finally, we track composition changes in newspaper articles, which have been evolving over time. The implications of these tests may be of interest to the users of these indices as well as suggesting a future direction for this approach.