How much data is sufficient for reliable bibliometric domain analysis? A multi-scenario experimental approach

Determining the adequate data size for bibliometric domain analysis is a crucial yet unresolved issue in bibliometric research. In this paper, we propose a systematic approach to address this challenge by considering multiple task scenarios and conducting sampling experiments on five domains. We introduce two indexes to quantitatively evaluate the reliability of sub-bibliographic datasets with different sample sizes in fitting the complete bibliographic datasets, focusing on the impact of scale on dataset completeness. We find that while larger datasets tend to yield better results, diminishing returns are observed as the dataset size increases due to higher costs and time investments. Specific analysis tasks, such as subject category and country analysis (including co-occurrence relationships), can be conducted with smaller dataset sizes. However, analyzing authors and their co-occurrence relationships necessitates a larger dataset size. Nevertheless, different analysis scenarios require varying dataset sizes, especially when considering result ranking, co-occurrence relationship analysis, and top high-frequency elements. We also find that the appropriate dataset scale for analyzing different elements depends on their power-law distribution in the bibliographic dataset. Our findings offer practical guidance for researchers in selecting the appropriate dataset size for their specific analysis tasks, taking into account factors such as domain size, analyzed objects, the number of top values to be analyzed, and result ranking requirements.