Domain matters: Towards domain-informed evaluation for link prediction

Link prediction, a foundational task in complex network analysis, has extensive applications in critical scenarios such as social recommendation, drug target discovery, and knowledge graph completion. However, existing evaluations of algorithmic often rely on experiments conducted on a limited number of networks, assuming consistent performance rankings across domains. Despite the significant disparities in generative mechanisms and semantic contexts, previous studies often improperly highlight “universally optimal” algorithms based solely on naive average over networks across domains. This paper systematically evaluates 16 mainstream link prediction algorithms across 740 real-world networks spanning seven domains. We present substantial empirical evidence elucidating the performance of algorithms in specific domains. This findings reveal a notably low degree of consistency (correlation ≈0.1015) in inter-domain algorithm rankings, a phenomenon that stands in stark contrast to the high degree of consistency observed within individual domains. Principal Component Analysis (PCA) shows that response vectors formed by the rankings of the 16 algorithms cluster distinctly by domain in low-dimensional space, thus confirming domain attributes as a pivotal factor affecting algorithm performance. We propose a metric called Winner Score that could identify the superior algorithm in each domain, which suggest the following domain-specific winners: Non-Negative Matrix Factorization (NMF) for social networks, Neighborhood Overlap-aware Graph Neural Networks (NeoGNN) for economics, Graph Convolutional Networks (GCN) for chemistry, and L3-based Resource Allocation (RA3) for biology. However, these domain-specific top-performing algorithms tend to exhibit suboptimal performance in other domains. This finding underscores the importance of aligning an algorithm’s mechanism (e.g., low-rank modeling, high-order path propagation) with the network structure. In addition, the proposed Ranking Stability Coefficient (RSC) –which quantifies the number of networks required for stable evaluation–reveals a significant discrepancy in the stability requirements across different domains. Highly homogeneous domains (e.g., chemical, social) achieve stability with approximately 10 networks, whereas highly heterogeneous domains (e.g., biology, transportation) require more networks. This study provides empirical evidence for algorithm selection, benchmark construction, and reproducible evaluation, thereby advancing link prediction from general comparison to scenario adaptation.