The link prediction problem for social networks
David Liben-NowellJon M. Kleinberg
Formalizes the link prediction problem for social networks and systematically evaluates a wide range of graph-theoretic proximity measures on large collaboration datasets, demonstrating that network topology alone holds substantial power to predict future interactions.
The link prediction problem asks how well future interactions in a social network can be inferred from a single snapshot of its current structure. This question matters because networks of collaborations, organizational ties, or security-relevant contacts evolve rapidly, and organizations increasingly seek to anticipate or encourage new connections using only the observable pattern of existing ones.
The paper set out to formalize this task and test whether a range of proximity measures drawn from graph theory and social-network analysis could extract useful predictive signal from topology alone. The authors constructed five large co-authorship networks from arXiv physics sections, using three-year windows (1994–1996 for training, 1997–1999 for testing) and restricting attention to a core set of active authors. They evaluated roughly twenty predictors—ranging from simple counts of common neighbors and shortest-path distance to more elaborate methods such as Katz summation over all paths, rooted PageRank, SimRank, and low-rank matrix approximations—by measuring how far each method improved upon a random baseline that merely guessed among non-edges in the training graph.
Several predictors substantially outperformed chance, with the best variants of the Katz measure and certain low-rank approximations achieving roughly 40- to 50-fold gains on three of the five datasets. Common-neighbor counting and the Adamic/Adar refinement also performed strongly and consistently, while shortest-path distance proved weaker once the “small-world” noise of distant cross-field links was taken into account. Even when the evaluation was restricted to pairs at graph distance three or greater, most methods still beat random by factors of five to thirty. Overlap among the top predictors was high, yet no single technique dominated every dataset.
These results indicate that network topology encodes substantial latent information about future collaborations, independent of external attributes such as geography or research topic. The practical implication is that organizations could use lightweight topological scores to surface promising but unrealized partnerships, and that security or intelligence applications could generate plausible hypotheses about unobserved ties. The findings also suggest that models of network growth should be judged not only by their ability to reproduce global statistics but by their success at local link-prediction tasks.
Further gains are likely possible. Promising directions include weighting recent edges more heavily, exploiting the richer bipartite author-paper representation, tuning parameters on temporal slices of the training data, and adapting support-estimation techniques from machine learning. Efficiency improvements for very large graphs remain necessary before the methods can be deployed at scale.
The study is limited to co-authorship data in physics, relies on noisy name-based author disambiguation, and treats all training edges equally; performance may therefore differ in other domains or with richer node attributes. Even the strongest predictor reached only about 16 percent accuracy, confirming that many new links arise for reasons outside the observed network.
- Paper: The PageRank Citation Ranking : Bringing Order to the Web, Lawrence Page et al. (1999). Understanding PageRank's foundational link-analysis algorithm provides the essential algorithmic basis for the network proximity measures adapted in the source paper.
- Paper: Mining the network value of customers, Pedro M. Domingos et al. (2001). This early work on valuing customer network ties provides foundational context for modeling interactions and influence within social networks.
- Paper: node2vec: Scalable Feature Learning for Networks, Aditya Grover et al. (2016). This paper extends the link prediction problem by introducing scalable feature learning methods that automatically generate node and edge representations.
- Paper: LINE: Large-scale Information Network Embedding, Jian Tang et al. (2015). This work builds directly upon network proximity concepts to develop scalable information network embeddings for link prediction.
- Paper: Variational Graph Auto-Encoders, Thomas N. Kipf et al. (2016). This paper extends graph-based link prediction into the realm of variational auto-encoders by combining network structure with node features.
