Self-supervised Graph Learning for Recommendation

Jiancan WuXiang WangFuli FengXiangnan HeLiang ChenJianxun LianXing Xie

article2020SIGIR1,846 citations

Develops Self-supervised Graph Learning (SGL), a contrastive framework for graph neural networks that overcomes degree bias and interaction noise to improve recommendation accuracy, especially for long-tail items.

Listen

Modern recommendation platforms increasingly use graph-based neural architectures to connect users with items. However, standard models suffer from three critical weaknesses: user-item interactions are highly sparse, popular items disproportionately bias recommendations at the expense of niche or long-tail products, and user click data frequently contains misleading noise. These limitations degrade recommendation quality, user satisfaction, and business catalog exposure.

The article demonstrates how integrating self-supervised learning—a method that creates auxiliary training signals directly from the input data without requiring manual labeling—into graph recommendation architectures resolves these core weaknesses.

To evaluate this framework, the authors introduced Self-Supervised Graph Learning (SGL) and integrated it with the state-of-the-art LightGCN baseline. SGL creates multiple views of the user-item network via three data augmentation techniques: dropping random nodes, dropping random edges, and conducting random walks. It then applies contrastive learning to enforce consistency between different views of the same user or item while maximizing divergence between distinct entities. The authors evaluated SGL against leading collaborative filtering and neural baselines across three large-scale benchmark datasets (Yelp2018, Amazon-Book, and Alibaba-iFashion) representing hundreds of thousands of users and millions of interactions.

The findings confirm substantial performance improvements across multiple dimensions. First, SGL-equipped models consistently outperformed all baseline methods, yielding statistically significant accuracy gains, such as a 9.13% recall improvement and a 12.46% ranking gain over the strongest baseline on Amazon-Book. Second, edge dropout proved to be the most effective augmentation strategy, significantly boosting recommendations for low-degree, long-tail items by reducing the overbearing influence of highly popular items. Third, SGL accelerated model training by roughly 40 times on benchmark datasets, reaching peak performance in fewer than 20 training epochs compared to over 700 epochs for standard LightGCN. Fourth, SGL exhibited marked resilience to noisy inputs, maintaining superior recommendation accuracy on contaminated datasets containing up to 20% adversarial noise.

These results demonstrate that auxiliary contrastive objectives can reduce compute cycles and lower infrastructure training costs while simultaneously improving revenue opportunities by surfacing long-tail inventory. Moreover, theoretical analysis reveals that the framework inherently performs hard negative mining—automatically prioritizing challenging, informative data points during optimization—which drives rapid model convergence.

Engineering and data science teams should consider adopting edge-dropout-based contrastive learning within existing graph recommendation pipelines. Organizations should perform hyperparameter sweeps to tune the contrastive temperature parameter, as values between 0.1 and 1.0 balance hard-negative mining without destabilizing optimization. For high-throughput environments, mini-batch approximations can be deployed to maintain training efficiency.

While the empirical results across the three evaluated benchmark datasets provide high confidence in SGL’s efficacy, the framework was evaluated primarily under uniform user-item interaction settings with discrete identifiers. Stakeholders should pilot the framework on internal datasets with domain-specific metadata and heterogeneous features before executing enterprise-wide production deployment.

Microsoft National University of Singapore Sun Yat-sen University University of Science and Technology of China contrastive learning multiple views random walk self-supervised graph learning graph-based recommendation graph contrastive learning

arXiv: 2010.10783 wujcan/SGL

Paper: LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation, Xiangnan He et al. (2020). LightGCN serves as the direct backbone architecture and base collaborative filtering model that SGL enhances with auxiliary self-supervised learning objectives.
Paper: Neural Graph Collaborative Filtering, Xiang Wang et al. (2019). This paper establishes high-order graph convolution on user-item interaction graphs for collaborative filtering, identifying foundational principles that SGL adapts and improves.
Paper: Graph Convolutional Neural Networks for Web-Scale Recommender Systems, Rex Ying et al. (2018). PinSage introduces scalable random-walk graph convolutions for recommendation, providing critical context on the graph aggregation techniques that SGL critiques for noise vulnerability.
Paper: Graph Contrastive Learning with Augmentations, Yuning You et al. (2020). GraphCL pioneers graph contrastive learning via structural augmentations like node and edge perturbation, directly inspiring SGL's multi-view generation operators on user-item graphs.
Paper: Deep Graph Infomax, Petar Veličković et al. (2019). Deep Graph Infomax introduces self-supervised mutual information maximization on graphs, providing foundational principles for self-discrimination objectives.
Paper: Graph Neural Networks in Recommender Systems: A Survey, Shiwen Wu et al. (2020). This comprehensive survey provides essential background on the design choices, strengths, and limitations of graph neural networks across recommendation domains.
Paper: Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning, Qimai Li et al. (2018). This work analyzes the smoothing dynamics and theoretical limits of neighborhood aggregation in graph convolutional networks.
Paper: Semi-Supervised Classification with Graph Convolutional Networks, Thomas N. Kipf et al. (2017). This seminal paper introduces the fundamental graph convolutional network architecture whose message-passing mechanics are built upon by recommendation GCNs.

Paper: Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss, Jeff Z. HaoChen et al. (2021). This theoretical study establishes provable guarantees for contrastive learning via spectral decomposition on augmentation graphs, formalizing principles underlying multi-view graph self-supervision.
Paper: Exploring Simple Siamese Representation Learning, Xinlei Chen et al. (2021). This paper investigates minimalist dual-view representation learning architectures, providing insights into non-contrastive and simplified Siamese learning dynamics.

Cover for Self-supervised Graph Learning for Recommendation

Abstract

Representation learning on user-item graph for recommendation has evolved from using single ID or interaction history to exploiting higher-order neighbors. This leads to the success of graph convolution networks (GCNs) for recommendation such as PinSage and LightGCN. Despite effectiveness, we argue that they suffer from two limitations: (1) high-degree nodes exert larger impact on the representation learning, deteriorating the recommendations of low-degree (long-tail) items; and (2) representations are vulnerable to noisy interactions, as the neighborhood aggregation scheme further enlarges the impact of observed edges.

In this work, we explore self-supervised learning on user-item graph, so as to improve the accuracy and robustness of GCNs for recommendation. The idea is to supplement the classical supervised task of recommendation with an auxiliary self-supervised task, which reinforces node representation learning via self-discrimination. Specifically, we generate multiple views of a node, maximizing the agreement between different views of the same node compared to that of other nodes. We devise three operators to generate the views -- node dropout, edge dropout, and random walk -- that change the graph structure in different manners. We term this new learning paradigm as \textit{Self-supervised Graph Learning} (SGL), implementing it on the state-of-the-art model LightGCN. Through theoretical analyses, we find that SGL has the ability of automatically mining hard negatives. Empirical studies on three benchmark datasets demonstrate the effectiveness of SGL, which improves the recommendation accuracy, especially on long-tail items, and the robustness against interaction noises. Our implementations are available at \url{this https URL}.

1 Introduction
2 Preliminaries
3 Methodology
3.1 Data Augmentation on Graph Structure
3.2 Contrastive Learning
3.3 Multi-task Training
3.4 Theoretical Analyses of SGL
3.5 Complexity Analyses of SGL
4 Experiments
4.1 Experimental Settings
4.1.1 Compared Methods
4.1.2 Hyper-parameter Settings
4.2 Performance Comparison (RQ1)
4.2.1 Comparison with LightGCN
4.2.2 Comparison with the State-of-the-Arts
4.3 Benefits of SGL (RQ2)
4.3.1 Long-tail Recommendation
4.3.2 Training Efficiency
4.3.3 Robustness to Noisy Interactions
4.4 Study of SGL (RQ3)
4.4.1 Effect of Temperature τ\tau
4.4.2 Effect of Pre-training
4.4.3 Effect of Negatives
5 Related Work
5.1 Graph-based Recommendation
5.2 Self-supervised Learning
6 Conclusion and Future Work
References
A Gradient of InfoNCE Loss w.r.t. node representation

Knowls

Knowl 1 — Self-Supervised Graph Learning Framework for Recommendation
model/method
Self-supervised Graph Learning (SGL) is a collaborative filtering paradigm that supplements the primary supervised recommendation task with an auxiliary self-supervised contrastive learning task on the user-item interaction graph $\mathcal{G} = (\mathcal{V}, \mathcal{E})$ , where $\mathcal{V} = \mathcal{U} \cup \mathcal{I}$ comprises user nodes $\mathcal{U}$ and item nodes $\mathcal{I}$ , and edges $\mathcal{E}$ represent observed interactions.

SGL operates by applying graph structure augmentation operators to create multiple corrupted views of the interaction graph. For an ego node $u$ , a Graph Convolutional Network (GCN) encoder (such as LightGCN) generates node representations for two augmented graph views $s_1(\mathcal{G})$ and $s_2(\mathcal{G})$ :

$\mathbf{z}_u' = f_{\text{readout}}(\{\mathbf{z}_u^{(l)\prime} \mid l = 0, \dots, L\}), \quad \mathbf{z}_u'' = f_{\text{readout}}(\{\mathbf{z}_u^{(l)\prime\prime} \mid l = 0, \dots, L\})$

where $\mathbf{z}_u^{(l)}$ denotes the node representation at layer $l$ , obtained via neighborhood aggregation on the augmented graph, and $L$ is the number of GCN layers. The auxiliary self-supervised learning (SSL) objective enforces representation agreement between different views of the same node (positive pair) while contrasting against views of different nodes (negative pairs). SGL trains the encoder jointly using both the supervised recommendation loss and the SSL contrastive loss in a multi-task learning formulation without adding extra trainable parameters.
Knowl 2 — Graph Structure Augmentation Operators for SGL
model/method
To generate augmented views of the user-item bipartite graph $\mathcal{G} = (\mathcal{V}, \mathcal{E})$ , SGL defines three stochastic graph augmentation operators parameterized by a dropout probability $\rho \in [0, 1)$ :
1. Node Dropout (ND): Drops nodes along with their incident edges with probability $\rho$ . Two augmented graph views $s_1(\mathcal{G})$ and $s_2(\mathcal{G})$ are formed by applying independent masking vectors $\mathbf{M}', \mathbf{M}'' \in \{0, 1\}^{|\mathcal{V}|}$ to the vertex set:
$s_1(\mathcal{G}) = (\mathbf{M}' \odot \mathcal{V}, \mathcal{E}), \quad s_2(\mathcal{G}) = (\mathbf{M}'' \odot \mathcal{V}, \mathcal{E})$
1. Edge Dropout (ED): Drops edges in the graph with probability $\rho$ . Two augmented views are created using independent masking vectors $\mathbf{M}_1, \mathbf{M}_2 \in \{0, 1\}^{|\mathcal{E}|}$ applied to the edge set:
$s_1(\mathcal{G}) = (\mathcal{V}, \mathbf{M}_1 \odot \mathcal{E}), \quad s_2(\mathcal{G}) = (\mathcal{V}, \mathbf{M}_2 \odot \mathcal{E})$
1. Random Walk (RW): Constructs layer-sensitive subgraphs by applying independent edge dropout masking at each GCN layer $l \in \{1, \dots, L\}$ :
$s_1^{(l)}(\mathcal{G}) = (\mathcal{V}, \mathbf{M}_1^{(l)} \odot \mathcal{E}), \quad s_2^{(l)}(\mathcal{G}) = (\mathcal{V}, \mathbf{M}_2^{(l)} \odot \mathcal{E})$

where $\mathbf{M}_1^{(l)}, \mathbf{M}_2^{(l)} \in \{0, 1\}^{|\mathcal{E}|}$ are layer-specific masking vectors. Unlike ND and ED which share one augmented subgraph across all GCN layers per epoch, RW allows high-order paths blocked in one layer to be reconnected in another.
Knowl 3 — Contrastive Node Self-Discrimination and Multi-Task Objective
equation
In SGL, contrastive learning maximizes the agreement of positive pairs (representations of the same node under two augmented views) and minimizes agreement with negative pairs (representations of different nodes). For user node $u \in \mathcal{U}$ , the InfoNCE loss with cosine similarity is defined as:

$\mathcal{L}_{\text{ssl}}^{\text{user}} = \sum_{u \in \mathcal{U}} -\log \frac{\exp(s(\mathbf{z}_u', \mathbf{z}_u'') / \tau)}{\sum_{v \in \mathcal{U}} \exp(s(\mathbf{z}_u', \mathbf{z}_v'') / \tau)}$

where $\mathbf{z}_u', \mathbf{z}_u''$ are the $L_2$ -normalized representations of user $u$ from the two augmented views, $s(\mathbf{a}, \mathbf{b}) = \frac{\mathbf{a}^\top \mathbf{b}}{\|\mathbf{a}\|_2 \|\mathbf{b}\|_2}$ is the cosine similarity, and $\tau > 0$ is the softmax temperature hyperparameter. The item-side contrastive loss $\mathcal{L}_{\text{ssl}}^{\text{item}}$ is defined symmetrically over all items $i \in \mathcal{I}$ . The overall self-supervised loss is:

$\mathcal{L}_{\text{ssl}} = \mathcal{L}_{\text{ssl}}^{\text{user}} + \mathcal{L}_{\text{ssl}}^{\text{item}}$

The overall training objective combines the Bayesian Personalized Ranking (BPR) recommendation loss $\mathcal{L}_{\text{main}}$ , the contrastive loss $\mathcal{L}_{\text{ssl}}$ , and an $L_2$ regularization term:

$\mathcal{L} = \sum_{(u, i, j) \in \mathcal{O}} -\log \sigma(\hat{y}_{ui} - \hat{y}_{uj}) + \lambda_1 \mathcal{L}_{\text{ssl}} + \lambda_2 \|\Theta\|_2^2$

where $\mathcal{O} = \{(u, i, j) \mid (u, i) \in \mathcal{O}^+, (u, j) \in \mathcal{O}^-\}$ is the pairwise training dataset formed by observed interactions $\mathcal{O}^+$ and unobserved interactions $\mathcal{O}^-$ , $\hat{y}_{ui} = \mathbf{z}_u^\top \mathbf{z}_i$ is the inner-product score, $\sigma(\cdot)$ is the sigmoid function, $\Theta$ is the set of trainable model parameters (ID embeddings), $\lambda_1$ controls the weight of SSL, and $\lambda_2$ controls $L_2$ regularization.
Knowl 4 — Dynamic Hard Negative Mining Property of SGL
theoretical result
Let $\mathbf{s}_u' = \frac{\mathbf{z}_u'}{\|\mathbf{z}_u'\|_2}$ and $\mathbf{s}_v'' = \frac{\mathbf{z}_v''}{\|\mathbf{z}_v''\|_2}$ be the normalized node representations of positive node $u$ and negative node $v$ from two augmented graph views. The gradient of the InfoNCE loss $\mathcal{L}_{\text{ssl}}^{\text{user}}(u)$ with respect to $\mathbf{z}_u'$ can be decomposed into contributions from the positive pair and negative pairs:

$\frac{\partial \mathcal{L}_{\text{ssl}}^{\text{user}}(u)}{\partial \mathbf{z}_u'} = \frac{1}{\tau \|\mathbf{z}_u'\|_2} \left[ \mathbf{c}(u) + \sum_{v \in \mathcal{U} \setminus \{u\}} \mathbf{c}(v) \right]$

where the negative contribution vector is:

$\mathbf{c}(v) = \left( \mathbf{s}_v'' - (\mathbf{s}_u^{\prime\top} \mathbf{s}_v'') \mathbf{s}_u' \right) P_{uv}, \quad P_{uv} = \frac{\exp(\mathbf{s}_u^{\prime\top} \mathbf{s}_v'' / \tau)}{\sum_{w \in \mathcal{U}} \exp(\mathbf{s}_u^{\prime\top} \mathbf{s}_w'' / \tau)}$

The $L_2$ norm of the gradient contribution $\mathbf{c}(v)$ from negative sample $v$ is proportional to $g(x)$ :

$\|\mathbf{c}(v)\|_2 \propto g(x) = \sqrt{1 - x^2} \exp\left(\frac{x}{\tau}\right)$

where $x = \mathbf{s}_u^{\prime\top} \mathbf{s}_v'' \in [-1, 1]$ represents the cosine similarity between the positive node $u$ and the negative node $v$ .

The function $g(x)$ attains its global maximum at:

$x^* = \frac{\sqrt{\tau^2 + 4} - \tau}{2}$

with maximum value $\ln g(x^*) = \ln \sqrt{1 - (x^*)^2} + \frac{x^*}{\tau}$ .

Theoretical implications:
1. For small temperatures (e.g., $\tau \to 0.1$ ), $x^*$ approaches $1$ (e.g., $x^* \approx 0.95$ ), and $g(x^*)$ increases super-exponentially (reaching magnitudes around $e^8 \approx 4000$ ), while $g(x) \to 0$ for easy negatives ( $x < 0$ ). Thus, lower temperature selectively forces the model to learn predominantly from hard negative samples ( $x > 0$ ).
2. For large temperatures (e.g., $\tau = 1.0$ ), $g(x)$ remains uniformly low ( $< 1.5$ ) across all $x \in [-1, 1]$ , treating hard and easy negatives almost identically.

Knowl 5 — Time and Space Complexity Analysis of SGL

theoretical result

For a user-item bipartite graph with $|\mathcal{V}|$ nodes, $|\mathcal{E}|$ edges, embedding dimension $d$ , $L$ graph convolution layers, batch size $B$ , number of epochs $s$ , and keep probability $\hat{\rho} = 1 - \rho$ :

Space Complexity: SGL introduces zero additional parameters beyond the node ID embeddings, maintaining the same space complexity as the base GCN (e.g., LightGCN), which is $O(|\mathcal{V}|d)$ .

Time Complexity Comparison per Epoch / Training Phase:

Component	LightGCN	SGL-ED
Adjacency Matrix Normalization	$O(2\|\mathcal{E}\|)$	$O(4\hat{\rho}\|\mathcal{E}\|s + 2\|\mathcal{E}\|)$
Graph Convolution	$O(2\|\mathcal{E}\|Lds \frac{\|\mathcal{E}\|}{B})$	$O(2(1+2\hat{\rho})\|\mathcal{E}\|Lds \frac{\|\mathcal{E}\|}{B})$
BPR Loss	$O(2\|\mathcal{E}\|ds)$	$O(2\|\mathcal{E}\|ds)$
Self-Supervised Loss (Full Negatives)	–	$O(\|\mathcal{E}\|d(2+\|\mathcal{V}\|)s)$
Self-Supervised Loss (In-Batch Negatives)	–	$O(\|\mathcal{E}\|d(2+2B)s)$

When using in-batch negative sampling, the time complexity per epoch for SGL-ED scales linearly with batch size $B$ rather than the full node set size $|\mathcal{V}|$ . On the Yelp2018 benchmark, SGL-ED with in-batch negatives requires approximately $3.7\times$ more time per epoch than LightGCN (60.6s vs. 15.2s), but overall training time is reduced due to substantially faster epoch convergence.

Knowl 6 — Top-K Recommendation Performance Comparison

data/table

The recommendation performance of SGL-ED (3-layer GCN architecture) evaluated against baseline models across three benchmark datasets under an all-ranking protocol with a 7:1:2 train/validation/test split at $K=20$ is summarized below:

Dataset	Yelp2018		Amazon-Book		Alibaba-iFashion
Method	Recall@20	NDCG@20	Recall@20	NDCG@20	Recall@20	NDCG@20
NGCF	0.0579	0.0477	0.0344	0.0263	0.1043	0.0486
LightGCN	0.0639	0.0525	0.0411	0.0315	0.1078	0.0507
Mult-VAE	0.0584	0.0450	0.0407	0.0315	0.1041	0.0497
DNN+SSL	0.0483	0.0382	0.0438	0.0337	0.0712	0.0325
SGL-ED	0.0675	0.0555	0.0478	0.0379	0.1126	0.0538
% Improv.	5.63%	5.71%	9.13%	12.46%	4.45%	6.11%
$p$ -value	5.92e-8	1.89e-8	5.07e-10	3.63e-10	3.34e-8	4.68e-10

SGL-ED achieves statistically significant improvements ( $p < 0.05$ ) over all baselines on all three datasets. The improvements are particularly pronounced on sparser datasets (Amazon-Book and Alibaba-iFashion), where supervision signal from observed interactions is limited.

Knowl 7 — Ablation of SGL Augmentation Variants Across Layer Depths
empirical result
Comparing SGL variants (SGL-ND, SGL-ED, SGL-RW) against LightGCN across 1, 2, and 3 GCN layers reveals the following patterns:
1. Operator Superiority: SGL-ED achieves the best performance in 10 out of 18 evaluation configurations (e.g., Recall@20 of 0.0675 on Yelp2018, 0.0478 on Amazon-Book with 3 layers). SGL-RW outperforms SGL-ND across all datasets, performing slightly better than SGL-ED on the ultra-sparse Alibaba-iFashion dataset (3-layer Recall@20 of 0.1139 vs 0.1126) because layer-specific sampling reconnects sparse node paths.
2. Instability of Node Dropout: SGL-ND shows unstable performance trends as depth increases (e.g., on Amazon-Book, Recall@20 is 0.0432 at 1 layer, drops to 0.0427 at 2 layers, and rises to 0.0440 at 3 layers). Dropping high-degree nodes causes abrupt structural alterations that disrupt message passing.
3. Mitigation of Over-smoothing: As the number of GCN layers increases from 1 to 3, SGL-ED consistently improves in Recall and NDCG across all datasets (e.g., on Amazon-Book, SGL-ED Recall rises from 0.0451 to 0.0478), demonstrating that node self-discrimination prevents representation over-smoothing in deeper GCN architectures.
Knowl 8 — Long-Tail Item Recommendation Improvement in SGL
empirical result
To evaluate performance across item popularity tiers, items were divided into 10 groups such that each group contains an equal number of total interactions, with Group 1 having the lowest item degrees (longest tail) and Group 10 containing the highest degree items (accounting for only 0.83%, 0.83%, and 0.22% of unique items in Yelp2018, Amazon-Book, and Alibaba-iFashion, respectively).
1. Base Model Bias: In LightGCN, Group 10 items account for 39.72%, 39.92%, and 51.92% of the total Recall@20 score on Yelp2018, Amazon-Book, and Alibaba-iFashion, respectively, indicating severe bias toward high-degree items.
2. De-biasing via SGL: SGL-ED reduces the contribution of Group 10 to 36.27%, 29.15%, and 35.07% on the respective datasets.
3. Tail Performance: The absolute improvements in top- $K$ recommendation accuracy generated by SGL come predominantly from increased recall in Groups 1 through 9 (long-tail and medium-degree items), demonstrating that auxiliary self-supervised learning helps learn higher-quality representations for items with sparse interaction signals.
Knowl 9 — Training Convergence Acceleration via SGL
empirical result
Incorporating the InfoNCE self-supervised loss into GCN training accelerates convergence:
1. Epochs to Convergence: On Yelp2018, SGL-ED reaches optimal validation and test performance at epoch 18, compared to epoch 720 for LightGCN. On Amazon-Book, SGL-ED achieves peak performance at epoch 16, compared to epoch 700 for LightGCN.
2. Mechanisms: This speedup is attributed to two properties of SGL: (a) InfoNCE uses multiple negative samples simultaneously in the denominator, expanding the gradient perception field compared to pairwise BPR which compares against only one negative per step; and (b) dynamic hard negative mining driven by temperature $\tau$ provides significantly larger, more informative gradient steps during early training.
Knowl 10 — Robustness of SGL to Interaction Noise
empirical result
When synthetic interaction noise is introduced by adding 5%, 10%, 15%, and 20% random negative user-item interactions to the training graph while keeping the test set clean:
1. Lower Performance Degradation: Both SGL-ED and LightGCN exhibit performance degradation as the noise ratio increases, but SGL-ED maintains a consistently lower relative performance drop compared to LightGCN across all noise ratios.
2. Absolute Superiority under Noise: On Amazon-Book, SGL-ED trained on data with 20% added noisy interactions still achieves a higher Recall@20 than LightGCN trained on the completely clean (0% noise) interaction data.
3. Mechanism: Contrasting representations across independently augmented graph views enforces invariance to isolated edges, attenuating the propagation of false-positive interaction noise during neighborhood aggregation.

Knowl 11 — Ablation of Training Paradigms and Negative Sampling Variants in SGL

empirical result

Evaluating design variations of SGL on Yelp2018 and Amazon-Book demonstrates:

Dataset	Yelp2018		Amazon-Book
Method	Recall@20	NDCG@20	Recall@20	NDCG@20
SGL-ED-batch	0.0670	0.0549	0.0472	0.0374
SGL-ED-merge	0.0671	0.0547	0.0464	0.0368
SGL-pre	0.0653	0.0533	0.0429	0.0333
SGL-ED	0.0675	0.0555	0.0478	0.0379

Joint Training vs. Pre-training: Joint multi-task optimization (SGL-ED) outperforms two-stage pre-training and fine-tuning (SGL-pre: pre-training on InfoNCE then fine-tuning on BPR), although SGL-pre still outperforms base LightGCN without SSL initialization.
Type-Specific vs. Merged Negatives: Differentiating node types during in-batch negative sampling (SGL-ED-batch, where users are negatives only for users and items for items) outperforms pooling all nodes in the batch as common negatives (SGL-ED-merge).
Batch Negatives Efficiency: In-batch negative sampling (SGL-ED-batch) achieves performance comparable to full-dataset negative sampling (SGL-ED) while substantially reducing memory and computational cost per batch.

Coverage note — None was omitted; all main contributions, theoretical gradient derivations/analyses, complexity results, model architectures, and empirical evaluations across datasets and variants were covered.

References

1.Hakan Bagci and Pinar Karagoz. 2016. Context-Aware Friend Recommendation for Location Based Social Networks using Random Walk. In WWW. 531–536.
2.Shumeet Baluja, Rohan Seth, D. Sivakumar, Yushi Jing, Jay Yagnik, Shankar Kumar, Deepak Ravichandran, and Mohamed Aly. 2008. Video suggestion and discovery for youtube: taking random walks through the view graph. In WWW. 895–904.
3.Immanuel Bayer, Xiangnan He, Bhargav Kanagal, and Steffen Rendle. 2017. A Generic Coordinate Descent Framework for Learning from Implicit Feedback. In WWW. 1341–1350.
4.Yixin Cao, Xiang Wang, Xiangnan He, Zikun Hu, and Tat-Seng Chua. 2019. Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences. In WWW. 151–161.
5.Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2020. Bias and Debias in Recommender System: A Survey and Future Directions. CoRR abs/2010.03240 (2020).
6.Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. 2020. A Simple Framework for Contrastive Learning of Visual Representations. CoRR abs/2002.05709 (2020).
7.Weijian Chen, Yulong Gu, Zhaochun Ren, Xiangnan He, Hongtao Xie, Tong Guo, Dawei Yin, and Yongdong Zhang. 2019. Semi-supervised User Profiling with Heterogeneous Graph Attention Networks. In IJCAI. 2116–2122.
8.Wen Chen, Pipei Huang, Jiaming Xu, Xin Guo, Cheng Guo, Fei Sun, Chao Li, Andreas Pfadler, Huan Zhao, and Binqiang Zhao. 2019. POG: Personalized Outfit Generation for Fashion Recommendation at Alibaba iFashion. In SIGKDD. 2662–2670.
9.Aaron Clauset, Cosma Rohilla Shalizi, and Mark E. J. Newman. 2009. Power-Law Distributions in Empirical Data. SIAM 51, 4 (2009), 661–703.
10.Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT. 4171–4186.
11.Spyros Gidaris, Praveer Singh, and Nikos Komodakis. 2018. Unsupervised Representation Learning by Predicting Image Rotations. In ICLR.
12.Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. 2017. Neural Message Passing for Quantum Chemistry. In ICML, Vol. 70. 1263–1272.
13.Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In AISTATS, Vol. 9. 249–256.
14.Michael Gutmann and Aapo Hyvärinen. 2010. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In AISTATS, Vol. 9. 297–304.
15.William L. Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In NeurIPS. 1024–1034.
16.Kaveh Hassani and Amir Hosein Khasahmadi. 2020. Contrastive Multi-View Representation Learning on Graphs. In ICML. 3451–3461.
17.Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross B. Girshick. 2020. Momentum Contrast for Unsupervised Visual Representation Learning. In CVPR. 9726–9735.
18.Ruining He and Julian J. McAuley. 2016. Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering. In WWW. 507–517.
19.Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yong-Dong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. In SIGIR. 639–648.
20.Xiangnan He, Zhankui He, Jingkuan Song, Zhenguang Liu, Yu-Gang Jiang, and Tat-Seng Chua. 2018. NAIS: Neural Attentive Item Similarity Model for Recommendation. TKDE 30, 12 (2018), 2354–2366.
21.Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural Collaborative Filtering. In WWW. 173–182.
22.R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Philip Bachman, Adam Trischler, and Yoshua Bengio. 2019. Learning deep representations by mutual information estimation and maximization. In ICLR.
23.Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay S. Pande, and Jure Leskovec. 2020. Strategies for Pre-training Graph Neural Networks. In ICLR.
24.Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised Contrastive Learning. In NeurIPS.
25.Yehuda Koren. 2008. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In KDD. 426–434.
26.Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In ICLR.
27.Zekun Li, Zeyu Cui, Shu Wu, Xiaoyu Zhang, and Liang Wang. 2019. Fi-GNN: Modeling Feature Interactions via Graph Neural Networks for CTR Prediction. In CIKM. 539–548.
28.Dawen Liang, Rahul G. Krishnan, Matthew D. Hoffman, and Tony Jebara. 2018. Variational Autoencoders for Collaborative Filtering. In WWW. 689–698.
29.Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In NIPS. 3111–3119.
30.Stasa Milojevic. 2010. Power law distributions in information science: Making the case for logarithmic binning. J. Assoc. Inf. Sci. Technol. 61, 12 (2010), 2417–2425.
31.Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. 2020. GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training. In KDD. 1150–1160.
32.Zhaochun Ren, Shangsong Liang, Piji Li, Shuaiqiang Wang, and Maarten de Rijke. 2017. Social Collaborative Viewpoint Regression with Explainable Recommendations. In WSDM. 485–494.
33.Steffen Rendle and Christoph Freudenthaler. 2014. Improving pairwise learning for item recommendation from implicit feedback. In WSDM. 273–282.
34.Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In UAI. 452–461.
35.Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. 2020. InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization. In ICLR.
36.Xianfeng Tang, Huaxiu Yao, Yiwei Sun, Yiqi Wang, Jiliang Tang, Charu Aggarwal, Prasenjit Mitra, and Suhang Wang. 2020. Investigating and Mitigating Degree-Related Biases in Graph Convolutional Networks. In CIKM.
37.Yonglong Tian, Dilip Krishnan, and Phillip Isola. 2019. Contrastive Multiview Coding. CoRR abs/1906.05849 (2019).
38.Rianne van den Berg, Thomas N. Kipf, and Max Welling. 2017. Graph Convolutional Matrix Completion. CoRR abs/1706.02263 (2017).
39.Aäron van den Oord, Nal Kalchbrenner, Lasse Espeholt, Koray Kavukcuoglu, Oriol Vinyals, and Alex Graves. 2016. Conditional Image Generation with PixelCNN Decoders. In NIPS. 4790–4798.
40.Aäron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation Learning with Contrastive Predictive Coding. CoRR abs/1807.03748 (2018).
41.Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In ICLR.
42.Petar Velickovic, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R. Devon Hjelm. 2019. Deep Graph Infomax. In ICLR.
43.Hongwei Wang, Miao Zhao, Xing Xie, Wenjie Li, and Minyi Guo. 2019. Knowledge Graph Convolutional Networks for Recommender Systems. In WWW. 3307–3313.
44.Wenjie Wang, Fuli Feng, Xiangnan He, Liqiang Nie, and Tat-Seng Chua. 2021. Denoising Implicit Feedback for Recommendation. In WSDM.
45.Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019. KGAT: Knowledge Graph Attention Network for Recommendation. In SIGKDD. 950–958.
46.Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural Graph Collaborative Filtering. In SIGIR. 165–174.
47.Zhirong Wu, Yuanjun Xiong, Stella X. Yu, and Dahua Lin. 2018. Unsupervised Feature Learning via Non-Parametric Instance Discrimination. In CVPR. 3733–3742.
48.Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks?. In ICLR.
49.Tiansheng Yao, Xinyang Yi, Derek Zhiyuan Cheng, Felix X. Yu, Aditya Krishna Menon, Lichan Hong, Ed H. Chi, Steve Tjoa, Jieqi Kang, and Evan Ettinger. 2020. Self-supervised Learning for Deep Models in Recommendations. CoRR abs/2007.12865 (2020).
50.Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, and Jure Leskovec. 2018. Graph Convolutional Neural Networks for Web-Scale Recommender Systems. In KDD. 974–983.
51.Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020. Graph Contrastive Learning with Augmentations. In NeurIPS.
52.Yuning You, Tianlong Chen, Zhangyang Wang, and Yang Shen. 2020. When Does Self-Supervision Help Graph Convolutional Networks?. In ICML, Vol. 119. 10871–10880.
53.Fajie Yuan, Xiangnan He, Alexandros Karatzoglou, and Liguang Zhang. 2020. Parameter-efficient transfer from sequential behaviors for user modeling and recommendation. In SIGIR. 1469–1478.
54.Weinan Zhang, Tianqi Chen, Jun Wang, and Yong Yu. 2013. Optimizing top-n collaborative filtering via dynamic negative item sampling. In SIGIR. 785–788.
55.Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020. Sˆ3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization. In CIKM.
56.Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. 2020. Graph Contrastive Learning with Adaptive Augmentation. CoRR abs/2010.14945 (2020).

Citation

MLA

Wu, J., et al. “Self-supervised Graph Learning for Recommendation”. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021, pp. 726–35, https://doi.org/10.1145/3404835.3462862.

APA

Wu, J., Wang, X., Feng, F., He, X., Chen, L., Lian, J., & Xie, X. (2021). Self-supervised Graph Learning for Recommendation. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 726–735. https://doi.org/10.1145/3404835.3462862

Chicago

Wu, J., X. Wang, F. Feng, et al. 2021. “Self-supervised Graph Learning for Recommendation”. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 726–35. https://doi.org/10.1145/3404835.3462862.

Harvard

Wu, J. et al. (2021) “Self-supervised Graph Learning for Recommendation”, Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, pp. 726–735. Available at: https://doi.org/10.1145/3404835.3462862.

Vancouver

1. Wu J, Wang X, Feng F, He X, Chen L, Lian J, Xie X (2021) Self-supervised Graph Learning for Recommendation. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, pp 726–735

BibTeX

@inproceedings{Wu_2021, series={SIGIR ’21}, title={Self-supervised Graph Learning for Recommendation}, url={http://dx.doi.org/10.1145/3404835.3462862}, DOI={10.1145/3404835.3462862}, booktitle={Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval}, publisher={ACM}, author={Wu, Jiancan and Wang, Xiang and Feng, Fuli and He, Xiangnan and Chen, Liang and Lian, Jianxun and Xie, Xing}, year={2021}, month=July, pages={726–735}, collection={SIGIR ’21} }

Metadata:Crossref

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/publicdomain/zero/1.0/

Self-supervised Graph Learning for Recommendation

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Self-Supervised Graph Learning Framework for Recommendation

Knowl 2 — Graph Structure Augmentation Operators for SGL

Knowl 3 — Contrastive Node Self-Discrimination and Multi-Task Objective

Knowl 4 — Dynamic Hard Negative Mining Property of SGL

Knowl 5 — Time and Space Complexity Analysis of SGL

Knowl 6 — Top-K Recommendation Performance Comparison

Knowl 7 — Ablation of SGL Augmentation Variants Across Layer Depths

Knowl 8 — Long-Tail Item Recommendation Improvement in SGL