# Are Graph Augmentations Necessary? Simple Graph Contrastive Learning for
Recommendation

Junliang Yu  The University of QueenslandBrisbaneAustralia<EMAIL_ADDRESS>,
Hongzhi Yin  The University of QueenslandBrisbaneAustralia<EMAIL_ADDRESS>,
Xin Xia  The University of QueenslandBrisbaneAustralia<EMAIL_ADDRESS>,  Tong
Chen  The University of QueenslandBrisbaneAustralia<EMAIL_ADDRESS>,
Lizhen Cui  Shandong UniversityJinanChina<EMAIL_ADDRESS>and  Nguyen Quoc Viet
Hung  Griffith UniversityGold CoastAustralia<EMAIL_ADDRESS>
(2022)

###### Abstract.

Contrastive learning (CL) recently has spurred a fruitful line of research in
the field of recommendation, since its ability to extract self-supervised
signals from the raw data is well-aligned with recommender systems’ needs for
tackling the data sparsity issue. A typical pipeline of CL-based
recommendation models is first augmenting the user-item bipartite graph with
structure perturbations, and then maximizing the node representation
consistency between different graph augmentations. Although this paradigm
turns out to be effective, what underlies the performance gains is still a
mystery. In this paper, we first experimentally disclose that, in CL-based
recommendation models, CL operates by learning more evenly distributed
user/item representations that can implicitly mitigate the popularity bias.
Meanwhile, we reveal that the graph augmentations, which were considered
necessary, just play a trivial role. Based on this finding, we propose a
simple CL method which discards the graph augmentations and instead adds
uniform noises to the embedding space for creating contrastive views. A
comprehensive experimental study on three benchmark datasets demonstrates
that, though it appears strikingly simple, the proposed method can smoothly
adjust the uniformity of learned representations and has distinct advantages
over its graph augmentation-based counterparts in terms of recommendation
accuracy and training efficiency. The code is released at
https://github.com/Coder-Yu/QRec.

Self-Supervised Learning, Recommendation, Contrastive Learning, Data
Augmentation

††journalyear: 2022††copyright: acmcopyright††conference: Proceedings of the
45th International ACM SIGIR Conference on Research and Development in
Information Retrieval; July 11–15, 2022; Madrid, Spain††booktitle: Proceedings
of the 45th International ACM SIGIR Conference on Research and Development in
Information Retrieval (SIGIR ’22), July 11–15, 2022, Madrid, Spain††price:
15.00††doi: 10.1145/3477495.3531937††isbn: 978-1-4503-8732-3/22/07††ccs:
Information systems Recommender systems Figure 1. Graph contrastive learning
with edge dropout for recommendation.

##  1\. Introduction

Recently, a resurgence of contrastive learning (CL) (Khosla et al., 2020;
Jaiswal et al., 2021; Liu et al., 2020) has been witnessed in deep
representation learning. Due to the ability to extract the general features
from massive unlabeled data and regularize representations in a self-
supervised manner, CL has led to major advances in multiple research fields
(You et al., 2020; Wu et al., 2021; Chen et al., 2020b; Gao et al., 2021a). As
data annotation is not required in CL, it is a natural antidote to the data
sparsity issue in recommender systems (Sarwar, 2001; Hao et al., 2021). An
increasing number of very recent studies (Wu et al., 2021; Yu et al., 2021b;
Xia et al., 2021b; Zhou et al., 2020; Zhou et al., 2021a; Yu et al., 2021a)
have sought to harness CL for improving recommendation performance and have
demonstrated significant gains. A typical way (Wu et al., 2021) to apply CL to
recommendation is first augmenting the user-item bipartite graph with
structure perturbations (e.g., stochastic edge/node dropout at a certain
ratio), and then maximizing the consistency of representations under different
views learned via a graph encoder. In this setting, the CL task acts as the
auxiliary task, and is jointly optimized with the recommendation task (Fig.
1).

Despite the encouraging results achieved by CL, however, what underlies the
performance gains still remains unclear. Intuitively, we presume that
contrasting different graph augmentations can capture the essential
information existing in the original user-item interactions, by randomly
removing the redundancy and impurity with the edge/node dropout. Unexpectedly,
a few latest works (Yu et al., 2021a; Zhou et al., 2021b; Lee et al., 2021)
have reported that even extremely sparse graph augmentations (with edge
dropout rate 0.9) in CL can bring desired performance gains. Such a phenomenon
is quite elusive and counter-intuitive because a large dropout rate will
result in a huge loss of the raw information and a highly skewed graph
structure. It naturally raises a meaningful question: Do we really need graph
augmentations when integrating CL with recommendation?

To answer this question, we first conduct experiments with and without the
graph augmentations respectively for a performance comparison. The results
show that the when the graph augmentations are absent, the performance is also
comparable to those with graph augmentations. We then investigate the
embedding space learned by non-CL and CL-based recommendation methods. By
visualizing the distributions of the representations and associating them with
their performances, we find that what really matters for the recommendation
performance is the CL loss, rather than the graph augmentation. Optimizing the
contrastive loss InfoNCE (Oord et al., 2018) learns more evenly distributed
user/item representations no matter if graph augmentations are applied, which
implicitly plays a role in mitigating the popularity bias (Chen et al.,
2020a). Meanwhile, despite not as effective as expected, graph augmentations
are not utterly useless in the sense that the properly perturbed versions of
the original graph help learn representations invariant to the disturbance
factors (Bachman et al., 2019; Chen et al., 2020b). However, generating hand-
crafted graph augmentations requires constant reconstruction of the graph
adjacency matrix during training, which is quite time-consuming. In addition,
dropping a critical edge/node (e.g., a cut edge) may split a connected graph
into a few disconnected components, at the risk of making the augmented graph
and the original graph share little learnable invariance. In view of these
defects, a follow-up question then arises: Are there more effective and
efficient augmentation approaches?

In this paper, we give an affirmative answer to the question. On top of our
finding that the uniformity of the representation distribution is the key
point, we develop a graph-augmentation-free CL method in which the uniformity
is more controllable. Technically, we follow the graph CL framework presented
in Fig. 1, but we discard the dropout-based graph augmentation and instead add
random uniform noises to the original representations for a representation-
level data augmentation. Imposing different random noises creates variance
between contrastive views, while the learnable invariance is still retained
due to the controlled magnitude. Compared with the graph augmentation, the
noise version directly regularizes the embedding space towards a more even
distribution, which is easy-to-implement and far more efficient.

The major contributions of this paper are summarized as follows:

  * •

We experimentally unravel why CL can boost recommendation performance and
illustrate that the InfoNCE loss, rather than the graph augmentation, is the
decisive factor.

  * •

We propose a simple yet effective graph-augmentation-free CL method for
recommendation that can regulate the uniformity in a smooth way. It can be an
ideal alternative of cumbersome graph augmentation-based CL methods.

  * •

We conduct a comprehensive experimental study on three benchmark datasets
showing that the proposed method has distinct advantages over its graph
augmentation-based counterparts in terms of recommendation accuracy and model
training efficiency.

##  2\. Investigation of Graph Contrastive Learning in Recommendation

###  2.1. Graph CL for Recommendation

CL is often applied to recommendation with a particular set of presumed
representational invariances to data augmentations (Zhou et al., 2020; Wu et
al., 2021; Yu et al., 2021b; Xie et al., 2020). In this paper, we revisit the
most commonly used dropout-based augmentation on graphs (Wu et al., 2021; You
et al., 2020) which assumes that the representations are invariant to partial
structure perturbations. An investigation is launched into a state-of-the-art
CL-based recommendation model, SGL (Wu et al., 2021), which performs node and
edge dropout to augment the original graph and adopts InfoNCE (Oord et al.,
2018) for CL. Formally, the joint learning scheme in SGL is defined as:

(1) $\mathcal{L}_{joint}=\mathcal{L}_{rec}+\lambda\mathcal{L}_{cl},$

which consists of two losses: recommendation loss $\mathcal{L}_{rec}$ and CL
loss $\mathcal{L}_{cl}$. The InfoNCE in SGL is formulated as:

(2)
$\mathcal{L}_{cl}=\sum_{i\in\mathcal{B}}-\log\frac{\exp(\mathbf{z}_{i}^{\prime\top}\mathbf{z}_{i}^{\prime\prime}/\tau)}{\sum_{j\in\mathcal{B}}\exp(\mathbf{z}_{i}^{\prime\top}\mathbf{z}_{j}^{\prime\prime}/\tau)},$

where $i,\ j$ are users/items in a sampled batch $\mathcal{B}$,
$\mathbf{z}^{\prime}$ ($\mathbf{z}^{\prime\prime}$) are $L_{2}$ normalized
$d$-dimensional node representations learned from two different dropout-based
graph augmentations, and $\tau>0$ (e.g., 0.2) is the temperature. The CL loss
encourages consistency between $\mathbf{z}_{i}^{\prime}$ and
$\mathbf{z}_{i}^{\prime\prime}$ which are the augmented representations of the
same node $i$ and are the positive sample of each other, while minimizing the
agreement between ${z}_{i}^{\prime}$ and $\mathbf{z}_{j}^{\prime\prime}$,
which are the negative samples of each other. To learn the representations
from the user-item graph, SGL employs a popular and effective graph encoder
LightGCN (He et al., 2020) as its backbone, whose message passing process is
defined as:

(3)
$\mathbf{E}=\frac{1}{1+L}(\mathbf{E}^{(0)}+\tilde{\mathbf{A}}\mathbf{E}^{(0)}+...+\tilde{\mathbf{A}}^{L}\mathbf{E}^{(0)}),$

where $\mathbf{E}^{(0)}\in\mathbb{R}^{|N|\times d}$ is the randomly
initialized node embeddings, $|N|$ is the number of nodes, $L$ is the number
of layers, and $\tilde{\mathbf{A}}\in\mathbb{R}^{|N|\times|N|}$ is the
normalized undirected adjacency matrix. By replacing $\tilde{\mathbf{A}}$ with
the adjacency matrices of the corrupted graph augmentations,
$\mathbf{z}^{\prime}$ ($\mathbf{z}^{\prime\prime}$) can be learned via Eq.
(3). Note that,
$\mathbf{z}_{i}^{\prime}=\frac{\mathbf{e}_{i}^{\prime}}{\|\mathbf{e}_{i}^{\prime}\|_{2}}$
and $\mathbf{e}_{i}^{\prime}$ is the corrupted version of $\mathbf{e}_{i}$ in
$\mathbf{E}$. For conciseness, here we just abstract the core ingredients of
SGL and LightGCN. More technical details can be found in the original papers
(Wu et al., 2021; He et al., 2020).

Table 1. Performance comparison of different SGL variants. Method | Yelp2018 | Amazon-Book
---|---|---
Recall@20 | NDCG@20 | Recall@20 | NDCG@20
LightGCN | 0.0639 | 0.0525 | 0.0410 | 0.0318
SGL-ND | 0.0644 | 0.0528 | 0.0440 | 0.0346
SGL-ED | 0.0675 | 0.0555 | 0.0478 | 0.0379
SGL-RW | 0.0667 | 0.0547 | 0.0457 | 0.0356
SGL-WA | 0.0671 | 0.0550 | 0.0466 | 0.0373
CL Only | 0.0245 | 0.0190 | 0.0314 | 0.0258

###  2.2. Necessity of Graph Augmentation

To demystify how CL-based recommendation methods work, we first investigate
the necessity of the graph augmentation in SGL. We construct a new variant of
SGL, termed SGL-WA (WA stands for ‘without augmentation’), in which the CL
loss is:

(4)
$\mathcal{L}_{cl}=\sum_{i\in\mathcal{B}}-\log\frac{\exp(1/\tau)}{\sum_{j\in\mathcal{B}}\exp(\mathbf{z}_{i}^{\top}\mathbf{z}_{j}/\tau)}.$

Because we only learn representations over the original user-item graph, then
we have
$\mathbf{z}_{i}^{\prime}=\mathbf{z}_{i}^{\prime\prime}=\mathbf{z}_{i}$. The
experiments for the performance comparison are conducted on two benchmark
datasets: Yelp2018 and Amazon-Book (He et al., 2020; Wang et al., 2019). A
three-layer setting is adopted and the hyperparameters are tuned according to
the original paper of SGL (more experimental details in Section 4.1). The
results are presented in Table 1. Three variants of SGL (with dropout rate
0.1) proposed in the paper are evaluated (-ND denotes node dropout, -ED is
short for edge dropout, and -RW means random walk (i.e., multi-layer edge
dropout). CL Only means that only the CL loss in SGL is minimized).

As can be observed, all the variants of SGL outperform LightGCN by large
margins, which demonstrates the effectiveness of CL in improving
recommendation performance. To our surprise, when the graph augmentation is
detached, the performance gains are still so remarkable that SGL-WA even
exhibits superiority over SGL-ND and SGL-RW. We conjecture that the node
dropout and random walk (especially the former) are very likely to drop the
key nodes and associated edges and hence break the correlated subgraphs into
disconnected pieces, which highly distort the original graph. Such graph
augmentations share little learnable invariance, and encouraging consistency
between them probably has a negative impact. By contrast, one-time edge
dropout is at a lower risk to largely disturb the semantics of the original
graph, so that SGL-ED can maintain a trivial advantage over SGL-WA, which
suggests the potential of a proper graph augmentation. However, considering
the time-consuming reconstruction of the adjacency matrices in each epoch, we
should rethink the necessity of graph augmentations and search for better
alternatives. Besides, we wonder what underlies the outstanding performance of
SGL-WA since no variances are provided in its CL part.

(a) Distribution of item representations learned from the dataset of Yelp2018.

(b) Distribution of item representations learned from the dataset of Amazon-
Book.

Figure 2. We plot feature distributions with Gaussian kernel density
estimation (KDE) in $\mathbb{R}^{2}$ (the darker the color is, the more points
fall in that area.) and KDE on angles (i.e., arctan2(y, x) for each point
(x,y) $\in\mathcal{S}^{1}$).

###  2.3. InfoNCE Loss Influences More

Wang and Isola (Wang and Isola, 2020) have identified that optimizing the
contrastive loss intensifies two properties in the visual representation
learning: alignment of features from positive pairs, and uniformity of the
normalized feature distribution on the unit hypersphere. It is unclear if the
CL-based recommendation methods exhibit similar patterns that can explain the
results in Section 2.2. Since top-N recommendation is a one-class problem, we
only investigate the uniformity by following the visualization method in (Wang
and Isola, 2020).

We first map the learned representations (randomly sample 2,000 users for each
dataset) to 2-dimensional normalized vectors on the unit hypersphere
$\mathcal{S}^{1}$ (i.e., circle with radius 1) by using t-SNE (Van der Maaten
and Hinton, 2008). All the representations are obtained when the methods reach
their best performance. Then we plot the feature distributions with the
nonparametric Gaussian kernel density estimation (Botev et al., 2010) in
$\mathbb{R}^{2}$ (shown in Fig. 2). For a clearer presentation, the density
estimations on angles for each point on $\mathcal{S}^{1}$ are also visualized.
According to Fig. 2, we can observe notably different feature/density
distributions. In the leftmost column, LightGCN shows highly clustered
features that mainly reside on some narrow arcs. While in the second and the
third columns, the distributions become more uniform, and the density
estimation curves are less sharp, no matter if the graph augmentations are
applied. In the forth column, we plot the features learned only by the
contrastive loss in Eq. (2). The distributions are almost completely uniform.

We think that two reasons may explain the highly clustered feature
distributions. The first is the message passing mechanism in LightGCN. With
the increase of layers, node embeddings become locally similar. The second is
the popularity bias (Chen et al., 2020a) in the recommendation data. Recall
the BPR loss (Rendle et al., 2009) used in LightGCN:

(5)
$\mathcal{L}_{rec}=-\log(\sigma(\mathbf{e}_{u}^{\top}\mathbf{e}_{i}-\mathbf{e}_{u}^{\top}\mathbf{e}_{j})),$

which is with a triplet input $(u,i,j)$. To optimize the BPR loss, we can get
the gradients w.r.t
$\mathbf{e}_{u}$:$\nabla_{\mathbf{e}_{u}}=-\eta(1-s)(\mathbf{e}_{i}-\mathbf{e}_{j})$,
where $\eta$ is the learning rate, $\sigma$ is the sigmoid function,
$s=\sigma(\mathbf{e}_{u}^{\top}\mathbf{e}_{i}-\mathbf{e}_{u}^{\top}\mathbf{e}_{j})$,
$\mathbf{e}_{u}$ is the user embedding, and $\mathbf{e}_{i}$ and
$\mathbf{e}_{j}$ denote the positive and negative item embeddings,
respectively. Since the recommendation data usually follows a long-tail
distribution, when $i$ is a popular item with a large number of interactions,
the user embedding will be constantly updated towards $i$’s direction (i.e.,
$-\nabla_{\mathbf{e}_{u}}$). The message passing mechanism further exacerbates
the clustering problem (i.e., $\mathbf{e}_{u}$ and $\mathbf{e}_{i}$ aggregate
information from each other in the graph convolution) and causes the
representation degeneration (Qiu et al., 2022).

As for the distributions in other columns, by rewriting Eq. (4), we can derive

(6)
$\mathcal{L}_{cl}=\sum_{i\in\mathcal{B}}-1/\tau+\log\big{(}\exp(1/\tau)+\sum_{j\in\mathcal{B}/\\{i\\}}\exp(\mathbf{z}_{i}^{\top}\mathbf{z}_{j}/\tau)\big{)}.$

Because $1/\tau$ is a constant, optimizing the CL loss is actually minimizing
the cosine similarity between different nodes embeddings $\mathbf{e}_{i}$ and
$\mathbf{e}_{j}$, which will push connected nodes away from the high-degree
hubs in the representation space and lead to a more even distribution.

By associating the results in Table 1 with the distributions in Fig. 2, we can
easily draw a conclusion that the uniformity of the distribution is the
underlying factor that has a decisive impact on the recommendation performance
in SGL, rather than the dropout-based graph augmentations. Optimizing the CL
loss can be seen as an implicit way to debias (discussed in section 4.2)
because a more even representation distribution can preserve the intrinsic
characteristics of nodes and improve the generalization ability. This can be a
persuasive explanation for the unexpected performance of SGL-WA. It also
should be noted that, by only minimizing the CL loss in Eq. (2), a poor
performance will be reached, which means that a positive correlation between
the uniformity and the performance only holds in a limited scope. The
excessive pursuit to the uniformity will overlook the closeness of interacted
pairs and similar users/items, and impairs recommendation performance.

##  3\. SimGCL: Simple Graph Contrastive Learning for Recommendation

Based on the findings in Section 2, we speculate that by adjusting the
uniformity of the learned representation in a certain scope, the optimal
performance can be reached. In this section, we aim to develop a Simple Graph
Contrastive Learning method (SimGCL) for recommendation that can smoothly
regulate the uniformity and provide informative variance to maximize the
benefit from CL.

###  3.1. Motivation and Formulation

Since manipulating the graph structure for a more evenly-distributed
representation space is intractable and time-consuming, we shift our attention
to the embedding space. Inspired by the adversarial examples (Goodfellow et
al., 2014) which are constructed by adding imperceptibly small perturbation to
the input images, we directly add random noises to the representation for an
efficient and effective augmentation.

Figure 3. An illustration of the proposed random noise-based data augmentation
in $\mathbb{R}^{2}$.

Formally, given a node $i$ and its representation $\mathbf{e}_{i}$ in the
$d$-dimensional embedding space, we can implement the following
representation-level augmentation:

(7)
$\mathbf{e}_{i}^{\prime}=\mathbf{e}_{i}+\Delta^{\prime}_{i},\,\,\,\mathbf{e}_{i}^{\prime\prime}=\mathbf{e}_{i}+\Delta^{\prime\prime}_{i},$

where the added noise vectors $\Delta^{\prime}_{i}$ and
$\Delta^{\prime\prime}_{i}$ are subject to $\|\Delta\|_{2}=\epsilon$ and
$\Delta=\bar{\Delta}\odot\operatorname{sign}(\mathbf{e}_{i}),\,\,\bar{\Delta}\in\mathbb{R}^{d}\sim
U(0,1)$. The first constraint controls the magnitude of $\Delta$, and $\Delta$
is numerically equivalent to points on a hypersphere with the radius
$\epsilon$. The second constraint requires that $\mathbf{e}_{i}$,
$\Delta^{\prime}$ and $\Delta^{\prime\prime}$ should be in the same
hyperoctant, so that adding the noises will not cause a large deviation of
$\mathbf{e}_{i}$, making less valid positive samples. In Fig. 3, we illustrate
Eq. (7) in $\mathbb{R}^{2}$. By adding the scaled noise vectors to the
original representation, we rotate $\mathbf{e}_{i}$ by two small angles
($\theta_{1}$ and $\theta_{2}$). Each rotation corresponds to a deviation of
$\mathbf{e}_{i}$, and leads to an augmented representation
($\mathbf{e}_{i}^{\prime}$ and $\mathbf{e}_{i}^{\prime\prime}$). Since the
rotation is small enough, the augmented representation retains most
information of the original representation and meanwhile also keeps some
variance. Note that, for each node representation, the added random noises are
different.

Following SGL, we adopt LightGCN as the graph encoder to propagate node
information and amplify the impact of the variance due to its simple structure
and effectiveness. At each layer, different scaled random noises are imposed
on the current node embeddings. The final perturbed node representations are
learned by:

(8)
$\displaystyle\mathbf{E}^{\prime}=\frac{1}{L}\big{(}(\tilde{\mathbf{A}}\mathbf{E}^{(0)}+\mathbf{\Delta}^{(1)})+(\tilde{\mathbf{A}}(\tilde{\mathbf{A}}\mathbf{E}^{(0)}+\mathbf{\Delta}^{(1)})+{\Delta}^{(2)}))+...$
$\displaystyle+(\tilde{\mathbf{A}}^{L}\mathbf{E}^{(0)}+\tilde{\mathbf{A}}^{L-1}\mathbf{\Delta}^{(1)}+...+\tilde{\mathbf{A}}\mathbf{\Delta}^{(L-1)}+\mathbf{\Delta}^{(L)})\big{)}$

It should be mentioned that we skip the input embedding $\mathbf{E}^{(0)}$ in
all the three encoders when calculating the final representations, because we
experimentally find that skipping it can lead to slight performance
improvement in our setting. However, without the CL task, this operation will
result in a performance drop of LightGCN. Finally, we also unify the BPR loss
(Eq. (5)) and the CL loss (Eq. (2)), and then use Adam to optimize the joint
loss presented in Eq. (1).

###  3.2. Regulating Uniformity

In SimGCL, two hyperparameters $\lambda$ and $\epsilon$ can influence the
uniformity of the representations which are critical to the performance. But
$\epsilon$ can explicitly and smoothly regulate the uniformity beyond by only
tuning $\lambda$. By adjusting the value of $\epsilon$, we can directly
control how far the augmented representations deviate from the original.
Intuitively, a larger $\epsilon$ will lead to a more roughly even distribution
of the learned representation, because when the augmented representations are
enough far away from the original, the information lying in their
representations is also considerably influenced by the noises. As the noises
are sampled from a uniform distribution, by contrasting the augmented
representations, the original representation is regularized towards higher
uniformity. We present the following experimental analysis to demonstrate it.

In (Wang and Isola, 2020), a metric is proposed to measure the uniformity of
the representation, which is the logarithm of the average pairwise Gaussian
potential (a.k.a. the Radial Basis Function (RBF) kernel):

(9) $\mathcal{L}_{\text{uniform }}(f)=\log\underset{\underset{u,v\ \sim\
p_{\text{node}}}{\scriptscriptstyle
i.i.d}}{\mathbb{E}}e^{-2\|f(u)-f(v)\|_{2}^{2}}.$

where $f(u)$ outputs the $L_{2}$ normalized embedding of $u$. We choose the
popular items (with more than 200 interactions) and randomly sample 5,000
users in the dataset of Yelp2018 to form the user-item pairs, and then compute
the uniformity of their representations in the SGL variants and SimGCL with
Eq. (9). For a fair comparison, a three-layer setting is applied to all the
compared methods with $\lambda=0.1$. We then tune $\epsilon$ to observe how
the uniformity changes. The uniformity is checked after every epoch, and we
record the values in the first 30 epochs during which the compared methods all
converge to their optimal solutions.

As clearly shown in Fig. 4, similar trends are observed on all the curves. At
the initial stage, all the methods have highly uniformly-distributed
representations because we use Xavier initialization, which is a special
uniform distribution. With the training proceeding, the uniformity declines
($\mathcal{L}_{\text{uniform }}$ gets higher), and after reaching the peak,
the uniformity improves till convergence and maintains this tendency. As for
SimGCL, with the increase of $\epsilon$, it tends to learn more even
representations, and even a very small $\epsilon=0.01$ leads to higher
uniformity compared with the SGL variants. As a result, users (especially the
long-tail users) are less affected by the popular items. In the rightmost
column in Fig. 2, we also plot the representation distributions of SimGCL with
$\epsilon=0.1$. We can clearly see that the distributions are evidently more
even than those learned by SGL variants and LightGCN. All these results can
support our claim that by replacing graph augmentations with the noise-based
augmentations, SimGCL is more capable of controlling the uniformity of learned
representations so as to debias.

Figure 4. Trends of uniformity. The star indicates the epoch where the best
recommendation performance is reached. Lower $\mathcal{L}_{\text{uniform }}$
numbers are better.

###  3.3. Complexity

In this section, we analyze the time complexity of SimGCL, and compare it with
that of LightGCN and its graph-augmentation based counterpart SGL-ED. We
hereby discuss the batch time complexity since the in-batch negative sampling
is a widely used trick in CL (Chen et al., 2020b). Let $|E|$ be the edge
number in the graph, $d$ be the embedding size, $B$ denote the batch size, $M$
represent the node number in a batch, and $\rho$ denote the edge keep rate in
SGL-ED. We can derive:

Table 2. The comparison of time complexity Component | LightGCN | SGL-ED | SimGCL
---|---|---|---
Adjacency Matrix |  $\mathcal{O}(2|E|)$ |  $\mathcal{O}(2|E|+4\rho|E|)$ |  $\mathcal{O}(2|E|)$
Graph Convolution |  $\mathcal{O}(2|E|Ld)$ |  $\mathcal{O}((2+4\rho)|E|Ld)$ |  $\mathcal{O}(6|E|Ld)$
BPR Loss | $\mathcal{O}(2Bd)$ | $\mathcal{O}(2Bd)$ | $\mathcal{O}(2Bd)$
CL Loss | - | $\mathcal{O}(Bd+BMd)$ | $\mathcal{O}(Bd+BMd)$
Table 3. Performance Comparison for different CL methods on three benchmarks. Method | Douban-Book | Yelp2018 | Amazon-Book
---|---|---|---
Recall | NDCG | Recall | NDCG | Recall | NDCG
1-Layer | LightGCN | 0.1394 | 0.1165 | 0.0631 | 0.0515 | 0.0384 | 0.0298
SGL-ND | 0.1619 (+16.1%) | 0.1448 (+24.3%) | 0.0643 (+1.9%) | 0.0529 (+2.7%) | 0.0432 (+12.5%) | 0.0334 (+12.1%)
SGL-ED | 0.1658 (+18.9%) | 0.1491 (+28.0%) | 0.0637 (+1.0%) | 0.0526 (+2.1%) | 0.0451 (+17.4%) | 0.0353 (+18.5%)
SGL-RW | 0.1658 (+18.9%) | 0.1491 (+28.0%) | 0.0637 (+1.0%) | 0.0526 (+2.1%) | 0.0451 (+17.4%) | 0.0353 (+18.5%)
SGL-WA | 0.1628 (+16.8%) | 0.1454 (+24.8%) | 0.0628 (-0.4%) | 0.0525 (+1.9%) | 0.0403 (+4.9%) | 0.0320 (+7.4%)
SimGCL | 0.1720 (+23.4%) | 0.1519 (+30.4%) | 0.0689 (+9.2%) | 0.0572 (+11.1%) | 0.0453 (+18.0%) | 0.0358 (+20.1%)
2-Layer | LightGCN | 0.1485 | 0.1272 | 0.0622 | 0.0504 | 0.0411 | 0.0315
SGL-ND | 0.1622 (+9.2%) | 0.1434 (+12.7%) | 0.0658 (+5.8%) | 0.0538 (+6.7%) | 0.0427 (+3.9%) | 0.0335 (+6.3%)
SGL-ED | 0.1721 (+15.9%) | 0.1525 (+19.9%) | 0.0668 (+7.4%) | 0.0549 (+8.9%) | 0.0468 (+13.9%) | 0.0371 (+17.8%)
SGL-RW | 0.1710 (+15.2%) | 0.1516 (+19.2%) | 0.0644 (+3.5%) | 0.0530 (+5.2%) | 0.0453 (+10.2%) | 0.0358 (+13.7%)
SGL-WA | 0.1687 (+13.6%) | 0.1501 (+18.0%) | 0.0653 (+5.0%) | 0.0544 (+7.9%) | 0.0453 (+10.2%) | 0.0358 (+13.7%)
SimGCL | 0.1770 (+19.2%) | 0.1582 (+24.4%) | 0.0719 (+15.6%) | 0.0601 (+19.2%) | 0.0507 (+23.4%) | 0.0405 (+28.6%)
3-Layer | LightGCN | 0.1501 | 0.1282 | 0.0639 | 0.0525 | 0.0410 | 0.0318
SGL-ND | 0.1626 (+8.3%) | 0.1450 (+13.1%) | 0.0644 (+0.8%) | 0.0528 (+0.6%) | 0.0440 (+7.3%) | 0.0346 (+8.8%)
SGL-ED | 0.1732 (+15.4%) | 0.1551 (+21.0%) | 0.0675 (+5.6%) | 0.0555 (+5.7%) | 0.0478 (+16.6%) | 0.0379 (+19.2%)
SGL-RW | 0.1730 (+15.3%) | 0.1546 (+20.6%) | 0.0667 (+4.4%) | 0.0547 (+4.2%) | 0.0457 (+11.5%) | 0.0356 (+12.0%)
SGL-WA | 0.1705 (+12.0%) | 0.1525 (+19.0%) | 0.0671 (+5.0%) | 0.0550 (+4.8%) | 0.0466 (+13.7%) | 0.0373 (+18.4%)
SimGCL | 0.1772 (+18.1%) | 0.1583 (+23.5%) | 0.0721 (+12.8%) | 0.0601 (+14.5%) | 0.0515 (+25.6%) | 0.0414 (+30.2%)

  * •

For LightGCN and SimGCL, no graph augmentations are required, so they just
need to normalize the original adjacency matrix which has $2|E|$ non-zero
elements. For SGL-ED, two graph augmentations are used and each has 2$\rho|E|$
non-zero elements.

  * •

In the graph convolution stage, a three-encoder architecture (see Fig. 1) is
employed in both SGL-ED and SimGCL to learn augmented node representations.
So, the time costs of SGL-ED and SimGCL are almost three times that of
LightGCN.

  * •

As for the recommendation loss, three methods all use the BPR loss and each
batch contains $B$ interactions, so they have the same time cost in this
component.

  * •

When calculating the CL loss, the computation costs between the
positive/negative samples are $\mathcal{O}(Bd)$ and $\mathcal{O}(BMd)$,
respectively, because each node only considers itself as the positive, while
the other nodes all are negatives.

Comparing SimGCL with SGL-ED, we can clearly see that SGL-ED theoretically
spends less time for graph convolution, and this bonus may offset SimGCL’s
advantage for the adjacency matrix construction. However, when putting them
into practice, we actually observe that SimGCL is much more time-efficient.
That is because, the computation for the graph convolution is mostly finished
on GPUs, while the graph perturbation is performed on CPUs. Besides, in each
epoch, the adjacency matrices of graph augmentations in SGL-ED need to be
reconstructed. While in SimGCL, the adjacency matrix of the original graph
only needs to be generated once before the training. In a nutshell, SimGCL is
far more efficient than SGL, beyond what we can observe from the theoretical
analysis.

##  4\. Experimental Results

###  4.1. Experimental Settings

Datasets. Three public benchmark datasets: Douban-Book (Yu et al., 2021a)
(#user 13,024, #item 22,347, #interaction 792,062), Yelp2018 (He et al., 2020)
(#user 31,668 #item 38,048, #interaction 1,561,406), and Amazon-Book (Wu et
al., 2021) (#user 52,463, #item 91,599, #interaction 2,984,108) are used in
our experiments to evaluate SimGCL. Because we focus on the Top-N
recommendation, following the convention in the previous research (Yu et al.,
2021b, b), we discard ratings less than 4 in Douban-Book, which is with a 1-5
rating scale, and reset the rest to 1. We split the datasets into three parts
(training set, validation set, and test set) with a ratio of 7:1:2. Two common
metrics: Recall@$K$ and NDCG@$K$ are used and we set $K$=20. For a rigorous
and unbiased evaluation, each experiment in this section is conducted 5 times
with ranking all the items and we then report the average result.

Baselines. Besides LightGCN and the SGL variants, the following recent data
augmentation-based methods are compared.

  * •

Mult-VAE (Liang et al., 2018) is a variational autoencoder-based
recommendation model. It can be seen as a special self-supervised
recommendation model because it has a reconstruction objective.

  * •

DNN+SSL (Yao et al., 2021) is a recent DNN-based recommendation method which
adopts the similar architecture in Fig. 1, and conducts feature masking for
CL.

  * •

BUIR (Lee et al., 2021) has a two-branch architecture which consists of a
target network and an online network, and only uses positive examples for
self-supervised learning.

  * •

MixGCL (Huang et al., 2021) designs the hop mixing technique to synthesize
hard negatives for graph collaborative filtering by embedding interpolation.

Hyperparameters. For a fair comparison, we refer to the best hyperparameter
settings reported in the original papers of the baselines and then fine-tune
all the hyperparameters of the baselines with the grid search. As for the
general settings of all the baselines, the Xavier initialization is used on
all the embeddings. The embedding size is 64, the parameter for $L_{2}$
regularization is 10-4 and the batch size is 2048. We use Adam with the
learning rate 0.001 to optimize all the models. In SimGCL and SGL, we
empirically let the temperature $\tau=0.2$, and this value is also reported as
the best in the original paper of SGL.

###  4.2. SGL vs. SimGCL: From a Comprehensive Perspective

As one of the core claims of this paper is that graph augmentations are not
indispensable and inefficient in CL-based recommendation, in this part, we
conduct a comprehensive comparison between SGL and SimGCL in terms of the
recommendation performance, convergence speed, running time, and ability to
debias.

####  4.2.1. Performance Comparison

We first further compare SGL with SimGCL on three different datasets with
different layer settings. We do not report the results with deeper layers
because both SGL and SimGCL reach their best performances with shallow
structures. The used hyperparameters are reported in section 4.3. We thicken
the figures representing the best performance and underline the second best.
The improvements are calculated by using LightGCN as the baseline. According
to Table 3, we can draw the following observations and conclusions:

  * •

All the SGL variants and SimGCL are effective in improving LightGCN under
different settings. The largest improvements are observed on Amazon-Book where
SimGCL can remarkably improve LightGCN by 25.6% on Recall, and 30.2% on NDCG
with a 3-layer setting.

  * •

SGL-ED is the most effective variant of SGL while SGL-ND is the least
effective. When a 2-layer or 3-layer setting is used, SGL-WA outperforms SGL-
ND in most cases and shows advantages over SGL-RW in a few cases. These
results demonstrate that the CL loss is the main driving force of the
performance improvement while intuitive graph augmentations may not be as
effective as expected, and some of them may even lower the performance.

  * •

SimGCL shows the best performance in all the cases, which proves the
effectiveness of the random noised-based data augmentation. Particularly, on
the two larger datasets: Yelp2018 and Amazon-Book, SimGCL significantly
outperforms the SGL variants by large margins.

####  4.2.2. Convergence Speed Comparison

In this part, we show that SimGCL converges much faster than SGL does. A
2-layer setting is used in this part and the other parameters remain
unchanged.

According to Fig. 5 and Fig. 6, we can observe that, SimGCL reaches its best
performance on the test set at the 25${}^{\text{th}}$, the 11${}^{\text{th}}$,
and the 10${}^{\text{th}}$ epoch on Douban-Book, Yelp2018, and Amazon-Book,
respectively. By contrast, SGL-ED peaks at the 38${}^{\text{th}}$, the
17${}^{\text{th}}$, and the 14${}^{\text{th}}$ epoch, respectively. SimGCL
only spends 2/3 epochs that the SGL variants need. Besides, the curve of SGL-
WA almost overlaps that of SGL-ED on Yelp2018 and Amazon-Book, and exhibits
the same tendency to convergence. It seems that the dropout-based graph
augmentations cannot speed up the model for a faster convergence. Despite
that, all the CL-based methods show advantages over LightGCN at the
convergence speed. When the other three methods begin to get overfitted,
LightGCN is still hundreds of epochs distant from getting converged.

In the paper of SGL, the authors guess that the multiple negatives in the CL
loss may contribute to the fast convergence. However, with almost infinite
negative samples created by dropout, SGL-ED is basically on par with SGL-WA in
speeding up the training, though the latter only has a certain number of
negative samples. As for SimGCL, we consider that the remarkable convergence
speed stems from the noises. By analyzing the gradients from the CL loss, we
find that the noises averagely provide a constant increment, working like a
momentum. In addition to the results in Fig. 5 and 6, we also find that the
training accelerates with $\epsilon$ getting larger. But when it is overlarge
(e.g., greater than 1), despite the rapid decrease of BPR loss, SimGCL
requires more time to get converged. A large $\epsilon$ acts like a large
learning rate, causing the progressive zigzag optimization that will overshoot
the minimum.

Table 4. Running time for per epoch (x in the brackets represents times). Method | Douban-Book | Yelp2018 | Amazon-Book
---|---|---|---
Time (s) | Time (s) | Time (s)
LightGCN | 3.6 | 13.6 | 41.5
SGL-WA | 4.4 (1.2x) | 16.3 (1.2x) | 47.0 (1.1x)
SGL-ED | 13.3 (3.7x) | 62.3 (4.6x) | 235.3 (5.7x)
SimGCL | 6.1 (1.7x) | 27.9 (2.1x) | 98.4 (2.4x)
Figure 5. The performance curves in the first 50 epochs. Figure 6. The loss
curves in the first 50 epochs.

####  4.2.3. Running Time Comparison

In this part, we report the real running time that the compared methods cost
for one epoch. The results in Table 4 are collected on an Intel(R) Xeon(R)
Gold 5122 CPU and a GeForce RTX 2080Ti GPU.

As shown in Table 4, we calculate how many times slower the other methods are
when compared with LightGCN. Because there is no graph augmentation in SGL-WA,
we can see its running speed is very close to that of LightGCN. For SGL-ED,
two graph augmentations are required and the computation in this part is
mostly finished on CPUs, so that it is even 5.7 times slower than LightGCN on
Amazon-Book. The running time increases with the volume of the datasets. By
contrast, despite not as fast as SGL-WA, SimGCL is only 2.4 times slower than
LightGCN on Amazon-Book, and the growth trend is far lower than that of SGL-
ED. Considering that SimGCL only needs 2/3 the epochs that SGL-ED spends, it
outperforms SGL in all aspects w.r.t efficiency.

####  4.2.4. Ability to Debias

The InfoNCE loss is found to have the ability to implicitly alleviate the
popularity bias by learning more even representations. To verify that SimGCL
upgrades this ability with the noise-based representation augmentation, we
divide the test set into three subsets in proportion to the popularity of
items. 80% items with the fewest number of clicks/purchases are labelled
‘Unpopular’, 5% items which are most clicked/purchased are labelled ‘Popular’,
and the rest items are labelled ‘Normal’. We then conduct experiments to check
the Recall@20 value that each group contributes (overall Recall@20 value is
the sum of the values from three groups). The results are illustrated in Fig.
7.

We can clearly see that the SimGCL’s improvements all come from the items with
lower popularity. Its prominent advantage on recommending long-tail items
largely compensates for its loss on the ‘Popular’ group. By contrast, LightGCN
is inclined to recommend popular items and achieves the highest recall value
on the last two datasets. The SGL variants fall between LightGCN and SimGCL on
exploring long-tail items and exhibit similar recommendation preference.
Combining Fig. 2 with Fig. 7, we can easily find that there is a positive
correlation between the uniformity of representations and the ability to
debias. Since the popular items probably have been exposed to users from other
sources, recommending them may no be a good choice. On this point, SimGCL
significantly outperforms SGL, and its extraordinary performance on
discovering long-tail items fits the real need of users.

###  4.3. Parameter Sensitivity Analysis

In this part, we investigate the impact of the two important hyperparameters
in SimGCL. Here we adopt the experimental settings used in section 4.2.2.

####  4.3.1. Impact of $\lambda$

By fixing $\epsilon$ at 0.1, we change $\lambda$ to a set of predetermined
representative values presented in Fig. 8. As can be observed, with the
increase of $\lambda$, the performance of SimGCL starts to increase at the
beginning, and it gradually reaches its peak when $\lambda$ is 0.2 on Douban-
Book, 0.5 on Yelp2018, and 2 on Amazon-Book. Afterwards, it starts to decline.
Besides, in contrast to Fig. 9, more dramatic changes are observed in Fig. 8
though $\epsilon$ and $\lambda$ are tuned in the same scope, which
demonstrates that $\epsilon$ can provide a finer-grained regulation beyond
that provided only by tuning $\lambda$.

####  4.3.2. Impact of $\epsilon$

We think a larger $\epsilon$ leads to a more even distribution that can help
to debias. However, when it is too large, the recommendation task will be
hindered because the high similarity between connected nodes cannot be
reflected by an over-uniform distribution. We fix $\lambda$ at the best values
on the three datasets as reported in Fig. 8, and then adjust $\epsilon$ to see
the performance change. As shown in Fig. 9, the shapes of the curves are as
expected. On all the datasets, when $\epsilon$ is near 0.1, SimGCL achieves
the best performance. We also find that initializing embeddings with uniform
distributions (including Xavier initialization) leads to 3%-4% performance
improvement compared with Gaussian initialization.

Figure 7. Performance comparison over different item groups Figure 8.
Influence of the magnitude $\lambda$ of CL.  Figure 9. Influence of the noise
magnitude $\epsilon$.

###  4.4. Performance Comparison with Other Methods

To further confirm the outstanding competence of SimGCL, we compare it with
other four recently proposed data augmentation-based methods. According to
Table 5, SimGCL outperforms all the baselines, and MixGCF is the runner-up.
Meanwhile, we find that some data augmentation-based recommendation methods
are not as powerful as expected, and even outperformed by LightGCN in many
cases. We attribute their failure to: (1). LightGCN, MixGCF, and SimGCL are
all based on the graph convolution mechanism, which are more capable of
modeling graph data compared with Mult-VAE. (2). DNNs are proved effective
when user/item features are available. In our datasets, no features are
provided and we mask embeddings learned by DNN to conduct self-supervised
learning, so it cannot fulfill itself in this situation. (3). In the paper of
BUIR, it removes long-tail nodes to achieve a good performance, but we use all
the users and items. Besides, its siamese network structure may also collapse
to a trivial solution on some long-tail nodes because it does not use negative
examples, which may account for its incompetence.

Table 5. Performance comparison with other SOTA models. Method | Douban-Book | Yelp2018 | Amazon-Book
---|---|---|---
Recall | NDCG | Recall | NDCG | Recal | NDCG
LightGCN | 0.1501 | 0.1282 | 0.0639 | 0.0525 | 0.0411 | 0.0315
Mult-VAE | 0.1310 | 0.1103 | 0.0584 | 0.0450 | 0.0407 | 0.0315
DNN+SSL | 0.1366 | 0.1148 | 0.0483 | 0.0382 | 0.0438 | 0.0337
BUIR | 0.1127 | 0.8938 | 0.0487 | 0.0404 | 0.0260 | 0.0209
MixGCF | 0.1731 | 0.1552 | 0.0713 | 0.0589 | 0.0485 | 0.0378
SimGCL | 0.1772 | 0.1583 | 0.0721 | 0.0601 | 0.0515 | 0.0410

###  4.5. Performance Comparison with Different Types of Noises

In SimGCL, we use the random noises sampled from a uniform distribution to
implement data augmentation. However, there are other types of noises
including the Gaussian noises and adversarial noises. Here we also test
different noises and report their best results in Table 6 (u denotes uniform
noises sampled from $U(0,1)$, p represents the positive uniform noises which
differ from u in not satisfying the second constraint in section 3.1, g
denotes Gaussian noises generated by the standard Gaussian distribution, and a
denotes adversarial noises generated by FGSM (Goodfellow et al., 2014)).
According to Table 6, SimGCL${}_{{}_{g}}$ shows comparable performance while
SimGCL${}_{{}_{a}}$ is less effective. The possible reason is that we apply
$L_{2}$ normalization to the noises. The normalized noises generated by the
standard Gaussian distribution can fit a much flatter Gaussian distribution
(can be easily proved) which approximates a uniform distribution. So, the
comparable results are observed. As for SimGCL${}_{{}_{a}}$, the adversarial
noises are generated by only targeting maximizing the CL loss while the
recommendation loss has a dominant status that impacts the performance more
during optimization. As for SimGCL${}_{{}_{p}}$, we notice a slight
performance drop compared with SimGCL${}_{{}_{u}}$ in most cases, which
suggests the necessity of the directional constraint for creating more
informative augmentations.

Table 6. Performance comparison between different SimGCL variants. Method | Douban-Book | Yelp2018 | Amazon-Book
---|---|---|---
Recall | NDCG | Recall | NDCG | Recal | NDCG
LightGCN | 0.1485 | 0.1272 | 0.0639 | 0.0525 | 0.0411 | 0.0315
SimGCL${}_{{}_{a}}$ | 0.1561 | 0.1379 | 0.0604 | 0.0505 | 0.0455 | 0.0358
SimGCL${}_{{}_{p}}$ | 0.1751 | 0.1565 | 0.0708 | 0.0593 | 0.0514 | 0.0409
SimGCL${}_{{}_{g}}$ | 0.1773 | 0.1586 | 0.0718 | 0.0599 | 0.0511 | 0.0408
SimGCL${}_{{}_{u}}$ | 0.1772 | 0.1583 | 0.0721 | 0.0601 | 0.0515 | 0.0414

##  5\. Related Work

###  5.1. Graph Neural Recommendation Models

Graph Neural Networks (GNNs) (Wu et al., 2020a; Gao et al., 2021b) now have
become widely acknowledged powerful architectures for modeling recommendation
data. This new neural network paradigm ends the regime of MLP-based
recommendation models in the academia, and boosts the neural recommender
systems to a new level. A large number of recommendation models, which adopt
GNNs as their bases, claim that they have achieved state-of-the-art
performance (Yu et al., 2021b; He et al., 2020; Yu et al., 2020) in different
subfields. Particularly, GCN (Kipf and Welling, 2017), as the most prevalent
variant of GNNs, further fuels the development of the graph neural
recommendation models like GCMC (Berg et al., 2017), NGCF (Wang et al., 2019),
LightGCN (He et al., 2020), and LCF (Yu and Qin, 2020). Despite the different
implementations in details, these GCN-driven models share a common idea that
is to acquire the information from the neighbors in the user-item graph layer
by layer to refine the target node’s embeddings and fulfill graph reasoning
(Wu et al., 2020b). Among these methods, LightGCN is the most popular one due
to its simple structure and decent performance. Following (Wu et al., 2019),
it removes the redundant operations including transformation matrices and
nonlinear activation functions. Such a design is proved efficient and
effective, and inspires a lot of follow-up CL-based recommendation models like
SGL (Wu et al., 2021) and MHCN (Yu et al., 2021a).

###  5.2. Contrastive Learning in Recommendation

As CL works in a self-supervised manner (Yu et al., 2022), it is inherently a
possible solution to the data sparsity issue (Yu et al., 2018, 2019) in
recommender systems. Inspired by the achievements of CL in other fields, there
has been a wave of new research that integrates CL with recommendation (Zhou
et al., 2020; Wu et al., 2021; Xia et al., 2021b; Yu et al., 2021a, b; Ma et
al., 2020; Qiu et al., 2021). Zhou et al. (Zhou et al., 2020) adopted random
masking on attributes and items to create sequence augmentations for
sequential model pretraining with mutual information maximization. Wei et al.
(Wei et al., 2021) reformulated the cold-start item representation learning
from an information-theoretic standpoint and maximized the mutual dependencies
between item content and collaborative signals to alleviate the data sparsity
issue. Similar ideas are also found in (Yao et al., 2021), where a two-tower
DNN architecture is developed for recommendation, in which the item tower is
also shared for contrasting augmented item features. SEPT (Yu et al., 2021a)
and COTREC (Xia et al., 2021a) further propose to mine multiple positive
samples with semi-supervised learning on the perturbed graph for
social/session-based recommendation. In addition to the dropout, CL4Rec (Xie
et al., 2020) proposes to reorder and crop item segments for sequential data
augmentation. Yu et al. (Yu et al., 2021b), Zhang et al. (Zhang et al., 2021)
and Xia et al. (Xia et al., 2021b) leveraged hypergraph to model
recommendation data, and proposed to contrast different hypergraph structures
for representation regularization. In addition to the data sparsity problem,
Zhou et al. (Zhou et al., 2021a) theoretically proved that CL can also
mitigate the exposure bias in recommendation, and developed a method named
CLRec to improve deep match in terms of fairness and efficiency.

##  6\. Conclusion

In this paper, we revisit the dropout-based CL in recommendation, and
investigate how it improves recommendation performance. We reveal that, in CL-
based recommendation models, the CL loss is the core and the graph
augmentation only plays a secondary role. Optimizing the CL loss leads to a
more even representation distribution, which helps to debias in the scenario
of recommendation. We then develop a simple graph-augmentation-free CL method
to regulate the uniformity of the representation distribution in a more
straightforward way. By adding directed random noises to the representation
for different data augmentations and contrast, the proposed method can
significantly enhance recommendation. The extensive experiments demonstrate
that the proposed method outperforms its graph augmentation-based counterparts
and meanwhile the training time is dramatically reduced.

## Acknowledgement

This work is supported by Australian Research Council Future Fellowship (Grant
No. FT210100624), Discovery Project (Grant No. DP190101985) and Discovery
Early Career Research Award (Grant No. DE200101465).

## References

  * (1)
  * Bachman et al. (2019) Philip Bachman, R Devon Hjelm, and William Buchwalter. 2019\.  Learning representations by maximizing mutual information across views.  _arXiv preprint arXiv:1906.00910_ (2019).
  * Berg et al. (2017) Rianne van den Berg, Thomas N Kipf, and Max Welling. 2017\.  Graph convolutional matrix completion.  _arXiv preprint arXiv:1706.02263_ (2017).
  * Botev et al. (2010) Zdravko I Botev, Joseph F Grotowski, and Dirk P Kroese. 2010\.  Kernel density estimation via diffusion.  _The annals of Statistics_ 38, 5 (2010), 2916–2957.
  * Chen et al. (2020a) Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2020a.  Bias and debias in recommender system: A survey and future directions.  _arXiv preprint arXiv:2010.03240_ (2020).
  * Chen et al. (2020b) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020b.  A simple framework for contrastive learning of visual representations. In _International conference on machine learning_. PMLR, 1597–1607.
  * Gao et al. (2021b) Chen Gao, Yu Zheng, Nian Li, Yinfeng Li, Yingrong Qin, Jinghua Piao, Yuhan Quan, Jianxin Chang, Depeng Jin, Xiangnan He, et al. 2021b.  Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions.  _arXiv preprint arXiv:2109.12843_ (2021).
  * Gao et al. (2021a) Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021a.  SimCSE: Simple Contrastive Learning of Sentence Embeddings.  _arXiv preprint arXiv:2104.08821_ (2021).
  * Goodfellow et al. (2014) Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014\.  Explaining and harnessing adversarial examples (2014).  _arXiv preprint arXiv:1412.6572_ (2014).
  * Hao et al. (2021) Bowen Hao, Jing Zhang, Hongzhi Yin, Cuiping Li, and Hong Chen. 2021.  Pre-training graph neural networks for cold-start users and items representation. In _Proceedings of the 14th ACM International Conference on Web Search and Data Mining_. 265–273.
  * He et al. (2020) Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yong-Dong Zhang, and Meng Wang. 2020\.  LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. In _Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval_. ACM, 639–648.
  * Huang et al. (2021) Tinglin Huang, Yuxiao Dong, Ming Ding, Zhen Yang, Wenzheng Feng, Xinyu Wang, and Jie Tang. 2021.  MixGCF: An Improved Training Method for Graph Neural Network-based Recommender Systems.  (2021).
  * Jaiswal et al. (2021) Ashish Jaiswal, Ashwin Ramesh Babu, Mohammad Zaki Zadeh, Debapriya Banerjee, and Fillia Makedon. 2021.  A survey on contrastive self-supervised learning.  _Technologies_ 9, 1 (2021), 2.
  * Khosla et al. (2020) Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020.  Supervised Contrastive Learning. In _Advances in Neural Information Processing Systems, NeurIPS 2020_.
  * Kipf and Welling (2017) Thomas N. Kipf and Max Welling. 2017.  Semi-Supervised Classification with Graph Convolutional Networks. In _5th International Conference on Learning Representations, ICLR 2017_.
  * Lee et al. (2021) Dongha Lee, SeongKu Kang, Hyunjun Ju, Chanyoung Park, and Hwanjo Yu. 2021.  Bootstrapping User and Item Representations for One-Class Collaborative Filtering. In _The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , Fernando Diaz, Chirag Shah, Torsten Suel, Pablo Castells, Rosie Jones, and Tetsuya Sakai (Eds.). 1513–1522.
  * Liang et al. (2018) Dawen Liang, Rahul G Krishnan, Matthew D Hoffman, and Tony Jebara. 2018.  Variational autoencoders for collaborative filtering. In _Proceedings of the 2018 world wide web conference_. 689–698.
  * Liu et al. (2020) Xiao Liu, Fanjin Zhang, Zhenyu Hou, Zhaoyu Wang, Li Mian, Jing Zhang, and Jie Tang. 2020.  Self-supervised learning: Generative or contrastive.  _arXiv preprint arXiv:2006.08218_ 1, 2 (2020).
  * Ma et al. (2020) Jianxin Ma, Chang Zhou, Hongxia Yang, Peng Cui, Xin Wang, and Wenwu Zhu. 2020\.  Disentangled Self-Supervision in Sequential Recommenders. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 483–491.
  * Oord et al. (2018) Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018.  Representation learning with contrastive predictive coding.  _arXiv preprint arXiv:1807.03748_ (2018).
  * Qiu et al. (2021) Ruihong Qiu, Zi Huang, and Hongzhi Yin. 2021.  Memory Augmented Multi-Instance Contrastive Predictive Coding for Sequential Recommendation. In _2021 IEEE International Conference on Data Mining (ICDM)_. IEEE, 519–528.
  * Qiu et al. (2022) Ruihong Qiu, Zi Huang, Hongzhi Yin, and Zijian Wang. 2022\.  Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation. In _Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining_. 813–823.
  * Rendle et al. (2009) Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009.  BPR: Bayesian personalized ranking from implicit feedback. In _Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence_. AUAI Press, 452–461.
  * Sarwar (2001) Badrul Munir Sarwar. 2001\.  Sparsity, scalability, and distribution in recommender systems.  (2001).
  * Van der Maaten and Hinton (2008) Laurens Van der Maaten and Geoffrey Hinton. 2008.  Visualizing data using t-SNE.  _Journal of machine learning research_ 9, 11 (2008).
  * Wang and Isola (2020) Tongzhou Wang and Phillip Isola. 2020.  Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In _International Conference on Machine Learning_. PMLR, 9929–9939.
  * Wang et al. (2019) Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019.  Neural graph collaborative filtering. In _Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval_. 165–174.
  * Wei et al. (2021) Yinwei Wei, Xiang Wang, Qi Li, Liqiang Nie, Yan Li, Xuanping Li, and Tat-Seng Chua. 2021\.  Contrastive learning for cold-start recommendation. In _Proceedings of the 29th ACM International Conference on Multimedia_. 5382–5390.
  * Wu et al. (2019) Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. 2019\.  Simplifying graph convolutional networks. In _International conference on machine learning_. PMLR, 6861–6871.
  * Wu et al. (2021) Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. 2021.  Self-supervised graph learning for recommendation. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 726–735.
  * Wu et al. (2020b) Shiwen Wu, Wentao Zhang, Fei Sun, and Bin Cui. 2020b.  Graph Neural Networks in Recommender Systems: A Survey.  _arXiv preprint arXiv:2011.02260_ (2020).
  * Wu et al. (2020a) Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020a.  A comprehensive survey on graph neural networks.  _IEEE Transactions on Neural Networks and Learning Systems_ (2020).
  * Xia et al. (2021a) Xin Xia, Hongzhi Yin, Junliang Yu, Yingxia Shao, and Lizhen Cui. 2021a.  Self-Supervised Graph Co-Training for Session-based Recommendation. In _Proceedings of the 30th ACM International Conference on Information & Knowledge Management_. 2180–2190.
  * Xia et al. (2021b) Xin Xia, Hongzhi Yin, Junliang Yu, Qinyong Wang, Lizhen Cui, and Xiangliang Zhang. 2021b.  Self-Supervised Hypergraph Convolutional Networks for Session-based Recommendation. In _Thirty-Fifth AAAI Conference on Artificial Intelligence_. 4503–4511.
  * Xie et al. (2020) Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Bolin Ding, and Bin Cui. 2020.  Contrastive Learning for Sequential Recommendation.  _arXiv preprint arXiv:2010.14395_ (2020).
  * Yao et al. (2021) Tiansheng Yao, Xinyang Yi, Derek Zhiyuan Cheng, Felix Yu, Ting Chen, Aditya Menon, Lichan Hong, Ed H Chi, Steve Tjoa, Jieqi Kang, et al. 2021\.  Self-supervised Learning for Large-scale Item Recommendations. In _Proceedings of the 30th ACM International Conference on Information & Knowledge Management_. 4321–4330.
  * You et al. (2020) Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020\.  Graph Contrastive Learning with Augmentations.  _Advances in Neural Information Processing Systems_ 33 (2020).
  * Yu et al. (2018) Junliang Yu, Min Gao, Jundong Li, Hongzhi Yin, and Huan Liu. 2018.  Adaptive Implicit Friends Identification over Heterogeneous Network for Social Recommendation. In _Proceedings of the 27th ACM International Conference on Information and Knowledge Management_. ACM, 357–366.
  * Yu et al. (2019) Junliang Yu, Min Gao, Hongzhi Yin, Jundong Li, Chongming Gao, and Qinyong Wang. 2019\.  Generating reliable friends via adversarial training to improve social recommendation. In _2019 IEEE International Conference on Data Mining (ICDM)_. IEEE, 768–777.
  * Yu et al. (2021a) Junliang Yu, Hongzhi Yin, Min Gao, Xin Xia, Xiangliang Zhang, and Nguyen Quoc Viet Hung. 2021a.  Socially-Aware Self-Supervised Tri-Training for Recommendation. In _KDD ’21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_ , Feida Zhu, Beng Chin Ooi, and Chunyan Miao (Eds.). ACM, 2084–2092.
  * Yu et al. (2020) Junliang Yu, Hongzhi Yin, Jundong Li, Min Gao, Zi Huang, and Lizhen Cui. 2020.  Enhance Social Recommendation with Adversarial Graph Convolutional Networks.  _arXiv preprint arXiv:2004.02340_ (2020).
  * Yu et al. (2021b) Junliang Yu, Hongzhi Yin, Jundong Li, Qinyong Wang, Nguyen Quoc Viet Hung, and Xiangliang Zhang. 2021b.  Self-Supervised Multi-Channel Hypergraph Convolutional Network for Social Recommendation. In _Proceedings of the Web Conference 2021_. 413–424.
  * Yu et al. (2022) Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Jundong Li, and Zi Huang. 2022\.  Self-Supervised Learning for Recommender Systems: A Survey.  _arXiv preprint arXiv:2203.15876_ (2022).
  * Yu and Qin (2020) Wenhui Yu and Zheng Qin. 2020.  Graph convolutional network for recommendation with low-pass collaborative filters. In _International Conference on Machine Learning_. PMLR, 10936–10945.
  * Zhang et al. (2021) Junwei Zhang, Min Gao, Junliang Yu, Lei Guo, Jundong Li, and Hongzhi Yin. 2021\.  Double-Scale Self-Supervised Hypergraph Learning for Group Recommendation. In _Proceedings of the 30th ACM International Conference on Information & Knowledge Management_. 2557–2567.
  * Zhou et al. (2021a) Chang Zhou, Jianxin Ma, Jianwei Zhang, Jingren Zhou, and Hongxia Yang. 2021a.  Contrastive learning for debiased candidate generation in large-scale recommender systems. In _Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining_. 3985–3995.
  * Zhou et al. (2020) Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020\.  S^ 3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization.  _arXiv preprint arXiv:2008.07873_ (2020).
  * Zhou et al. (2021b) Xin Zhou, Aixin Sun, Yong Liu, Jie Zhang, and Chunyan Miao. 2021b.  SelfCF: A Simple Framework for Self-supervised Collaborative Filtering.  _arXiv preprint arXiv:2107.03019_ (2021).