Decentralized Graph Neural Network for Privacy-Preserving Recommendation
Building a graph neural network (GNN)-based recommender system without
violating user privacy proves challenging. Existing methods can be divided into
federated GNNs and decentralized GNNs. But both methods have undesirable
effects, i.e., low communication efficiency and privacy leakage. This paper
proposes DGREC, a novel decentralized GNN for privacy-preserving
recommendations, where users can choose to publicize their interactions. It
includes three stages, i.e., graph construction, local gradient calculation,
and global gradient passing. The first stage builds a local inner-item
hypergraph for each user and a global inter-user graph. The second stage models
user preference and calculates gradients on each local device. The third stage
designs a local differential privacy mechanism named secure gradient-sharing,
which proves strong privacy-preserving of users' private data. We conduct
extensive experiments on three public datasets to validate the consistent
superiority of our framework.
Xiaolin Zheng, Zhongyu Wang, Chaochao Chen, Jiashu Qian, Yao Yang
# Decentralized Graph Neural Network for Privacy-Preserving Recommendation

###### Abstract.

Building a graph neural network (GNN)-based recommender system without violating user privacy proves challenging. Existing methods can be divided into federated GNNs and decentralized GNNs. But both methods have undesirable effects, i.e., low communication efficiency and privacy leakage. This paper proposes DGREC, a novel decentralized GNN for privacy-preserving recommendations, where users can choose to publicize their interactions. It includes three stages, i.e., graph construction, local gradient calculation, and global gradient passing. The first stage builds a local inner-item hypergraph for each user and a global inter-user graph. The second stage models user preference and calculates gradients on each local device. The third stage designs a local differential privacy mechanism named secure gradient-sharing, which proves strong privacy-preserving of users' private data. We conduct extensive experiments on three public datasets to validate the consistent superiority of our framework.

Recommender system, decentralized graph neural network, privacy protection +
Footnote †: journal: Information systems – Personalization; Security and privacy \(\rightarrow\) Web application security

+
Footnote †: journal: Information systems – Personalization; Security and privacy \(\rightarrow\) Web application security
media, users like to share their moments to gather friends with similar preferences. **CH2:**_protecting user privacy cost-effectively._ Existing privacy-preserving methods, e.g., secret sharing (Kang et al., 2017) and homomorphic encryption (Krishnan et al., 2017), equip heavy mathematical schemes, causing high communication costs. **CH3:**_mitigating the trade-off between effectiveness and efficiency._ In a decentralized GNN, involving one user for training cannot provide sufficient data to train an effective model, whereas involving too many users slows down the training process.

This paper proposes DGREC, a decentralized GNN for privacy-preserving recommendations to solve the above challenges. To overcome **CH1**, we propose allowing users to publicize their interactions freely. DGREC mainly has three stages, i.e., **Stage1:** graph construction, **Stage2:** local gradient calculation, and **Stage3:** global gradient passing. In **Stage1**, all users constitute an _inter-user graph_ collaboratively, and each user constructs an _inner-item hypergraph_ individually. For the inter-user graph, edges come from their friendship or proximity. For the inner-item hypergraph, nodes are the items that the user interacts with or publicizes, and edges come from item tags. In **Stage2**, we model user preference based on the inner-item hypergraph to protect user privacy and enhance preference representation. Specifically, we condense the item hypergraph into an interest graph by mapping items into different interests. Given that the distilled interests can be noisy, we propose an interest attention mechanism that leverages GNN architecture to minimize noisy interests. Finally, we pool the interests to model user preference and calculate local gradients. In **Stage3**, each user samples multi-hop neighbors in the inter-user graph and trains the models collaboratively to overcome **CH3**. We propose a mechanism based on the Local Differential Privacy (LDP) named secure gradient-sharing, which proves strong privacy-preserving of users' private data to overcome **CH2**. Specifically, the calculated gradients in Stage 2 are first encoded, then propagated among the neighbourhood, and finally decoded to restore in a noise-free way.

We summarize the main contributions of this paper as follows:

* [leftmargin=*,noitemsep,topsep=0pt]
* We propose a novel decentralized GNN for privacy-preserving recommendations. To the best of our knowledge, this is the first decentralized GNN-based recommender system.
* We propose secure gradient-sharing, a novel privacy-preserving mechanism to publish model gradients, and theoretically prove that it is noise-free and satisfies Renyi differential privacy.
* We conduct extensive experiments on three public datasets, and consistent superiority validates the success of the proposed framework.

## 2. Related Work

In this section, we discuss prior work on GNN and privacy protection, as we propose a decentralized GNN for privacy-preserving recommendations.

### Gnn

Recent studies have extensively investigated the mechanisms and applications of GNN. For mechanisms, we investigate graph convolutional networks and graph pooling. For applications, we investigate centralized GNNs, federated GNNs, and decentralized GNNs.

**Graph convolutional networks.** Most graph convolutional networks focus on pairwise relationships between the nodes in a graph (Krizhevsky et al., 2014; Krizhevsky et al., 2015). However, they neglect the higher-order connectivity patterns beyond the pairwise relationships (Krizhevsky et al., 2014; Goyal et al., 2015). (Krizhevsky et al., 2015) first introduces hypergraph learning as a propagation process to minimize label differences among nodes. HGNN (Krizhevsky et al., 2015) proposes a hypergraph neural network by truncated Chebyshev polynomials of the hypergraph Laplacian. HyperGNN (Krizhevsky et al., 2015) further enhances the capacity of representation learning by leveraging an optional attention module. Our paper leverages HyperGNN without an attention module to aggregate node features and learn node assignment as we model a user's preferences based on his item hypergraph.

**Graph Pooling.** Graph pooling is widely adopted to calculate the entire representation of a graph. Conventional approaches sum or average node embeddings (Krizhevsky et al., 2014; Goyal et al., 2015; Goyal et al., 2015). However, these methods cannot learn the hierarchical representation of a graph. To address this issue, DiffPool (Krizhevsky et al., 2015) first proposes an end-to-end pooling operator, which learns a differentiable soft cluster assignment for nodes at each layer. MinCutPool (Krizhevsky et al., 2015) further formulates a continuous relaxation of the normalized Min-cut problem to compute the cluster assignment. Motivated by DiffPool, our paper designs a different constraint on cluster assignment: one node can be mapped into multiple clusters, but each cluster should be independent.

**Centralized GNNs.** Centralized GNNs have been vastly applied in recommendation scenarios. Leveraging a graph structure can enhance representations for users and items (Krizhevsky et al., 2016) and capture complex user preferences behind their interactions (Krizhevsky et al., 2015). PinSage (Krizhevsky et al., 2015) and LightGCN (Krizhevsky et al., 2015) refine user and item representations via multi-hop neighbors' information. To deal with noisy interactions and node degree bias, SGL(Wang et al., 2016) explores self-supervised learning on a user-item graph and reinforces node representation via self-discrimination. Although these methods achieve superior performance, they cannot be applied in privacy-preserving recommendation scenarios as most user interactions are private. In contrast, our paper designs a decentralized GNN for privacy-preserving recommendations.

**Federated GNNs.** Federated GNNs involve a central server to or-chestrate the training process and leverage an LDP mechanism (Krizhevsky et al., 2015) to protect user privacy. LPGNN(Wang et al., 2016) and GWGNN(Krizhevsky et al., 2015) require clients to publish local features to the server, where feature aggregation is executed. However, these two methods violate the privacy-preserving requirements in recommendation scenarios: local interactions should not be published. FedGNN (Krizhevsky et al., 2015) assumes a trustworthy third party to share user-item interactions securely and adds Gaussian noises to protect user privacy. However, finding a credible third party in real scenarios is difficult and hence limits the application of this method. FeSoG (Wang et al., 2016) shares user features to make recommendations and adds Laplacian noises to preserve users' privacy. However, user features are sensitive data as malicious participants can leverage these data to infer private user interactions (Krizhevsky et al., 2015). Federated GNNs have high communication costs in the central server, causing training speed bottlenecks. In contrast, our method adopts a decentralized training mechanism to improve training efficiency, where each client updates their model asynchronously.

**Decentralized GNNs.** Decentralized GNNs require clients to cooperate in training prediction models. SpreadGNN (Krizhevsky et al., 2015) and DLGNN (Krizhevsky et al., 2015) calculate model gradients with local interactions and
share gradients among clients to update models. D-FedGNN (Wang et al., 2018) introduces Diffie-hellman key exchange to secure communication and utilizes decentralized stochastic gradient descent algorithm (Srivastava et al., 2017) to train models. P2PGNN (Wang et al., 2018) makes local predictions and uses page rank to diffuse predictions. All these methods exchange gradients without using a privacy-preserving mechanism. Consequently, malicious neighbors can infer users' private data from received gradients. In contrast, our method proposes an LDP mechanism to secure shared gradients, protecting user privacy.

### Privacy-preserving mechanisms

Preserving users' privacy is a topic with increasing attention, including the Differential Privacy (DP) (Beng et al., 2015; Li et al., 2016; Li et al., 2017; Wang et al., 2018; Wang et al., 2018), Homomorphic Encryption (Beng et al., 2015; Li et al., 2016; Li et al., 2017), and Secure Multi-party Computation (Zhu et al., 2018; Wang et al., 2018). Among them, Local Differential Privacy, an implementation of DP, has been embraced by numerous machine-learning scenarios (Wang et al., 2018), where noise is added to individual data before it is centralized in a server. (Li et al., 2017) gives a standard definition of \(e\)-DP, where \(\epsilon\) is the privacy budget. (Li et al., 2017) further defines \((\epsilon,\delta)\)-dp to give a relaxation of \(e\)-DP, where \(\delta\) is an additive term and \(1-\delta\) represents the probability to guarantee privacy. To the best of our knowledge, most existing privacy-preserving GNN-based recommender systems (Wang et al., 2018; Wang et al., 2018) define their privacy protection based on \((\epsilon,\delta)\)-DP. However, \((\epsilon,\delta)\)-DP has an explosive privacy budget for iterative algorithms, and hence it cannot guarantee privacy protection for multiple training steps. To solve this problem, we take \((\alpha,\epsilon)\)-RDP (Wang et al., 2018) to guarantee tighter bounds on the privacy budget for multiple training steps. Our work proves that the secure gradient-sharing mechanism satisfies \((1.5,\epsilon)\)-RDP, guaranteeing stronger privacy protection in real scenarios.

## 3. Method

As seen in Figure 1, the framework of DGREC consists of three stages, i.e., graph construction, local gradient calculation, and global gradient passing. The first stage constructs an inter-user graph and an inner-item hypergraph. The second stage models user preference and calculates local gradients. The third stage proposes a novel privacy-preserving mechanism named secure gradient-sharing. Finally, users update their local models with the decoded gradients. DGREC achieves competitive recommendation performance and meets diverse demands for privacy protection.

### Preliminary

**Renyi differential privacy (RDP).** A randomized algorithm \(M\) satisfies \((\alpha,\epsilon)\)-Renyi differential privacy (Wang et al., 2018), if for any adjacent vectors \(x\) and \(x^{\prime}\), it holds that \(D_{\alpha}(M(x)\parallel M(x^{\prime}))\leq\epsilon\), with \(\epsilon\) denoting the privacy budget and \(\alpha\) denoting the order of Renyi divergence. Small values of \(\epsilon\) guarantee higher levels of privacy, whereas large values of \(\epsilon\) guarantee lower levels of privacy. We can define RDP for any \(\alpha\geq 1\).

**Problem formulation.** Assume we have a set of users and items, denoted by \(\mathbb{U}\) and \(\mathbb{I}\). Generally, user \(u\) has a set of neighbor users \(N_{u}\):\(\{u_{1},u_{2},\cdots,u_{m}\}\) and interacted items \(\mathbb{I}_{u}\):\(\{i_{1},i_{2},\cdots,i_{n}\}\), with \(m\) and \(n\) denoting the number of users and items. User \(u\) can decide whether to publicize their interactions, and we let \(\mathbb{I}_{u}^{pub}\) denote his publicized interactions. The decentralized privacy-preserving recommendation aims to recommend items matching user preferences by leveraging the user's interactions and his neighbors' publicized interactions.

### Graph construction

In the first stage, we construct the inter-user graph and inner-item hypergraph. The inter-user graph will support neighbor sampling and secure gradient-sharing in the third stage. Each user trains his recommendation model collaboratively with his multi-hop neighbors, who are more likely to share similar preferences. The inner-item hypergraph will support modeling user preference in the second stage.

**Inter-user graph construction.** Recent studies demonstrate that user friendship or proximity reveals behavior similarity between users (Wang et al., 2018; Wang et al., 2018). Driven by this motivation, we construct a global inter-user graph, utilizing the communication protocol as described in reference (Wang et al., 2018). As delineated in "Section B: Communication Protocol" of the reference, the implemented protocol demonstrates proficiency in handling both static and dynamic situations. Each user can anonymously disseminate messages among multi-hop neighbors by communicating with his 1-hop neighbors. The inter-user graph is defined as \(\mathcal{G}=\{\mathcal{V},\mathcal{E}\}\), where \(\mathcal{V}\) denotes the user set and \(\mathcal{E}\) denotes the edge set. We set \(N_{u}=\{\sigma:(u,v)\in\mathcal{E}\}\) to denote the neighbors of user \(u\).

**Inner-item hypergraph construction.** To make privacy-preserving recommendations, we model user preferences based on limited interactions, as users' features and most interactions are private. By representing user interactions as a graph, it is easier to distinguish his core and temporary interests, as core interests result in frequent and similar interactions. Here, we build an item hypergraph for each user, where a hyperedge connects multiple items (Beng et al., 2015). To stabilize model training, we further leverage tag information to establish item relationships.

The inner-item hypergraph for user \(u\) is defined as \(\mathbb{G}_{u}=\{\mathbb{V}_{u},A_{u}\}\), where \(\mathbb{V}_{u}=\{i\mid i\in\mathbb{I}_{u}\cup\mathbb{I}_{p}^{pub},\,v\in \mathcal{N}_{u}\}\) denotes the item set and \(A_{u}\) is the incidence matrix. Each entry \(A_{u}(i,t)\) indicates whether item \(i\in\mathbb{V}_{u}\) is connected by a hyperedge \(t\in\mathbb{T}\). In our setting, \(\mathbb{T}\) corresponds to a tag set. We set \(A_{u}(i,t)=1\) if item \(i\) has tag \(t\) and \(A_{u}(i,t)=0\) otherwise. We let \(deg(i)\) denote the degree of item \(i\) and \(deg(t)\) denote the degree of hyperedge \(t\), where \(deg(i)=\sum_{t\in\mathbb{T}}A_{u}(i,t)\) and \(deg(t)=\sum_{i\in\mathbb{V}_{u}}A_{u}(i,t)\). We let \(D_{\theta}\in\mathbb{R}^{[\mathbb{V}_{u}]\times[\mathbb{V}_{u}]}\) and \(D_{t}\in\mathbb{R}^{[\mathbb{T}]\times[\mathbb{T}]}\) denote diagonal matrices of the item and hyperedge degrees, respectively.

### Local gradient calculation

In the second stage, we model each user's preference for an item by his inner-item hypergraph and calculate gradients based on ground-truth interactions.

**Modelling user preference.** We model user preference through three steps, i.e., graph condensation, interest pooling, and preference prediction.

**(1) Graph condensation.** One user interaction provides weak signals for his preferences. To model user preferences, an intuitive way is to gather these weak signals into strong ones. Motivated
by DiffPool(Wang et al., 2017), we condense his item hypergraph into an interest graph by learning a soft cluster assignment.

We first aggregate features among items and learn an item assignment by hypergraph convolutional networks. We define the aggregation and assignation process as,

\[E_{u}^{\prime} =D_{\text{o}}^{-1/2}\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{ \mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{ \mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{   \mathbb{    }}}}}}}}}}}}} \mathcal{D}_{t}^{-1}\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{ \mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{ \mathbb{ \mathbb{ \mathbb{       }}}}}}}}}}}} \mathcal{D}_{t}^{-1/2}\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{ \mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{\mathbb{ \mathbb{ \mathbb{   \mathbb{       \mathbb{ \mathbb{      \mathbb{ \mathbb{      \mathbb{  \mathbb{   \mathbb{    \mathbb{  \mathbb{   \mathbb{   \mathbb{  \mathbb{   \mathbb{  \mathbb{   \mathbb{   \mathbb{  \mathbb{   \mathbb{  \mathbb{   \mathbb{   \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{   \mathbb{  \mathbb{   \mathbb{  \mathbb{   \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{   \mathbb{   \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{   \mathbb{  \mathbb{   \mathbb{  \mathbb{   \mathbb{  \mathbb{  \mathbb{   \mathbb{   \mathbb{  \mathbb{   \mathbb{  \mathbb{   \mathbb{  \mathbb{   \mathbb{   \mathbb{  \mathbb{   \mathbb{  \mathbb{   \mathbb{  \mathbb{   \mathbb{   \mathbb{  \mathbb{   \mathbb{   \mathbb{  \mathbb{   \mathbb{   \mathbb{  \mathbb{   \mathbb{   \mathbb{   \mathbb{   \mathbb{  \mathbb{   \mathbb{   \mathbb{   \mathbb{   \mathbb{   \mathbb{   \mathbb{   \mathbb{   \mathbb{   \mathbb{   \mathbb{   \mathbb{   \mathbb{   \mathbb{   \mathbb{   \mathbb{   \mathbb{   \mathbb{   \mathbb{    \mathbb{  \mathbb{    \mathbb{   \mathbb{   \mathbb{    \mathbb{   \mathbb{   \mathbb{    \mathbb{   \mathbb{   \mathbb{    \mathbb{    \mathbb{  \mathbb{    \mathbb{    \mathbb{   \mathbb{   \mathbb{    \mathbb{    \mathbb{   \mathbb{    \mathbb{   \mathbb{    \mathbb{   \mathbb{    \mathbb{   \mathbb{    \mathbb{    \mathbb{    \mathbb{   \mathbb{    \mathbb{    \mathbb{    \mathbb{   \mathbb{     \mathbb{   \mathbb{    \mathbb{    \mathbb{   \mathbb{    \mathbb{    \mathbb{    \mathbb{    \mathbb{    \mathbb{    \mathbb{    \mathbb{   \mathbb{    {     \mathbb{    \mathbb{     \mathbb{    \mathbb{    \mathbb{    {     \mathbb{    \mathbb{    \mathbb{     \mathbb{   {      \mathbb{    \mathbb{    \mathbb{     \mathbb{    \mathbb{    {     \mathbb{    \mathbb{    {     \mathbb{     \mathbb{    {      \mathbb{    {       \mathbb{  \mathbb{     \mathbb{    {      \mathbb{    {       \mathbb{ {        \mathbb{  {        \mathbb{   {         \mathbb{   \mathbb{  {        \mathbb{   {        \mathbb{ {         \mathbb{ {         \mathbb{ {         \mathbb{ {          \mathbb{ {         \mathbb{ {          \mathbb{ {        \mathbb{ {          \mathbb{ {          \mathbb{ {         \mathbb{ {          \mathbb{ {         \mathbb{ {          \mathbb{{          \mathbb{ {          \mathbb{ {         \mathbb{ {          \mathbb{{          {         \mathbb{ {        {         \mathbb{ {        {          \mathbb{ {        \mathbb{ {  {             \mathbb{ {   {             \mathbb{ { {              \mathbb{{  {              \mathbb{{ {         {            \mathbb{{ {{                    \mathbb{{{{{                         \mathbb{{{{{                         \mathbb{{{{{                         {          \mathbb{ { { {    {{{{                              {\mathbb{{{{                         {\mathbb{{{{                        {        \mathbb{{{{{{                            {\mathbb{{{                     { \mathbb{{{                    { \mathbb{{{                    \mathbb{{{{                    { \mathbb{{                { \mathbb{        {        \mathbb{{       {       {      \mathbb{{         {       {     \mathbb{{          {       {    \mathbb{{{             {         \mathbb{{{            {         \mathbb{{{            {         \mathbb{{{{                {        \mathbb{{{              {      \mathbb{{{              {      \mathbb{{           {     {    \mathbb{      {       {   \mathbb{      {     {      \mathbb{     {      {     \mathbb{{        {     {      \mathbb{{         {       \mathbb{   {      {      \mathbb{    {      {     \mathbb{      {     {     \mathbb{   {       {    \mathbb{     {       \mathbb{  {     {      \mathbb{   {       {     \mathbb{   {       {    \mathbb{      {     \mathbb{    {      {    \mathbb{     {     \mathbb{   {      \mathbb{  {      {     \mathbb{     {     \mathbb{    {      \mathbb{  {      {    \mathbb{    {      \mathbb{    {     \mathbb{    {    \mathbb{     {     \mathbb{    {     \mathbb{     {  \mathbb{      {  \mathbb{       \mathbb{   \mathbb{     {   \mathbb{    \mathbb{   {    \mathbb{     \mathbb{  \mathbb{      \mathbb{  \mathbb{     \mathbb{ \mathbb{    \mathbb{  \mathbb{ \mathbb{ {
item \(i\). We define preference prediction as \(\hat{y}_{u,i}=\mathrm{MLP}(e_{u}\parallel e_{i})\), where \(\parallel\) denotes the concatenation function.

**Gradient calculation.** We take Bayesian personalized ranking (BPR) loss (Wang et al., 2017) as the loss function to learn model parameters. It assumes that the observed interactions should be assigned higher prediction values than unobserved ones. The BPR loss \(\mathcal{L}_{u}^{P}\) is calculated as,

\[\mathcal{L}_{u}^{P}=\frac{1}{|\mathbb{I}_{u}|}\sum_{i\in\mathbb{I}_{u},e_{i} \in\mathbb{I}_{u}^{+}}-\mathrm{ln}\sigma(\hat{y}_{u,i}-\hat{y}_{u,i}), \tag{2}\]

where \(\mathbb{I}_{u}^{-}\) denotes the unobserved interactions of user \(u\) and \(\sigma\) denotes the sigmoid function.

Finally, we calculate local loss \(\mathcal{L}_{u}\) for user \(u\) as,

\[\mathcal{L}_{u}=\mathcal{L}_{u}^{P}+\mathcal{L}_{u}^{d}+\lambda||\Theta_{u}|| _{2}, \tag{3}\]

where \(\lambda\) is the weight of L2 regularization and \(\Theta_{u}\) denotes trainable parameters of the recommendation model for user \(u\).

### Global gradient passing

In the third stage, we propose neighbor sampling and secure gradient-sharing based on the inter-user graph to enhance model performance and training efficiency in a privacy-preserving way.

**Neighbor sampling.** An intuitive way of decentralized recommendations is that users collaboratively train models with their neighbors. However, two problems remain unresolved, damaging recommendation performance: (1) Some item embeddings are not fully trained or even not trained, as interacted items are sparse in recommendation scenarios. (2) Different users have different convergence speeds. For example, popular users are more likely to be involved in training. Inadequately-trained models can bias fully-trained models by shared gradients. Thus, we design a sampling strategy to upgrade the neighbor-based collaborative training into the neighborhood-based one.

We describe a neighbor sampling strategy for user \(u\in\mathcal{V}\) in Algorithm 1. The probability for user \(u\) to sample his neighbor \(v\in\mathcal{N}_{u}\) is defined as,

\[p_{u,v}=\frac{\mathcal{L}_{v}/(\ln(cnt_{u}+1)+1)}{\sum_{w\in\mathcal{N}_{u}} \mathcal{L}_{w}/(\ln(cnt_{w}+1)+1)}, \tag{4}\]

where \(\mathcal{L}_{v}\) is the training loss for user \(v\) as calculated in Equation (3). We let \(cnt_{v}\) denote the training iterations for user \(v\). The sampled probability increases with local training loss and decreases with training iterations. Thus, underfit models are more likely to be sampled for training.

We employ \(U\) to denote all sampled users, i.e., \(U=\{v|size(N_{u}^{s})>0\}\). Each sampled user \(v\in U\) simultaneously calculates his local gradients \(g_{v}\) based on the loss defined in Equation (3).

**Secure gradient-sharing.** Inspired by the one-bit encoder (Han et al., 2017), we propose a novel privacy-preserving mechanism to share gradients in the neighborhood efficiently. Unlike the one-bit encoder, our mechanism shares the calculated gradients multiple times and satisfies RDP. The secure gradient-sharing involves three steps, i.e., gradient encoding, gradient propagation, and gradient decoding.

**(1) Gradient encoding.** We first encode the calculated gradients to protect user privacy and minimize communication costs. In the training neighborhood, each user clips his local gradients into \([-\delta,\delta]\) and takes the Bernoulli distribution to sample the mapped encoding for gradients. Let \(\beta\) denote the perturbation strength to protect user privacy. We define the gradient encoding for user \(u\) as,

\[g_{u}^{*}\sim 2*\mathrm{Bernoulli}(\frac{1}{e^{\beta}+1}+\frac{(e^{\beta}-1)( \mathrm{clip}(g_{u},\delta)+\delta)}{2(e^{\beta}+1)\delta})-1, \tag{5}\]

where \(g_{u}^{*}\) denotes the encoded gradients of user \(u\) with each element either equalling to \(-1\) or \(1\). Large gradients are more likely to be mapped to \(1\), whereas small gradients are more likely to be mapped to \(0\).

Theorem 1 ().: _For a gradient with a size of \(n_{s}\), our secure gradient-sharing mechanism satisfies \((\ref{eq:1},\epsilon)\)-RDP, where \(\epsilon=2n_{s}log(\frac{1.5\pi\delta(e^{\beta}+1)}{2(e^{\beta}-1)}+\frac{e^{- \epsilon\cdot\beta}\epsilon+1\beta t}{e^{\beta}+1})\)._

Proof.: Let \(\mathcal{M}(g)\) be the secure gradient-sharing as described in Equation (5). We need to show that for any two input gradients \(g\) and \(g^{\prime}\), we have \(D_{\alpha}(M(g)\parallel M(g^{\prime}))\leq\epsilon\). We set \(z=e^{\beta}+1\) to simplify the description. For two continuous distributions defined over the real interval with densities \(p\) and \(q\), we have

\[p(g)=\mathrm{P}[\mathcal{M}(g)=1]=\frac{\delta z+(z-2)g}{2\delta z},\]

\[q(g)=\mathrm{P}[\mathcal{M}(g)=-1]=\frac{\delta z-(z-2)g}{2\delta z}.\]

Given that we clip gradients \(g\) into \([-\delta,\delta]\), we evaluate \(D_{\alpha}(M(g)\parallel M(g^{\prime}))\) separately over three intervals, i.e., \((-\infty,-\delta)\), \([-\delta,\delta]\), and \((\delta,+\infty)\). Let \(n_{s}\) denote the size of the gradient. We have

\[D_{\alpha}(M(g)\parallel M(g^{\prime}))=\frac{1}{\alpha-1}\sum_{i=0}^{ns}log \int_{-\infty}^{\infty}p(g_{i})^{\alpha}q(g_{i})^{1-\alpha}\mathrm{d}g_{i}.\]

For intervals \((-\infty,-\delta)\) and \((\delta,+\infty)\), we have

\[\int_{-\infty}^{-\delta}p(g)^{\alpha}q(g)^{1-\alpha}\,\mathrm{d}g =\frac{(z-1)^{1-\alpha}}{z},\] \[\int_{\delta}^{\infty}p(g)^{\alpha}q(g)^{1-\alpha}\,\mathrm{d}g =\frac{(z-1)^{\alpha}}{z}.\]

For intervals \([-\delta,\delta]\), we set \(x=(z-2)g\) and \(y=\delta z\) and have

\[\int_{-\delta}^{\delta}p(g)^{\alpha}q(g)^{1-\alpha}\,\mathrm{d}g =\frac{1}{2\delta z(z-2)}\int_{-y=2\delta}^{y-2\delta}(x+y)^{\alpha }(-x+y)^{1-\alpha}\,\mathrm{d}x\] \[\leq\frac{1}{2\delta z(z-2)}\int_{-y}^{y}(x+y)^{\alpha}(-x+y)^{1- \alpha}\,\mathrm{d}x.\]

Given the reverse chain rule, we set \(t=(x+y)/2y\) and have

\[\int_{-\delta}^{\delta}p(g)^{\alpha}q(g)^{1-\alpha}\,\mathrm{d}g =\frac{1}{z-2}\int_{0}^{1}(2yt)^{\alpha}(-2yt+2y)^{1-\alpha}\, \mathrm{d}t\] \[=\frac{2\delta z}{z-2}\int_{0}^{1}(t)^{\alpha}(-t+1)^{1-\alpha}\, \mathrm{d}t.\]

According to the definition of beta and gamma functions, we have

\[\int_{-\delta}^{\delta}p(g)^{\alpha}q(g)^{1-\alpha}\,\mathrm{d}g =\frac{2\delta z}{z-2}\mathcal{B}(\alpha+1,2-\alpha)\] \[=\frac{2\delta z}{z-2}\frac{\Gamma(\alpha+1)\Gamma(2-\alpha)}{ \Gamma(3)}\] \[=\frac{2\delta z\alpha(\alpha-1)}{(z-2)}\frac{\Gamma(\alpha-1) \Gamma(1-(\alpha-1))}{\Gamma(3)}.\]
By the reflection formula for the gamma function, we have

\[\int_{-\delta}^{\delta}p(g)^{\alpha}q(g)^{1-\alpha}\,\mathrm{d}g=\frac{2\delta \varepsilon\alpha(\alpha-1)\pi}{(z-2)\Gamma(3)sin((\alpha-1)\pi)}.\]

To calculate \(D_{\alpha}(M(g)\parallel M(g^{\prime}))\), here we set \(\alpha=1.5\) and have

\[D_{\alpha}(M(g)\parallel M(g^{\prime}))=2n_{s}log(\frac{1.5\pi\delta(e^{\beta }+1)}{2(e^{\beta}-1)}+\frac{e^{-0.5\beta}+e^{1.5\beta}}{e^{\beta}+1})\leq\epsilon,\]

which concludes the proof of (1.5, \(\epsilon\))-RDP. 

Given \(T\) training iterations, the privacy budget of the secure gradient-sharing mechanism is bounded.

Proof.: Given \(T\) training iterations, the secure gradient-sharing satisfies (1.5, \(\epsilon T\))-RDP according to the sequential composition (Hendle and Krizhevsky, 2014). One property of RDP (Krizhevsky, 2014) is that if \(\mathcal{M}(g)\) satisfies \((\alpha,\epsilon)\)-RDP, for \(\forall\gamma>0\), \(\mathcal{M}(g)\) satisfies \((\epsilon^{\prime},\gamma)\)-DP with \(\epsilon^{\prime}=\epsilon+\frac{log(1/\gamma)}{\alpha-1}\). Thus, the secure gradient-sharing mechanism satisfies \((\epsilon T+2log(1/\gamma),\gamma)\)-DP. We can further choose large \(\gamma\) to reduce the privacy budget, which concludes that the privacy budget is bounded. 

**(2) Gradient propagation.** We devise a topology-based diffusion method named _gradient propagation_ to share the encoded gradients among the neighborhood. The gossip training protocol has been widely applied in decentralized learning (Hendle and Krizhevsky, 2014; Krizhevsky, 2014), where users exchange gradients with their 1-hop neighbors, given its robustness to device breakdown. However, the encoded gradients are highly inaccurate in our setting, as the number of averaged gradients is insufficient to reduce noise.

To solve the above issue, we extend the gossip training protocol to disseminate gradients among the neighborhood. Algorithm 2 describes the process of gradient propagation. Similar to the message-passing architecture in GNN (Ghezhi et al., 2017; Wang et al., 2018), each user in the neighborhood sends and receives gradients in parallel (line 5 and 6). Given the longest path in the training neighborhood is \(2H\), the encoded gradients are disseminated among all users after \(2H\) steps.

**(3) Gradient decoding.** Conventional DP mechanisms (Zheng et al., 2017; Wang et al., 2018; Wang et al., 2018) protect user privacy at the cost of recommendation performance as it introduces noises to model training. We propose gradient decoding to decode received gradients in a noise-free way.

```
0: Encoded gradients \(g_{u}^{*}\) for user \(u\in U\), number of sampling hops \(H\), sampling neighbors \(\mathcal{N}_{u}^{\mathrm{a}}\) for \(u\in U\)
0: Aggregated gradients \(\widetilde{g}_{u}\) for user \(u\in U\)
1:for user \(u\in U\)paralleldo
2:\(\widetilde{g}_{u}\leftarrow\{g_{u}^{*}\}\)
3:for\(h\gets 1\) to \(2H\)do
4:for user \(u\in U\)paralleldo
5: send \(\widetilde{g}_{u}\) to neighbor \(v\in\mathcal{N}_{u}^{\mathrm{a}}\)
6:\(\widetilde{g}_{u}\leftarrow\widetilde{g}_{u}\cup\widetilde{g}_{v},v\in \mathcal{N}_{u}^{\mathrm{a}}\)
7:return\(\widetilde{g}_{u}\) for user \(u\in U\)
```

**Algorithm 2** Gradient propagation

After gradient propagation, each user locally decodes the received gradients and updates their model with the decoded gradients by stochastic gradient descent (SGD) optimizer. Here, we describe the decoding process for user \(u\) as,

\[\tilde{g}_{u}=\delta(e^{\beta}+1)\mathrm{mean}(\widetilde{g}_{u})/(e^{\beta}- 1). \tag{6}\]

The secure gradient-sharing mechanism is unbiased.

Proof.: To prove the secure gradient-sharing mechanism is noise-free, we need to verify that \(E(\tilde{g})=E(g)\). Since the gradients are encoded from the Bernoulli distribution in Equation (5), we have

\[E(g^{*})=\frac{e^{\beta}-1}{e^{\beta}+1}\frac{E(g)+\delta}{\delta}-\frac{e^{ \beta}-1}{e^{\beta}+1}=\frac{(e^{\beta}-1)E(g)}{(e^{\beta}+1)\delta}.\]

Combining Equation (6), we then have

\[E(\tilde{g}) =\frac{\delta(e^{\beta}+1)}{e^{\beta}-1}E(g^{*})\] \[=\frac{\delta(e^{\beta}+1)}{e^{\beta}-1}\frac{(e^{\beta}-1)E(g)}{ (e^{\beta}+1)\delta}\] \[=E(g),\]

which concludes that the secure gradient-sharing mechanism is noise-free. 

The secure gradient-sharing mechanism can approach the same convergence speed as centralized SGD.

Proposition 2 guarantees the convergence of our proposed method, and please refer to paper (Ghezhi et al., 2017) for its detailed proof.

## 4. Analysis on Communication Cost

Our proposed method has the lowest communication cost among all competitors. Here, we provide a comparison of each method's communication cost theoretically. Note that each user samples \(n_{u}\) users from his neighbors, the training process involves \(H\)-hop users, and the gradient to be sent has a size of \(n_{s}\). (1) Our method requires clients to cooperate during model training and leverages a gradient encoder to minimize the communication cost. Each user has the worst communication cost as \(2Hn_{s}(1-n_{u}^{H})/(1-n_{u})\). (2) Federated methods involve a central server to orchestrate the training process. They have a communication cost as \(d_{r}n_{s}(1-n_{u}^{H})/(1-n_{u})\), where \(d_{r}\) is the number of bits to express a real number, e.g., (64). (3) For decentralized methods, each user has a communication cost
as \(d_{r}n_{s}(1-n_{u}^{H})/(1-n_{u})\). In summary, our method consistently beats federated and decentralized methods since hop number \(H\) is generally a small number, e.g., 3.

## 5. Experiment

This section conducts empirical studies on three public datasets to answer the following four research questions.

* **RQ1:** How does DGREC perform, compared with the state-of-the-art methods (Section 5.2)?
* **RQ2:** How do existing methods perform under diverse demands for privacy protection (Section 5.3)?
* **RQ3:** What are the effects of different components in DGREC (Section 5.4)?
* **RQ4:** What are the impacts of different parameters on DGREC (Section 5.5)?

### Experimental settings

**Datasets.** We conduct experiments on three public datasets, i.e., _Flixster_, _Book-crossing_, and _Weeplaces_. These datasets are collected with permissions and preprocessed through data anonymization. _Flixster_ is a movie site where people meet others with similar movie tastes (Wang et al., 2017), and it provides user friendships and item relationships. _Book-crossing_ contains a crawl from a book sharing community (Zhu et al., 2018). We connect users from the same area and build item relationships according to authors and publishers. To ensure the dataset quality, we retain users and items with at least 18 interactions (Wang et al., 2017). _Weeplaces_ is a website that visualizes user check-in activities in location-based social networks (Wang et al., 2017). We derive item relationships from their categories and remove users and items with less than 10 interactions. We show the statistics of these datasets after processing them in Table 1. For each dataset, we randomly select 80% of the historical interactions of each user as the training dataset and treat the rest as the test set. We randomly select 10% of interactions in the training dataset as the validation set to tune hyperparameters.

**Comparison methods.** The comparison methods involve three types of recommendation methods, i.e., centralized methods, federated methods, and decentralized methods.

**(1) Centralized methods.** We implement three centralized recommendation methods, namely **NeuMF**(Wang et al., 2017), **LightGCN**(Chen et al., 2018), and **HyperGNN**(Chen et al., 2018). NeuMF equips MLP to capture user preference on the item. LightGCN and HyperGNN adopt graph neural networks. The difference between LightGCN and HyperGNN is that LightGCN leverages pairwise relationships between users and items, whereas HyperGNN utilizes higher-order relationships.

**(2) Federated methods.** We implement two federated recommendation methods, i.e., **FedRec**(Chen et al., 2018) and **FedGNN**(Wang et al., 2019). FedRec is a federated matrix factorization (MF) method. FedGNN models high-order user-item relationships and adds Gaussian noises to protect user privacy.

**(3) Decentralized methods.** We implement two decentralized recommendation methods named **DMF**(Chen et al., 2018) and **DLGNN**. DMF is a decentralized MF method. DLGNN is a decentralized implementation of LightGCN. Each user trains his model collaboratively with his neighbors by sharing gradients in these two methods.

_To give a fair comparison, we provide side information for all methods_. (1) For MF methods, we leverage side information as regularization on embeddings, encouraging users and items to share similar embeddings with their neighbors (Wang et al., 2019). (2) For GNN methods, we leverage side information to establish user-user and item-item relationships in the graph (Wang et al., 2019).

**Evaluation metrics.** For each user in the test set, we treat all the items that a user has not interacted with as negative. Each method predicts each user's preferences on all items except the positive ones in the training set. To evaluate the effectiveness of each model, we use two widely-adopted metrics, i.e., recall@K and ndcg@K (Wang et al., 2019; Wang et al., 2019). In our experiments, we set \(K=20\).

**Parameter settings.** For all methods, we use SGD as the optimizer. We search the learning rate in \([0.001,0.005,0.01]\), the L2 regularization coefficient in \([10^{-3},10^{-2},10^{-1}]\), and the embedding size in \([16,32,48,64]\). For federated methods, we set the gradient clipping threshold \(\delta\) to \(0.1\) and privacy-preserving strength \(\epsilon\) to \(1\). For our method, we set \(\delta\) to \(0.1\) and perturbation strength \(\beta\) to \(1\), which satisfies \((1.5,0.54)\)-RDP. For GNN methods, we search the number of layers in \([1,2,3]\). For NeuMF, we search the dropout in \([0.0,0.1,0.2,0.3,0.4,0.5]\). For our method, we search the interest number \(n_{i}\) in \([6,12,18]\) and interest dimension \(d_{i}\) in \([5,10,15,30]\), respectively. Besides, we set sampling hops \(H=4\) and sampling numbers \(n_{u}=3\).

We optimize all hyperparameters carefully through grid search for all methods to give a fair comparison. We repeat the experiment with different random seeds five times and report their average results. The consumed resources vary with the methods and hyper-parameters. We simulate a decentralized environment on a single device, establishing individual recommendation models for each user on that device.

### Overall performance comparisons (RQ1)

We compare each method in conventional recommendation scenarios where all user interactions are public. We show the comparison results in Table 2. From it, we observe that: (1) Centralized methods achieve superior performance on all datasets as they can leverage all user interactions to make recommendations. For protecting user privacy, federated and decentralized methods achieve competitive performance, demonstrating the feasibility of making accurate recommendations without violating user privacy. (2) GNN methods generally outperform MF methods. FedGNN, DLGNN, and DGREC outperform the centralized MF method, i.e., NeuMF, on all datasets. This result demonstrates the superiority of GNN in recommendations, as it can vastly enhance user and item representations. (3) DGREC achieves the best recommendation performance among all privacy-preserving methods. It performs close to LightGCN and

\begin{table}
\begin{tabular}{l c c c} \hline \hline Dataset & Flixster & Book-crossing & Weeplaces \\ \hline \# Users & 3,060 & 3,060 & 8,720 \\ \hline \# Items & 3,000 & 5,240 & 7,407 \\ \hline \# Interactions & 48,369 & 222,287 & 546,781 \\ \hline Sparsity & 0.9946 & 0.9860 & 0.9915 \\ \hline \hline \end{tabular}
\end{table}
Table 1. Statistics of datasets.
HyperGNN, which are two centralized GNNs. These two results demonstrate that DGREC can protect user privacy cost-effectively.

### Generalization research (RQ2)

We evaluate how different methods meet diverse demands for privacy protection. Specifically, we set different publicized interaction ratios for each user, i.e., the proportion of publicized interactions to total interactions. (1) Centralized and Decentralized methods can only utilize publicized interactions to build graphs and train models. (2) Federated methods can only utilize publicized interactions to build the graph but all interactions to train models.

We show the comparison results in Figure 2. From it, we observe that: (1) All centralized and decentralized methods have performance degradation as publicized interaction ratio decreases. In contrast, DGREC and federated methods can keep certain recommendation performance. (2) When we set publicized interaction ratio below 1, DGREC achieves consistent superiority on all datasets, demonstrating its ability to meet diverse demands for privacy protections.

### Ablation study (RQ3)

We verify the effectiveness of different components in DGREC on _Book-Crossing_ and _Weeplaces_. (1) **w/non-item graph**, **w/non-neighbor**, and **w/non-item hypergraph** are three variants of constructing item hypergraph. w/non-item graph averages embeddings of interacted items to model user preferences. w/non-neighbor constructs the item hypergraph without neighbors' publicized interactions. w/non-item hypergraph constructs an item-tag bipartite graph for each user. (2) **w/non-attention** and **w/non-person** are two variants of modeling user preference. w/non-attention replaces the interest attention with attention-pooling (Wang et al., 2018). w/non-person removes the Pearson loss defined in Equation (1). (3) **w/non-sharing** is the variant of secure gradient-sharing, which adds zero-mean Laplacian noise to protect user privacy in place of gradient encoding and decoding.

We show the comparison results of the ablation study in Table 3. From it, we observe that: (1) All proposed components are indispensable in DGREC. Constructing an item graph and utilizing neighbors' publicized interactions are two dominant reasons for performance improvement. The former contributes to distinguishing a user's core and temporary interests, whereas the latter leverages user behavior similarity. (2) Constructing an item-tag bipartite graph cannot achieve competitive performance, especially for the datasets with the most user interactions, i.e., _Weeplaces_. Adding tag nodes hinders modeling user preference as tags are not related to the main task. (3) Utilizing interest attention and Pearson loss increases model performance, as these two methods can

\begin{table}
\begin{tabular}{c|c c|c c|c c} \hline \hline  & \multicolumn{2}{c|}{Flixster} & \multicolumn{2}{c|}{Book-Crossing} & \multicolumn{2}{c}{Weeplaces} \\  & recall & ndcg & recall & ndcg & recall & ndcg \\ \hline NeuMF & 3.28 \(\pm\) 0.24 & 2.68 \(\pm\) 0.20 & 6.70 \(\pm\) 0.28 & 3.76 \(\pm\) 0.23 & 15.85 \(\pm\) 0.26 & 10.15 \(\pm\) 0.12 \\ LightGCN & 3.47 \(\pm\) 0.41 & 3.13 \(\pm\) 0.27 & **9.94 \(\pm\) 0.23** & **5.49 \(\pm\) 0.14** & 18.78 \(\pm\) 0.10 & 12.25 \(\pm\) 0.07 \\ HyperGNN & **3.89 \(\pm\) 0.34** & **3.49 \(\pm\) 0.30** & 8.36 \(\pm\) 0.41 & 4.34 \(\pm\) 0.24 & **20.17 \(\pm\) 0.28** & **13.12 \(\pm\) 0.21** \\ \hline FedRec & 3.24 \(\pm\) 0.46 & 2.62 \(\pm\) 0.34 & 6.32 \(\pm\) 0.39 & 3.50 \(\pm\) 0.32 & 15.18 \(\pm\) 0.35 & 9.27 \(\pm\) 0.25 \\ FedGNN & 3.38 \(\pm\) 0.53 & 3.02 \(\pm\) 0.36 & 7.40 \(\pm\) 0.44 & 3.89 \(\pm\) 0.33 & 17.29 \(\pm\) 0.29 & 10.84 \(\pm\) 0.13 \\ \hline DMF & 3.15 \(\pm\) 0.20 & 2.52 \(\pm\) 0.17 & 6.56 \(\pm\) 0.36 & 3.62 \(\pm\) 0.27 & 15.46 \(\pm\) 0.22 & 9.77 \(\pm\) 0.14 \\ DLGNN & 3.30 \(\pm\) 0.37 & 2.84 \(\pm\) 0.28 & 7.73 \(\pm\) 0.31 & 4.08 \(\pm\) 0.24 & 18.50 \(\pm\) 0.26 & 11.89 \(\pm\) 0.19 \\ \hline
**DGREC** & **3.75 \(\pm\) 0.41** & **3.27 \(\pm\) 0.25** & **9.68 \(\pm\) 0.35** & **5.13 \(\pm\) 0.21** & **19.91 \(\pm\) 0.31** & **12.76 \(\pm\) 0.18** \\ \hline \hline \end{tabular}
\end{table}
Table 2. Performance comparison of DGREC with state-of-the-art methods. In one column, the bold values correspond to the methods with the best and runner-up performances.

\begin{table}
\begin{tabular}{l|c c|c c} \hline \hline  & \multicolumn{2}{c|}{Book-Crossing} & \multicolumn{2}{c}{Weeplaces} \\  & recall & ndcg & recall & ndcg \\ \hline w/non-item graph & 6.98 & 3.83 & 16.18 & 10.45 \\ w/non-neighbor & 6.77 & 3.61 & 17.07 & 10.71 \\ w/non-item hypergraph & 9.20 & 4.66 & 17.69 & 11.17 \\ \hline w/non-attention & 9.19 & 4.78 & 19.35 & 12.43 \\ w/non-pearson & 9.41 & 4.84 & 18.53 & 11.88 \\ \hline w/non-sharing & 8.17 & 4.11 & 18.46 & 11.96 \\ \hline
**DGREC** & **9.68** & **5.13** & **19.91** & **12.76** \\ \hline \hline \end{tabular}
\end{table}
Table 3. Performance on all variants of DGREC.

Figure 2. Performance with different ratios of public interactions.
mitigate damage from noisy and redundant interests. (4) Adding Laplacian noise to protect user privacy causes performance degradation, whereas secure gradient-sharing is noise-free and hence retains model performance.

### Parameter analyses (RQ4)

We first evaluate the number of recommended items \(K\) for model performance on _Weeplaces_ and depict the results in Figure 4. We can observe that the performance of all models increases with \(K\) and DGREC achieves runner-up performances for all choices of \(K\).

We then evaluate the impacts of different parameters on graph condensation, neighbor sampling, and secure gradient-sharing on _Book-Crossing_ and _Weeplaces_. We give the results of parameter analyses in Figure 3. From it, we observe that: (1) For graph condensation (number of interests \(n_{i}\) and interest dimension \(d_{i}\)), model performance increases with \(d_{i}\) but decreases with \(n_{i}\). A large interest dimension generally results in better interest representations, but redundant interests are bad for model convergence. (2) For neighbor sampling (sampling hops \(H\) and sampling numbers \(n_{u}\)), model performance increases with \(H\) and \(n_{u}\). When we set \(H\) to 1, our gradient propagation is degraded to the gossip training mechanism. Although users and their 1-hop neighbors are more likely to share similar preferences, involving 1-hop neighbors for model training cannot achieve competitive results. The result motivates us to form a more extensive neighborhood to train recommendation models. (3) For secure gradient-sharing mechanism (clip value \(\delta\) and perturbation strength \(\beta\)), our model performance increases with \(\delta\) and \(\beta\). However, setting larger \(\delta\) and \(\beta\) consumes a greater privacy budget, decreasing privacy-preserving strength. In our scenario, setting \(\delta=0.1\) and \(\beta=1\) is sufficient to achieve good results.

## 6. Conclusion

Under strict data protection rules, online platforms and large corporations are now precluded from amassing private user data to generate accurate recommendations. Despite this, the imperative need for recommender systems remains, particularly in mitigating the issue of information overload. The development and refinement of privacy-preserving recommender systems receive increasing attention.

In this paper, we propose DGREC for privacy-preserving recommendation, which achieves superior recommendation performance and provides strong privacy protection. DGREC allows users to publicize their interactions freely, which includes three stages, i.e., graph construction, local gradient calculation, and global gradient passing. In future work, we will extend our research to untrusted environments, where other clients may be semi-honest and malicious, and study cryptography techniques, e.g., zero-knowledge proof, to solve the issue. We will also investigate defence methods against poisoning attacks.

## Acknowledgment

This work is supported by National Key R&D Program of China (2022YFB4501500, 2022YFB4501504).

Figure 4. Performance with different numbers of recommended items.

Figure 3. Performance with different model parameters.