Incorporating Like-Minded Peers to Overcome Friend Data Sparsity in Session-Based Social Recommendations
Chunyan An Yunhan Li Qiang Yang Winston K. G. Seah Zhixu Li Conghao Yanga

1
.001




    



C. An et al.


mode = title]Incorporating Like-Minded Peers to Overcome Friend Data Sparsity in Session-Based Social Recommendations  


1]Chunyan An
1]Yunhan Li
2]Qiang Yang[1]
3]Winston K.G. Seah
4]Zhixu Li
1]Conghao Yang

[1]College of Computer Science, Inner Mongolia University, Hohhot, Inner Mongolia, China 
[2]College of Medicine, University of Florida, Gainesville, USA 
[3]School of Engineering and Computer Science, Victoria University of Wellington, Wellington, New Zealand 
[4]School of Computer Science, Fudan University, Shanghai, China 
[1]Corresponding author at: College of Medicine, University of Florida, Gainesville, USA.


E-mail address: ann@imu.edu.cn (C. An), 32109154@mail.imu.edu.cn
(Y. Li), qiangyang@ufl.edu (Q. Yang).  



§ ABSTRACT 
Session-based Social Recommendation (SSR) leverages social relationships within online networks to enhance the performance of Session-based Recommendation (SR). However, existing SSR algorithms often encounter the challenge of “friend data sparsity”. Moreover, significant discrepancies can exist between the purchase preferences of social network friends and those of the target user, reducing the influence of friends relative to the target user's own preferences.

To address these challenges, this paper introduces the concept of “Like-minded Peers” (LMP), representing users whose preferences align with the target user's current session based on their historical sessions. This is the first work, to our knowledge, that uses LMP to enhance the modeling of social influence in SSR. This approach not only alleviates the problem of friend data sparsity but also effectively incorporates users with similar preferences to the target user.

We propose a novel model named Transformer Encoder with Graph Attention Aggregator Recommendation (TEGAARec), which includes the TEGAA module and the GAT-based social aggregation module. The TEGAA module captures and merges both long-term and short-term interests for target users and LMP users. Concurrently, the GAT-based social aggregation module is designed to aggregate the target users' dynamic interests and social influence in a weighted manner. 

Extensive experiments on four real-world datasets demonstrate the efficacy and superiority of our proposed model and ablation studies are done to illustrate the contributions of each component in TEGAARec.


















Session-based Social Recommendation Session-based Recommendation Recommendation System Graph Neural Network


[
    [
    September 9, 2024
=====================






§ INTRODUCTION


Session-based recommendation (SR) is tasked with modeling a user's interest preferences based on historical data, such as the user's past purchases within a given timeframe, and predicting the next item of interest <cit.>. Previous studies have demonstrated that incorporating users' social relationships can enhance recommendation performance <cit.>. When social relationships are integrated into the session-based recommendation task, it becomes known as session-based social recommendation (SSR) <cit.>.

Numerous methods have been proposed to enhance the performance of SSR tasks, which can be broadly categorized into two groups, i.e.,  historical behavioral information based methods <cit.> and social/interactive information based methods <cit.>. 

The first group focuses on leveraging historical behavioral information to predict the recommendation information of the target user. Early methods included collaborative filtering techniques <cit.>, while more recent approaches focus on treating session data as sequences and modeling them using recurrent neural networks (RNNs) <cit.>. 

The second group centers on information aggregation, wherein the target user's embedded representation is learned by integrating information from both the user-user social network and the user-item interaction network <cit.>. Many existing methods adopt Graph Neural Networks (GNNs) to construct their models, aiming to capture the intricate item transitions within sessions <cit.>.

However, real-world SSR algorithms encounter several challenges. Firstly, the issue of “friend data sparsity” arises, where the recommendation data may lack sufficient information about the friends of the target user, resulting in lost interaction data in the current or historical sessions. Secondly, the preferences of social network friends may differ significantly from those of the target user, and the influence of friends may be limited compared to the target user's own preferences especially facing the problem of friend data sparsity, potentially leading to inaccurate recommendation. Last but not the least, users' interests may evolve dynamically over time, posing a considerable challenge in accurately modeling interest changes of users <cit.>.

To tackle these challenges, we propose the concept of “Like-minded Peers” (LMP) to represent users whose preferences are aligned with the target user's current session in their history sessions. We posit that users who exhibit similarity in interaction patterns with the target user, even if they are not social friends, can offer valuable insights. To the best of our knowledge, this is the first work using LMP to enhance SSR. It not only alleviates the problem of friend data sparseness but also effectively incorporates users with similar preferences to the target user.

For example, consider a scenario where a mother purchases diapers and formula. In this case, she is more likely to share interests with other users who have also bought diapers and formula in the past. This alignment is attributed to the fact that social friends may not necessarily share the same motherhood identity, thus their purchasing behavior has a lesser influence on her buying propensity.

To promote the understanding of LMP, we also provide an 
example, as shown in Fig. <ref>. Given the target user Mike in different sessions, at the moment of session a/b, his LMP is defined as the other users who have interacted with Mike for the same item in session a/b before that.

It is notable that while the concept of LMP bears similarities to user-based collaborative filtering algorithms, traditional collaborative filtering methods often suffer from data sparsity issues and limited model expressiveness <cit.>. 

In this paper, we collectively refer to the target user's LMP users and social friends as the target user's neighbours. 



In addition, considering that the interests of the users (including LMP users and social friends) are dynamic over time, mainstream approaches typically focus only on short-term interests in the current session <cit.>, or consider only the long- and short-term dynamic interests of neighbours, ignoring the long-term interests of the target user <cit.>. In this paper, we model the long-term and short-term interests of the target users as well as their neighbours (including LMP users and social friends) to better capture their dynamic interests. Particularly, we propose a novel model called Transformer Encoder with Graph Attention Aggregator Recommendation (TEGAARec), consisting of a TEGAA module and a graph attention (GAT)-based social aggregator module. TEGAA module focuses on modeling and fusing the dynamic interests of the target user and her/his neighbours. It contains a Transformer <cit.> encoder to encode the short-term interests, a long-term interests encoder to get the long-term interests and a GAT-based Dynamic Interest Aggregator to aggregate the information of items. The learned representations are fed into a GAT-based Social Aggregation module to weightedly aggregate the target users' dynamic interests and the social influence.

The main contributions of this paper are as follows:

    
  ∙ To the best of our knowledge, we are the first work to introduce the concept of “Like-minded Peers” (LMP) to alleviate the influence of “friend data sparsity” in SSR .
    
  ∙ We propose a novel TEGAARec model, which can model the dynamic interests of the target user and her/his neighbours separately through our designed TEGAA module. A GAT-based Social aggregation module is proposed to weightedly aggregate the target users' dynamic interests and the social influence.
    
  ∙ We conduct extensive experiments on four real-world datasets, which shows a large improvement compared to the state-of-the-art work at several metrics. 






§ RELATED WORK





 §.§ Session-based Recommendation

Session-based recommendation tasks involve sequential recommendation, where early approaches predominantly utilized models from the Recurrent Neural Network (RNN) family to capture temporal order <cit.>. For example, Hidasi et al. <cit.> proposed to use a variant of RNN, namely the Gated Recurrent Unit (GRU) <cit.>, to treat sessions as sequence prediction tasks for session-based recommendation. Li et al. <cit.> further improved upon this by incorporating attention mechanisms into the GRU to better capture users' sequential behaviors and personal intentions.

GNN-based methods have been introduced to address the limitations of RNN models in capturing complex item transitions within sessions. Wu et al. <cit.> proposed the SR-GNN model, which represents sessions as directed graphs and learns item transitions using Gated Graph Neural Networks (GGNN) <cit.>. Xu et al. <cit.> proposed a GC-SAN model that introduces an attention mechanism to learn global dependencies on top of SR-GNN to enhance the session representation. Li et al. <cit.> proposed a HIDE model that goes a step further by introducing a hypergraph neural network approach to the SR task, which models possible interest transitions from different perspectives, and then uses a combination of micro- and macro-approaches to learn the intent behind each clicked item.
However, the increasing complexity of GNN-based approaches has led to only marginal improvements. Zhang et al. <cit.> introduced the Attn-mixer, a multilevel attention mixer network that achieves multilevel reasoning for item transitions without relying on GNNs. Wang et al. <cit.> proposed the GCE-GNN model, which significantly enhances recommendation performance by constructing a global graph to learn item-transition information from other sessions. 

Despite their successes, these approaches often overlook the modeling of users' dynamic interests and lacks the modeling of long-term interests across sessions. In contrast, our approach integrates both long-term and short-term user interests to achieve a dynamic representation of users' interests.



 §.§ Social Recommendation

User-based collaborative filtering (CF) models <cit.> are the classical algorithm to recommend items to the target user based on user similarity by finding other users with similar preferences or purchase history to the target user and recommending items to the target user that have been liked or purchased by these similar users. They operate under the assumption that users who have interacted with similar items in the past tend to share similar preferences, making information about such users valuable for recommendation. However, they often encounter problems of data sparsity and limited model expressiveness <cit.>.



 §.§ Session-based Social Recommendation

Several existing methods tried to address friend data sparsity problem by employing repeated friend selection <cit.>.
For instance, Song et al. <cit.> proposed the DGRec model, which utilized RNN-based models to capture the dynamic interests of users and their friends, while employing graph attention networks to model the influence of friends. Building upon DGRec, Liu et al. <cit.> introduced GNNRec, an enhanced model that constructed session graphs and leveraged Gated Graph Neural Networks (GGNN) to model complex item transitions, further improving recommendation performance. 
Additionally, Chen et al. <cit.> proposed SERec, a generalized social recommendation framework that utilized heterogeneous graph neural networks to learn user-item representations, effectively capturing item transitions across sessions. Feng et al. <cit.> proposed a Hierarchical Social Similarity-guided Model with Dual-mode Attention (HMDA) for Session-based Recommendation where the social influence exerted by friends are aggregated.

Although these methods perform well, they are less effective when the user has fewer friends. To overcome this limitation, we introduce the concept of “Like-minded Peers” (LMP), which can enable the identification of users who genuinely share similar preferences with the target users.





§ PROBLEM DEFINITION


Before delving into the problem definition, we first give the basic definitions of concepts used throughout this paper and then the definitions relevant to users: Like-minded Peers (LMP), social friends, and neighbours. 

We denote G=(U,E) as the social network where U and E are the users and the social relations among users, respectively. e_(u̅,u)=1 represents the existence of a social edge between the target user u̅ and user u, otherwise e_(u̅,u)=0. Q represents the set of all sessions of users, and Q_u = {S_1^u, S_2^u,…, S_T^u } denotes the set of sessions for a user u ∈ U, where S_t^u is the t-th session of user u. We denoted S_t^u = {i_t,1^u, i_t,2^u, …, i_t,n^u} as the set of n items that the user u preferred at the t-th session. T is the number of historical sessions of user, and n is the number of items in the session. Each user's sessions are chronologically arranged.


Like-Minded Peers. Given a target user u̅ in the T+1-th session, the Like-Minded Peers (𝒳_u̅^T+1) of u̅ are the set of users ranging from the historical session interval [1,T] who have item intersections with the user u̅ in session T+1. It is formally defined below:

    𝒳_u̅^T+1   ={∀ u, u ∈ U}
          S_T+1^u̅∩⋃_t=1^TS_t^u∅,    
        u ≠u̅




  Social Friends. Given a target user u̅ in the T+1-th session, social friends (𝒵_u̅^T+1) are the set of users ranging from the historical session interval [1,T] who have connection edge in the social network G with the target user u̅ and have interaction behaviors in the historical session. It is formally defined below:

    𝒵_u̅^T+1   ={∀ u, u ∈ U}
          e_(u̅,u)=1, ⋃_t=1^TS_t^u∅,    
        u ≠u̅




   Neighbours of Users. Traditionally, neighbours are the users who directly/indirectly connect the target user in the social network. In this paper, we give a broad definition of neighbour which is the union set of LMP users 𝒳_u̅^T+1 and social friends 𝒵_u̅^T+1 for the target user u̅, denoted as:
   
    𝒩_u̅^T+1 = 𝒳_u̅^T+1∪𝒵_u̅^T+1




    Session-based Social Recommendation (SSR). Given a new session S_T+1^u̅ for target user u̅ and her/his neighbours 𝒩_u̅^T+1, the goal of SSR is to recommend a set of items from I that u̅ is likely to be interested in during next time step T+1. Here, we employ the session information coming from the dynamic interests including the historical sessions ⋃_t=1^TS_t^u̅ and the current session S_T+1^u̅ as well as the neighbour information including LMP users, i.e., 𝒳_u̅^T+1 and social friends i.e., 𝒵_u̅^T+1. 









§ PROPOSED METHOD

In this section, we present the detail of our proposed TEGAARec model. Fig. <ref> illustrates the overall framework, which first generates neighbours by randomly sampling LMP users and social friends from the historical sessions, then encodes and merges the long- and short-term interests of the target user and neighbours with our proposed TEGAA module. Next, a GAT-based social aggregation layer is designed to weightedly aggregate the target users' dynamic interests and the social influence. Finally, a prediction layer is used to calculate the similarities between the item embeddings and target user embedding for selecting the top-k items as the recommendation.





 §.§ Historical Session-based Neighbour Sampling

In this section, we introduce how to get the neighbours  𝒩_u̅^T+1, i.e., LMP users and social friends, of the target user u̅ given the historical sessions ⋃_t=1^T S_t^u̅. Particularly, for the LMP users acquisition, we first collect the items of u̅ at the current session S_T+1^u̅ and then check the users from the historical sessions ranging from the historical session interval [1, T] where users sharing the same items with u̅ are selected as the candidates. Then, we randomly choose users from candidates as the LMP users according to a predefined value called L_l. 
Note that if the size of LMP users is less than L_l, we randomly select duplicate LMP from the existing LMP set. Finally, the LMP users, i.e., 𝒳_u̅^T+1 are captured from the historical sessions. Note that our sampling strategy can be extended to the more complex one by replacing the random selection with the volume-based selection of shared items.

Similarly, for social friends, we first extract users from the social networks in the historical sessions. The users directly connected to u̅ and having interaction behaviors in the historical session are treated as the candidates. Next, the random sampling strategy is employed to select users based on the predefined value L_s 
to get 𝒵_u̅^T+1 as the social users.

We union the LMP users and social friends to acquire neighbours of the target user u̅.



 §.§ Autoregressive Mask Sequence Construction


To augment the number of training examples <cit.>, we try to split the current session sequence S_T+1^u̅ into several sub-sessions sequence MaskedInput_k and the corresponding labels Target_k. Specifically, for the sequence of interaction item sequence S_T+1^u̅ = { i_T+1,1^u̅,i_T+1,2^u̅,…,i_T+1,n^u̅}, we utilize
an Autoregressive Mask Sequence Construction module to split it as follows:

    MaskedInput_k   = {i_T+1,1^u̅, …, i_T+1,k^u̅, 0, …, 0},
    
    		Target_k   = {i_T+1,k+1^u̅}

where MaskedInput_k represents the k-th constructed masked input sequence and Target_k represents the k-th item to be predicted, k ∈ [1, n-1].

Different from existing methods which augment data in the data preprocessing step <cit.>, we implement it during the model's training phase. Our strategy can avoid the repeated sampling of neighbours in the process of historical session-based neighbour sampling.





 §.§ Long- and Short-Term Interests Encoding and Fusing with TEGAA Module
 
Generally, interests of users may vary over time, therefore, it is essential to encode the long- and short-term interests, which can systemically learn the preference of target users. Given any user u_m either from the neighbours 𝒩_u̅^T+1 or from the target user u̅, and her/his interacted item sequence S_t^u_m  = {i_t,1^u_m, i_t,2^u_m, ...,i_t,n^u_m}, we aim to learn the fused representations of long-term and short-term interests of the user u_m. 



  §.§.§ TEGAA Module

To capture the long-term and short-term interests and merge these interests of an user u_m, we propose a novel TEGAA module which consists of a Transformer Encoder, a user-embedding look-up table and a GAT-based Dynamic Interest Aggregator. 

Long- and Short-Term Interests Encoding. Given the item interaction sequence S_t^u_m ={i_t,1^u_m,i_t,2^u_m,...,i_t,n^u_m}, we first use the Transformer Encoder to get the embeddings of S_t^u_m as the short-term interests below:

    h_t^u_m = Transformer_Encoder(S_t^u_m)

where Transformer_Encoder(·) is the function to get the encoded information of the interaction sequence of the session S_t^u_m.

We employ a single vector from the user-embedded lookup table to learn the representation of the long-term interests of the user u_m below:

    emb_u_m = Embedding_user (u_m)

where Embedding_user∈ℝ^|U| × d denotes the user embedding lookup table, |U| is the number of all users, and d is the size of the embedding dimension.

Long- and Short-Term Interests Fusing. Given the encoded information of items and users, we first modify the Multi-head Attention Mechanism to obtain the Multi-head Graph Attention Mechanism (MHGAT) by adjusting the query vectors to the center node, i.e., the target user embedding, and the key vectors to the neighbour embedding. MHGAT is utilzied to capture the short-term interest of the user u_m.


Particularly, we treat the embeddings of target user as the center node and the embeddings of neighbours as the neighbouring nodes of multi-head attention. They are calculated as follows:

    Q= emb_u_mW_i^Q,K=h_t^u_mW_i^K,V=h_t^u_m


    head_i = Attention(Q, K, V)= SoftMax(QK^⊺/√(d_k))V


    h_s    = MHGAT(Q, K, V)
       = Concat(head_1; …; head_z) W^O + b^O

where head_1,…, head_z denotes the z heads of the multi-head attention mechanism, Concat denotes the vector splicing operation, 
W^O∈ ℝ^zd_k×d, W_i^Q∈ ℝ^d×d_k, W_i^K∈ ℝ^d×d_k are the mapping parameter matrices, b^O∈ ℝ^d is the bias term, ⊺ denotes the transpose operation, and d_k=d/h.

Then, we fuse the short-term interest result h_s obtained from the Dynamic Interest Aggregator with the long-term interest representation emb_u_m to obtain the final representation h_u_m below:

    h_u_m   = TensorFusion(h_s,emb_u_m)
       = ReLU(Concat[h_s; emb_u_m] W^F + b^F)

where ReLU(x) = max(0, x) is the nonlinear activation function, W^F∈ℝ^2d× d is the mapping parameter matrix, b^F∈ℝ^d is the bias term, and d is the size of the embedding dimension.



  §.§.§ Neighbour and Target User Interests Encoding

Given the designed TEGAA module, we apply it for the neighbour interests encoding and the target user interesting encoding to get the fused long-term and short-term interests. Particularly, for the neighbour interest encoding, we use the TEGAA module to get the encoded results for all |𝒩_u̅^T+1| of the neighbours in the T-th session H_n_m = {h_n_m,1, h_n_m,2, …, h_n_m,|𝒩_𝐮̅^𝐓+1|}

    h_n_m,k=TEGAA_Module(S_T^k)

where S_T^k represents the sequence of interaction items for the k-th neighbour's T-th session, h_n_m, k represents the coded representation of neighbour k in the neighbour set through the TEGAA module, 
and k ∈ [1, |𝒩_u̅^T+1|].

Similarly, for the target user interests encoding, we feed the constructed masked input sequences into the TEGAA module, and it outputs the encoded information corresponding to each masked sequence of the target user u̅. 

The encoded embeddings of n-1 sub-sessions of the target user
u̅ is denoted as H_u̅={h_u̅,1, h_u̅,2, …, h_u̅,n-1}. Each element h_u̅,k∈ H_u̅ is calculated below:

    h_u̅,k = TEGAA_Module(MaskedInput_k)

where k is the length of the session.




 §.§ GAT-based Social Aggregation Layer


Due to the difference of social influence for neighbours for  target user, it is necessary to discern their contributions.  In this section, we consider to use the MHGAT module (as mentioned in section <ref>) to weightedly aggregate the target users' dynamic interests and the social influence. On one hand, the neighbours of the target user u̅ in are encoded in session T with H_n_m = {h_n_m,1, h_n_m,2, …, h_n_m,|𝒩_𝐮̅^𝐓+1|} as neighbour nodes. On the other hand, the encoding result H_u̅={h_u̅,1, h_u̅,2, …, h_u̅,n-1} of each mask sequence in Session T+1 for the target user u̅ is used as the target aggregation node. Node types of the target aggregation node and the neighbour nodes are both users, i.e.,  homogeneous nodes, so we add the self-attention mechanism to the aggregation. The final aggregated result is obtained as H_u̅^'= {h_u̅,1^', h_u̅,2^', …, h_u̅,n-1^'}, whose element is calculated below:


    h_u̅,k^'=MHGAT(h_u̅, kW_i^Q,h_n_m^'W_i^K,h_n_m^')

where h_n_m^'=
[h_n_m,1, …, h_n_m,|𝒩_u̅^T+1}|, h_u̅,k]
, and
h_u̅,k represents the encoding result of the target user u̅ for the k-th mask sequence in session T+1, k ∈  [1, n-1]. 



 §.§ Prediction Layer

In this section, we introduce how to select the top relevant items that the target user preferred.

We first encode all the items using an item-embedded lookup table. Given any item i with the id i_v, we get its embedding below:

    h_v^i=Embedding_item (i_v)

Then, we use the SoftMax function to predict the probability of the next item as follows:

    p(Target_k|   MaskedInput_k;{S_t^n_m,n_m∈𝒩_u̅^T+1})
       =exp((h_u̅,k')^⊺ h_Target_k^i)/∑_j=1^|I|exp((h_u̅,k')^⊺ h_j^i)

where S_t^n_m represents the t-th session data of neighbour n_m, t ∈  [1, T ], and 𝒩_u̅^T+1 is the set of neighbours of target user u̅. Here, we denote h_u̅,k' as the encoding result of the k-th mask sequence of target user u̅ in Session T+1 into H_u̅, ⊺ as the transpose operation, and |I| as the total number of items.



 §.§ Model Training

We treat the SSR task as a classification task that allows the model to learn which items the user is interested in. We use the maximum likelihood estimation to train the model and  gradient descent to optimize the model:

    -∑_u∈ U∑_t=2^T∑_k=1^n-1 log   (p( Target_k| MaskedInput_k;
       {S_t<T^n_m,n_m∈𝒩_u̅^T+1}))






§ EXPERIMENTS

In this section, we describe the experimental setup including the used real-world datasets and the comparative baseline methods as well as the used evaluation metrics. We also introduce the implementation details to ensure the reproduction of our model. Finally, we provide  the quantitative analysis of the results, the ablation study and the hyper-parameter analysis.



 §.§ Experimental Setup



  §.§.§ Datasets
 We conducted experiments on the following four commonly used real-world datasets, for which descriptive statistical information is shown in Table <ref>. 










Douban music and Douban movie[http://www.douban.com]. Douban is a popular communication platform including the inforamtion of music and movie. Users can rate and communicate about music and movie. The dataset includes user's ratings of music and movie and the corresponding timestamps, as well as social network information.

Extended epinions[http://www.trustlet.org/epinions.html]. This dataset contains information about users' trust and distrust. Users usually rate other users based on the quality of their reviews of the item.

Yelp[https://www.yelp.com/dataset]. A very popular online review platform where users can review local businesses (e.g. restaurants and shops), providing content-rich social networking information. 

Following <cit.>, we partitioned a dataset by segmenting user behavior into weekly sessions as the train, validation, and test sets. Specifically, sessions within the last s weeks were retained for evaluation. For the Douban music, Douban movie, Extended Epinions, and Yelp datasets, we selected s values of 26, 26, 14, and 24, respectively. Additionally, we filtered out items that did not appear in the training set to ensure consistency across sets. We randomly partitioned the sessions utilized for the testing and evaluation, and evenly distributed them.














  §.§.§ Comparative Methods


We selected a number of representative works for comparison including:

    
  * NextItNet<cit.> – A CNN-based model with long-range modeling capabilities. It expands the convolutional layers to increase the receptive field and introduces residual networks into multiple convolutional layers.
    
  * NARM<cit.> – A RNN-based model. It better captures users' sequential behaviours and personal intentions by introducing attention mechanisms into GRUs.
    
  * STAMP<cit.> – An attention-based model. It captures both users' long-term and short-term interests using the attention mechanism.
    
  * SR-GNN<cit.> – A GNN-based model. It converts sequential problems into graph problems by constructing sequences of conversations as directed graphs and then learns the complex transitions between items through gated neural networks.
    
  * GCE-GNN<cit.> – A GNN-based model. It learns information about item-transitions from other sessions through the construction of a global graph, which are further fed into the graph encoder.
    
  * HIDE<cit.> – This method delineates potential shifts in interest from diverse perspectives by generating a hypergraph for each session. Following this, both micro and macro methodologies are employed to discern the underlying purpose of each clicked item.
    
  * Attn-Mixer<cit.> – An attention-based model. It proposes a multi-level attention hybrid network and uses multi-level user intentions to achieve multi-level reasoning for item transitions.
    
  * SERec<cit.> – A social recommendation model. It learns the user-item representations using heterogeneous graph neural networks to capture item transitions across sessions.
    
  * DGRec<cit.> – A social recommendation model. It uses RNNs and graph attention networks to model users' dynamic interests and the influence of their friends.
    
  * GESU<cit.> – A social recommendation model. It captures the rapid user interest changes using GGNN and multi-head attention mechanisms, and aggregates social influence from friends using graph attention mechanisms.
    
  * GNNRec<cit.> – A social recommendation model. It improves performance of DGRec by constructing session graphs and uses GRU to model complex transitions between items.











  §.§.§ Evaluation Metrics

We used two popular ranking-based metrics for our evaluation<cit.>: R@K(Recall@K) and N@K(NDCG@K). For R@K, we use K={10, 20}, while for N@K, we set K=20 following <cit.>.  






  §.§.§ Implementation Details

We use the Pytorch framework <cit.> to build our models, and all experiments are run on two Tesla V100 GPUs. For faster model training convergence,
we used the warm-up technique <cit.> to help the model parameters converge faster for learning rate and training epochs. Specifically, we use the grid search method to search for the following hyper-parameters: learning rate in {0.01, 0.005, 0.0001, 0.00005}, number of neighbours in {25, 50}, number of LMP in {5, 15, 25}, number of layers of TEGAA modules in {1, 3, 5}, number of warm-up learning steps in {5, 10, 20}, and the number of early stop tolerances in {10, 20}. We used a uniform batch size of 50 and the Adam <cit.> optimiser. For all baseline model experiments, we used their default hyper-parameters, and we chose the highest value between our experimental results and those reported in the original paper for other baseline models.





 §.§ Quantitative Results

Table <ref> presents a performance comparison between our proposed TEGAARec model and selected baseline models. The experimental findings reveal that, apart from the Attn-mixer model, models incorporating social influence such as DGRec and GNNRec outperform those lacking social influence. This underscores the beneficial impact of social influence on recommendation performance, particularly evident in datasets like Yelp with numerous users and dense social networks. Moreover, GCE-GNN model gains a significant advantage by leveraging global graph construction using other session information, further validating the benefits of incorporating such data in recommendation tasks.

In contrast, the Attn-mixer model, which enhances inference through multi-level user intent modeling, achieves competitive results, highlighting the advantages of employing attention mechanisms.

Our proposed TEGAARec model effectively identifies users with genuinely similar preferences to the target user by leveraging Like-minded Peers (LMP) mining in the recommendation data. Additionally, it accurately models the dynamic interests of users. The results demonstrate a substantial performance improvement compared to other state-of-the-art works.







 §.§ Ablation Studies


To assess the effectiveness of each key component in the TEGAARec model, we conducted ablation experiments from six different perspectives. These experiments were carried out on the Douban music and Yelp datasets, and the results are summarized in Table <ref>:


  * TEGAARec w/o LMP: This variant utilizes only the social relationships provided by the dataset and excludes the use of Like-minded Peers (LMP).

  * TEGAARec w/o GAL: It directly concatenates dynamic interest influences of different neighbours without employing graph attention aggregation.

  * TEGAARec w/o SF: It excludes the social friends of the target user while keeps the LMP only as the neighbours.

  * TEGAARec w/ PE: This variant incorporates position encoding into the TEGAA module.

  * TEGAARec w/o ULI: It excludes the target user’s long-term interest embedding from the model.

  * TEGAARec w/o ALI: This variant removes all user long-term interest embeddings from the model.


The experimental results demonstrate that the presence of Like-minded Peers (LMP), the Graph Attention Aggregation Layer, and the inclusion of long-term interests of target users and their neighbours are pivotal factors influencing the model's performance. Removal or replacement of any of these components, as observed in TEGAARec w/o LMP, TEGAARec w/o GAL, and TEGAARec w/o ALI, respectively, leads to a significant degradation in performance.

Interestingly, the absence of significant performance improvement in TEGAARec w/ PE suggests that users' short-term interests are more accurately captured through a collection of interaction items over a brief period, without the need for sequence and position information, which may be redundant.

Also, the removal of social friends has a small influence on the model's performance. This proves the contribution of our designed LMP to some extent especially in the case of sparse social friends.

Additionally, TEGAARec w/o ULI resulted in a slight performance degradation. This observation could be attributed to the fact that the target user's long-term interests are inherently incorporated when they act as a neighbour to other users.



 §.§ Hyper-parameter Sensitivity Analysis

During the model training phase, we employed a grid search technique to optimize the hyperparameters. However, considering the large number of hyperparameters involved, we conducted the search on a reduced subset to maintain training efficiency. To further investigate the influence of hyperparameters on model performance, we selected specific key hyperparameters for sensitivity analysis. Consistent with the ablation experiments, we also performed hyper-parameter sensitivity experiments on the Douban music and Yelp datasets.



  §.§.§ Impact of Embedding Dimension

To assess the effect of different embedding dimensions (utilized for both user and item embeddings) on model performance, we experimented with embedding dimensions ranging from 16 to 512. The results, depicted in Fig. <ref>, reveal that the model's performance improvement becomes limited when the embedding dimension exceeds 128 on both datasets, with a marginal decreasing effect thereafter.





  §.§.§ Impact of Transformer Encoder Layer

To evaluate the impact of varying Transformer Encoder layers on model performance, we conducted experiments with different layer numbers ranging from 1 to 15. The results, as illustrated in Fig. <ref>, indicate that employing different Transformer Encoder layers does not substantially influence the model's performance on both datasets. Surprisingly, our findings suggest that a single layer of Transformer Encoder is adequate for achieving effective modeling.





  §.§.§ Impact of the number of LMP users

To evaluate the impact of different LMP users numbers on model performance, we conducted experiments with the numbers ranging from 5 to 55. As depicted in Fig. <ref>, the results indicate that increasing the number of LMP users enhances the model performance when the number of samples is less than 25 on both datasets. However, when the number of LMP samples exceeds 25, a slight degradation in model performance is observed, possibly attributable to the introduction of noise from an excessive number of LMP users.





  §.§.§ Impact of the number of social friend samples

To evaluate the impact of varying social friend sampling numbers on model performance, we experimented with numbers ranging from 5 to 55. As illustrated in Fig.<ref>, the results indicate that employing different numbers of social friend samples does not significantly affect the model performance on both datasets. This observation suggests that the influence from social friends is limited especially when it occurs with the problems of social friends sparsity. This finding is further corroborated by our ablation experiments.













 §.§ Scalability of LMP

To further explore the influence of LMP, we conducted the extended experiments on the DGRec and GESU models to verify whether LMP can work well on other models on the four datasets. We denote the improved DGRec and GESU models by introducing LMP as DGRecLMP and GESULMP, respectively.

Fig. <ref> shows the performance comparison of the DGRec model before and after the introduction of LMP. The experimental results show that 
the introduction of LMP helps to improve the performance on all four datasets, especially on the Douban music and Douban movie with a 66.7% and 90% improvement for Recall@20, respectively. 

Similarly, Fig. <ref> shows the performance comparison of the GESU model before and after the introduction of LMP. The experimental results show that GESULMP achieves more significant improvements compared with the DGRec model. Moreover, on the metrics of the Douban music dataset, the performance of the GESULMP model achieves quite excellent results, with a 247.6% and 109.1% improvement for the metric Recall@20 and NDCG@20, respectively. By analyzing the model structure, both GESU and TEGAARec use the multi-head attention module, which has more powerful data modeling capabilities, thus achieving the superior results compared to the GRU network used in the DGRec model. These experimental results further verify the scalability of the LMP concept.

In addition, we compare the results of the improved DGRec model and GESU model, i.e.,  DGRecLMP and GESULMP, with our TEGAARec model on four dataset in Table <ref>. 

Although the GESULMP model achieves very competitive performance by introducing LMP, our proposed TEGAARec model still maintains a large advantage, especially in modeling large-scale data more efficiently, like the Douban movie dataset, which is an order of magnitude larger than the other three datasets.





§ CONCLUSIONS


In this paper, we introduced the concept of “Like-minded Peers” (LMP) to tackle the problem of social friend sparsity and promote the identification of users who genuinely share similar preferences with the target users. Additionally, we proposed the Transformer Encoder with Graph Attention Aggregator Recommendation (TEGAARec) model, which offers improved modeling of dynamic interests and social influences among users. Both the long-term and short-term interests for target users and LMP users are captured and merged to get the final representations of target users.

The substantial performance enhancement achieved by our model across four real-world datasets from diverse domains underscores the effectiveness and superiority of our approach.

For future work, we aim to delve deeper into two key aspects. Firstly, we plan to explore more fine-grained mining of LMP, including considerations of the relative importance of different LMP to a target user. Secondly, we intend to investigate methods for better modeling the impact of different items within a session, thereby enhancing the overall recommendation accuracy and user experience.





§ ACKNOWLEDGEMENT

The authors wish to thank the Projects of Natural Science Foundation of Inner Mongolia under Grant No.2024MS06014, Inner Mongolia Science & Technology Plan under Grant No.2021GG0164, Natural Science Foundation of China under Grant No.61962039, 62162046, 62362054, 62366038, Inner Mongolia Engineering Lab of Cloud Computing and Service Software and Inner Mongolia Engineering Lab of Big Data Analysis Technology,the Engineering Research Center of Ecological Big Data, Ministry of Education, China.



cas-model2-names













