possible 3000. Here, a recall$@100$ of 0.45 for InfVAE is quite impressive,
especially considering the absence of explicit re-share links and the noise
associated with real-world diffusion processes. We restrict our remaining
analyses to InfVAE+GCN since it consistently beats the MLP variant.

####  5.5.5 Impact of Social and Interaction Sparsity

In this section, we analyze the benefits of explicitly modeling social
homophily through VAEs, compared to the best baseline (Topo-LSTM) that only
considers local propagation structures.

Users with sparse diffusion activities. We divide users into quartiles by
their activity levels, which is the number of participating cascades per user.
We evaluate target recall$@100$ for each user $u$, defined as the fraction of
times $u$ was predicted correctly within top-$100$ ranks. In Figure 5.5(a), we
depict both recall scores and relative performance gains of InfVAE (over Topo-
LSTM) across diffusion activity quartiles.

While target recall increases with activity levels, InfVAE significantly
improves performance for inactive users (quartiles Q1-Q3). Thus, modeling
social homophily through VAEs contributes to massive gains for users with
sparse diffusion activities. Interestingly, Topo-LSTM performs comparably on
the most active users (quartile Q4), which indicates the potential of pure
sequential modeling techniques for highly active users.

Figure 5.5: Performance across user quartiles on diffusion activity level, and
seed neighbor fraction (Q1: lowest, Q4: highest). InfVAE has higher gains for
users with sparse activities and lacking direct neighbors in seed sets
(quartiles Q1-Q3).

Users that lack direct social connections in seed sets. We separate users into
quartiles by seed neighbor fraction, which is computed as the fraction of seed
users that are direct social neighbors, averaged over the training examples.
We similarly report target recall$@K$ and relative gains across quartiles
(Figure 5.5(b)).

As expected, performance increases with seed neighbor fraction. Note higher
relative gains over Topo-LSTM for users that lack direct neighbors in the seed
set (quartiles Q1-Q3). This demonstrates the ability of InfVAE to implicitly
regularize seed user representations based on higher-order social
neighborhoods captured by GCN-based autoencoders. Again, we find that local
sequential models suffice for users with large seed neighbor fractions, as
evidenced by the results of Topo-LSTM in quartile Q4.

####  5.5.6 Model Analysis

In this section, we first present an ablation study on the architectural
design choices in InfVAE, followed by a sensitivity analysis on seed set
percentage and hyper-parameters.

##### Ablation Study

We analyze model design choices including modeling social homophily via VAEs
and co-attention to capture co-variance of homophily with infuence, on Android
and Weibo datasets.

Social Homophily: We examine ways to condition the sender ${\bm{V}}_{S}$ and
receiver ${\bm{V}}_{R}$ variables on ${\bm{Z}}$:

  1. 1.

${\bm{V}}_{S}$ and ${\bm{V}}_{R}$ are identical and are conditioned on
${\bm{Z}}$ through hyper-parameter $\lambda_{s}(=\lambda_{r})$, i.e.,
${\bm{V}}_{S}={\bm{V}}_{R}\not\mathchoice{\mathrel{\hbox
to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}{\bm{Z}}$ (note that this is different from
setting $\lambda_{s}=\lambda_{r}$ without enforcing
${\bm{V}}_{S}={\bm{V}}_{R}$).

  2. 2.

${\bm{V}}_{S}$ is a free variable conditionally independent of ${\bm{Z}}$,
i.e., ${\bm{V}}_{S}\mathchoice{\mathrel{\hbox
to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}{\bm{Z}}$, which is equivalent to setting
$\lambda_{s}=0$.

  3. 3.

${\bm{V}}_{R}$ is a free variable, i.e.,
${\bm{V}}_{R}\mathchoice{\mathrel{\hbox
to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}{\bm{Z}}$, which is the inverse of (3).

  4. 4.

${\bm{V}}_{S}$ and ${\bm{V}}_{R}$ are both free variables conditionally
independent of ${\bm{Z}}$ ($\lambda_{s}=\lambda_{r}=0$), i.e.,
${\bm{V}}_{S}\mathchoice{\mathrel{\hbox
to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}{\bm{Z}},{\bm{V}}_{R}\mathchoice{\mathrel{\hbox
to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}{\bm{Z}}$.

Independent conditioning of ${\bm{V}}_{S}$ and ${\bm{V}}_{R}$ on ${\bm{Z}}$
(default) achieves best results. Enforcing ${\bm{V}}_{S}={\bm{V}}_{R}$ (row 1)
is clearly inferior, which validates the choice of differentiating user roles.
Notably, allowing ${\bm{V}}_{S}$ to be a free variable results in minor
performance degradation (row 2), while the drop is significant when
${\bm{V}}_{R}$ is independent of ${\bm{Z}}$ (row 3). As expected, setting both
${\bm{V}}_{S}$ and ${\bm{V}}_{R}$ as free variables (row 4), performs the
worst due to lack of social homophily signals.

Metric | Weibo | Android
---|---|---
MAP |  @10 |  @50 |  @100 |  @10 |  @50 |  @100
(0) Default |  0.0373 |  0.0230 |  0.0257 |  0.0601 |  0.0290 |  0.0304
(1)${\bm{V}}_{S}={\bm{V}}_{R}\not\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern 2.0mu{\displaystyle\perp}}}{\mathrel{\hbox to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox to0.0pt{$\scriptstyle\perp$\hss}\mkern 2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern 2.0mu{\scriptscriptstyle\perp}}}{\bm{Z}}$ |  0.0353 |  0.0220 |  0.0248 |  0.0558 |  0.0275 |  0.0287
(2)${\bm{V}}_{S}\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern 2.0mu{\displaystyle\perp}}}{\mathrel{\hbox to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox to0.0pt{$\scriptstyle\perp$\hss}\mkern 2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern 2.0mu{\scriptscriptstyle\perp}}}{\bm{Z}}$ |  0.0351 |  0.0213 |  0.0240 |  0.0595 |  0.0285 |  0.0301
(3)${\bm{V}}_{R}\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern 2.0mu{\displaystyle\perp}}}{\mathrel{\hbox to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox to0.0pt{$\scriptstyle\perp$\hss}\mkern 2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern 2.0mu{\scriptscriptstyle\perp}}}{\bm{Z}}$ |  0.0326 |  0.0217 |  0.0241 |  0.0567 |  0.0276 |  0.0291
(4)${\bm{V}}_{S}\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern 2.0mu{\displaystyle\perp}}}{\mathrel{\hbox to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox to0.0pt{$\scriptstyle\perp$\hss}\mkern 2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern 2.0mu{\scriptscriptstyle\perp}}}{\bm{Z}},{\bm{V}}_{R}\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern 2.0mu{\displaystyle\perp}}}{\mathrel{\hbox to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox to0.0pt{$\scriptstyle\perp$\hss}\mkern 2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern 2.0mu{\scriptscriptstyle\perp}}}{\bm{Z}}$ |  0.0313 |  0.0205 |  0.0235 |  0.0542 |  0.0274 |  0.0289
(5) Remove Coattention |  0.0307 |  0.0207 |  0.0233 |  0.0553 |  0.0270 |  0.0284
(6) Separate Attentions |  0.0293 |  0.0217 |  0.0192 |  0.0570 |  0.0277 |  0.0291
(7) Static-Pretrain |  0.0342 |  0.0203 |  0.0226 |  0.0606 |  0.0281 |  0.0292
Table 5.5: Ablation study on architecture design ($MAP@K$ scores for
$K=10,50,100$), $\mathchoice{\mathrel{\hbox
to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern 2.0mu{\scriptscriptstyle\perp}}}$
denotes variable independence

Co-attention: We conduct two ablation studies defined by:

  1. 5.

Replace co-attention with meanpool over concatenated sender and temporal
influence vectors, followed by a dense layer.

  2. 6.

Replace co-attention with two separate attentions on the sender and temporal
influence sequences, followed by concatenation.

Learning co-attentional weights (default) consistently outperforms mean
pooling (5), illustrating the benefits of assigning variable contributions to
seed users. Using separate attentions (6) significantly deteriorates results,
which indicates the existence of complex non-linear correlations between the
social and temporal latent factors.

Joint Training: In (2), we replace joint block-coordinate optimization (Alg
3.1) with a single step over cascades with pre-trained user embeddings (line
2), i.e., ${\bm{Z}}$ is not updated based on cascades.

Joint training is beneficial when social interactions are noisy (e.g., Weibo)
in comparison to focused stack-exchanges such as Android.

Figure 5.6: Impact of seed set percentage in Weibo. InfVAE achieves higher
gains for larger seed set fractions.

##### Impact of Seed Set Percentage.

We divide the test set into quartiles based on seed set percentage, and report
performance per quartile. Since we require a sizable number of test examples
per quartile to obtain unbiased estimates, we use the Weibo dataset.

Figure 5.6 depicts Recall$@10$ scores in different ranges. First, recall
scores increase with seed set percentage since larger seed sets enable better
model predictions; and target set size reduces with increase in seed set
percentage. Second, relative gains of InfVAE over baselines increase with seed
set percentage. This highlights the capability of co-attention in focusing on
relevant users based on both social homophily and temporal influence factors.

##### Impact of $\lambda_{s}$ and $\lambda_{r}$

Hyper-parameters $\lambda_{s}$ and $\lambda_{r}$ control the degree of
dependence of the sender and receiver variables ${\bm{V}}_{S},{\bm{V}}_{R}$ on
the social variables ${\bm{Z}}$. Figure 5.7 depicts performance ($MAP@10$) on
Android and Weibo datasets. The performance is sensitive to variations in
$\lambda_{r}$ with best values around 0.01 and 0.1, while $\lambda_{s}$
results in minimal variations. Furthermore, the best values of
$\lambda_{s},\lambda_{r}$ are stable in a broad range of values that transfer
across datasets, indicating that InfVAE requires minimal tuning in practice.
Since $\lambda_{p}$ has minimal performance impact, we exclude it from our
analysis.

Figure 5.7: $MAP@10$ on varying $\lambda_{s}$, $\lambda_{r}$ over Android and
Weibo. Performance is more sensitive to variations in $\lambda_{r}$ than
$\lambda_{s}$.

(a) Runtime on Weibo and Android

(b) Scalability on synthetic dataset

Figure 5.8: Running time and scalability comparison of InfVAE with several
baselines. InfVAE is faster than recurrent models (Topo-LSTM, CYANRNN) by an
order of magnitude.

#####  Runtime Analysis.

In our empirical runtime analysis experiments, all diffusion prediction
methods converge within 50 epochs with similar convergence rates. Thus, for
the sake of brevity, we only compare model runtime per training epoch, which
includes one optimization step over the social network and diffusion cascades
for our model InfVAE.

From figure 5.8(a), Inf2vec is the fastest while our model InfVAE comes
second. Thus, InfVAE achieves a good trade-off between expensive recurrent
neural network models (e.g., Topo-LSTM) and simpler embedding methods (e.g.,
Inf2vec), while achieving consistently superior performance over all of them.

#####  Scalability Analysis.

We analyze scalability on cascade sequences of varying lengths. Since real-
world datasets possess heavily biased length distributions, we instead
synthetically generate a Barabasi-Albert [1] network of 2000 users and
simulate diffusion cascades using an IC model. We compare training times per
epoch for each cascade length ($l$) in the range of 10 to 50.

Figure 5.8(b) depicts linear scaling for InfVAE and Inf2vec wrt cascade
length. Recurrent methods scale poorly due to the sequential nature of back-
propagation through time (BPTT), resulting in prohibitive costs for long
cascade sequences. On the other hand, InfVAE avoids BPTT through efficient
parallelizable co-attentions.

###  5.6 Acknowledgements

Research was sponsored in part by U.S. Army Research Lab. under Cooperative
Agreement No. W911NF-09-2-0053 (NSCTA), DARPA under Agreements No.
W911NF-17-C-0099 and FA8750-19-2-1004, National Science Foundation IIS
16-18481, IIS 17-04532, and IIS-17-41317, and DTRA HDTRA11810026.

###  5.7 Conclusion

In this chapter, we present a novel variational autoencoder framework (InfVAE)
to jointly embed homophily and influence in diffusion prediction. Given a
sequence of seed user activations, InfVAE employs an expressive co-attentive
fusion mechanism to jointly attend over their social and temporal variables,
capturing complex correlations. Our experimental results on two social
networks and three stack-exchanges indicate significant gains over state-of-
the-art methods.

In future, InfVAE can be extended to include multi-faceted user attributes
owing to the generalizable nature of our VAE framework. While the current
implementation employs GCN networks, we foresee direct extensions with
neighborhood sampling [41] to enable scalability to social networks with
millions of users. We also plan to explore neural point processes to predict
user activation times. Finally, similar frameworks may be examined for joint
temporal co-evolution of social network and diffusion cascades.

In this chapter, we have examined our first inductive learning application
that models the spread (or diffusion) of user-generated content in social
networking platforms. Here, our proposed framework effectively overcomes
interaction sparsity challenges for the vast population of users who seldom
post content (sparse diffusion actions). In the next chapter, we explore a new
social interaction setting in modern social platforms, involving group
interactions; we focus on personalized item recommendations for ephemeral
groups with limited or no historical interactions together.

##  Chapter 6 GroupIM: Self-supervised Ephemeral Group Interaction Modeling

###  6.1 Introduction

We address the problem of recommending items to ephemeral groups, which
comprise users who purchase very few (or no) items together [231]. The problem
is ubiquitous, and appears in a variety of familiar contexts, e.g., dining
with strangers, watching movies with new friends, and attending social events.
We illustrate key challenges with an example: Alice (who loves Mexican food)
is taking a visitor Bob (who loves Italian food) to lunch along with her
colleagues, where will they go to lunch? There are three things to note here:
first, the group is ephemeral, since there is no historical interaction
observed for this group. Second, individual preferences may depend on other
group members. In this case, the group may go to a fine-dining Italian
restaurant. However, when Alice is with other friends, they may go to Mexican
restaurants. Third, groups comprise users with diverse individual preferences,
and thus the group recommender needs to be cognizant of individual
preferences.

Prior work primarily target persistent groups which refer to fixed, stable
groups where members have interacted with numerous items together as a group
(e.g., families watching movies). They mainly fall into two categories:
heuristic pre-defined aggregation (e.g., least misery [122]) that disregards
group interactions; data-driven strategies such as probabilistic models [123,
129] and neural preference aggregators [131, 132]. A key weakness is that
these methods either ignore individual user activities [132, 133] or assume
that users have the same likelihood to follow individual and collective
preferences, across different groups [123, 129, 131]. Lack of expressive power
to distinguish the role of individual preferences across groups results in
degenerate solutions for sparse ephemeral groups. A few related methods
exploit external side information in the form of a social network [232, 233,
234], user personality traits [235] and demographics [236], for group decision
making. However, such auxiliary side information may often be unavailable.

We train robust ephemeral group recommenders without resorting to any extra
side information. Two observations help: first, while groups are ephemeral,
group members may have rich individual interaction histories; this can
alleviate group interaction sparsity. Second, since groups are ephemeral with
sparse training interactions, base group recommenders need reliable guidance
to learn informative (non-degenerate) group representations, but the guidance
needs to be data-driven and learnable, rather than a heuristic.

Present Work: To overcome group interaction sparsity, we propose two key
insights: First, we propose a self-supervised learning approach to utilize the
intrinsic structure in observed group interactions for model training; we
exploit the preference covariance amongst individuals who are in the same
group, to design an auxiliary training objective that enhances the
informativeness of user and group representations. Second, we learn robust
estimates of the contextual relevance of users’ individual preferences in each
group and present a novel contextual regularization objective to incorporate
knowledge of individual preferences.

We realize the above insights through two novel data-driven model training
strategies for group recommendation. First, we introduce a self-supervised
learning objective to contrastively regularize the user-group latent space to
capture social user associations and distinctions across groups. We achieve
this by maximizing mutual information (MI) between representations of groups
and group members, which encourages group representations to encode shared
group member preferences while regularizing user representations to capture
their social associations. Second, we contextually identify informative group
members and regularize the corresponding group representation to reflect their
personal preferences. We introduce a novel regularization objective that
contextually weights users’ personal preferences in each group, in proportion
to their user-group MI. Group-adaptive preference weighting precludes
degenerate solutions that arise during static regularization over ephemeral
groups with sparse activities. We summarize our key contributions below:

Architecture-agnostic Group Recommendation Framework: To the best of our
knowledge, Group Information Maximization (GroupIM) is the first recommender
architecture-agnostic framework for group recommendation. Unlike prior work
[132, 131] that design customized group preference aggregators, GroupIM can
integrate arbitrary neural preference encoders and aggregators. We show state-
of-the-art results with simple efficient aggregators (such as meanpool) that
are regularized within our framework. The effectiveness of meanpool signifies
substantially reduced inference costs without loss in model expressive power.
Thus, GroupIM facilitates straightforward enhancements to base neural group
recommenders.

Group-adaptive Preference Learning and Prioritization: We learn robust
estimates of group-specific member relevance. In contrast, prior work
incorporate personal preferences through static regularization [123, 129,
131]. We use Mutual Information for self-supervised learning over user and
group representations to capture preference covariance across individuals in
the same group; and prioritize the preferences of highly relevant members
through group-adaptive preference weighting; thus effectively overcoming group
interaction sparsity in ephemeral groups. An ablation study confirms the
superiority of our MI based learning strategies over static alternatives.

Robust Experimental Results: Our experimental results indicate significant
performance gains for GroupIM over prior state-of-the-art group recommendation
models on four publicly available datasets (relative gains of 31-62% NDCG@20
and 3-28% Recall@20). Significantly, GroupIM achieves stronger gains for
groups of larger sizes, and groups with diverse member preferences.

We organize the rest of the chapter as follows. In Section 6.3, we formally
define the problem, introduce a base group recommender unifying existing
neural methods, and discuss its limitations. We describe our proposed
framework GroupIM in Section 6.4, present experimental results in Section 6.5,
and finally conclude in Section 6.6.

###  6.2 Related Work

Group Recommendation: This line of work can be divided into two broad
categories based on group types: persistent and ephemeral. Persistent groups
have stable members with rich activity history together, while ephemeral
groups comprise users who interact with very few items together [231]. A
common approach is to consider persistent groups as virtual users [121]; thus,
personalized user-level recommenders can be directly applied. However, such
methods cannot handle ephemeral groups with sparse item interactions. We focus
on the more challenging scenario—recommendations to ephemeral groups.

Prior work either aggregate recommendation results (or item scores) for each
member, or aggregate individual member preferences, towards group predictions.
They fall into two classes: score (or late) aggregation [122] and preference
(or early) aggregation [123].

Popular score aggregation strategies include least misery [122], average
[124], maximum satisfaction [125], and relevance and disagreement [126].
However, these are hand-crafted heuristics that overlook real-world group
interactions. An empirical comparison of different heuristic strategies [122]
has demonstrated the absence of a clear winner, especially with the variance
in group sizes and coherence levels.

Early preference aggregation strategies [128] generate recommendations by
constructing a group profile that combines the profiles (raw item histories)
of group members. Recent methods adopt a model-based perspective to learn
data-driven models. Probabilistic methods [237, 123, 129] model the group
generative process by considering both the personal preferences and relative
influence of members, to differentiate their contributions towards group
decisions. However, a key weakness is their assumption that users have the
same likelihood to follow individual and collective preferences, across
different groups. Neural methods explore attention mechanisms [130] to learn
data-driven preference aggregators [131, 132, 133]. MoSAN [132] models group
interactions via sub-attention networks; however, MoSAN operates on persistent
groups while ignoring users’ personal activities. AGREE [131] employs
attentional networks for joint training over individual and group
interactions; yet, the extent of regularization applied on each user (based on
personal activities) is the same across groups, which results in degenerate
solutions when applied to ephemeral groups.

An alternative approach to tackle interaction sparsity is to exploit external
side information, e.g., social network of users [234, 233, 179, 106],
personality traits [235], demographics [236], and interpersonal relationships
[232, 238]. In contrast, our setting is conservative and does not include
extra side information: we know only user and item ids, and item implicit
feedback. We address interaction sparsity through novel data-driven
regularization and training strategies [9]. Our goal is to enable a wide
spectrum of neural group recommenders to seamlessly integrate suitable
preference encoders and aggregators.

Mutual Information: Recent neural MI estimation methods [164] leverage the
InfoMax [239] principle for self-supervised representation learning. They
exploit the intrinsic structure of the input data (e.g., spatial locality in
images, community structure in graphs) via MI maximization objectives to
improve representational quality. Recent advances employ auto-regressive
models [145] and aggregation functions [144, 44, 240] with noise-contrastive
loss functions to preserve MI between structurally related inputs.

We leverage the InfoMax principle to exploit the preference covariance
structure shared amongst group members. A key novelty of our approach is MI-
guided weighting to regularize group embeddings with the personal preferences
of highly relevant members.

###  6.3 Preliminaries

In this section, we first formally define the ephemeral group recommendation
problem. Then, we present a base neural group recommender ${\bm{R}}$ that
unifies existing neural methods into a general framework. Finally, we analyze
the key shortcomings of ${\bm{R}}$ to discuss motivations for propose new
model learning strategies for ephemeral group recommendation.

####  6.3.1 Problem Definition

We consider the implicit feedback setting (only visits or clicks, no explicit
ratings) with a user set ${\mathcal{U}}$, an item set ${\mathcal{I}}$, a group
set ${\mathcal{G}}$, a binary $|{\mathcal{U}}|\times|{\mathcal{I}}|$ user-item
interaction matrix ${\bm{X}}_{U}$, and a binary
$|{\mathcal{G}}|\times|{\mathcal{I}}|$ group-item interaction matrix
${\bm{X}}_{G}$. We denote ${\bm{x}}_{u}$, ${\bm{x}}_{g}$ as the corresponding
rows for user $u$ and group $g$ in user-item ${\bm{X}}_{U}$ and group-item
${\bm{X}}_{G}$ interaction matrices, with $|{\bm{x}}_{u}|$, $|{\bm{x}}_{g}|$
indicating their respective number of interacted items.

######  Definition 6.1 (Ephemeral Group).

An ephemeral group $g\in{\mathcal{G}}$ comprises a set of $|g|$ users
$u^{g}=\\{u^{g}_{1},\dots,u^{g}_{|g|}\\}\subset{\mathcal{U}}$ with sparse
historical interactions ${\bm{x}}_{g}$.

######  Definition 6.2 (Ephemeral Group Recommendation).

We evaluate group recommendation on strict ephemeral groups, which have never
interacted together during training. Given a strict ephemeral group $g$ during
testing, our goal is to generate a ranked list over the item set
${\mathcal{I}}$ relevant to users in $u^{g}$, i.e., learn a function
$f_{G}:P({\mathcal{U}})\times{\mathcal{I}}\mapsto{\mathbb{R}}$ that maps an
ephemeral group and an item to a relevance score, where $P({\mathcal{U}})$ is
the power set of ${\mathcal{U}}$.

Figure 6.1: Neural architecture diagram of a base neural group recommender
with user preference encoder, group preference aggregator and joint user and
group losses.

####  6.3.2 Base Neural Group Recommender

Several neural group recommendation models have achieved impressive results
[131, 132]. Despite their diversity in modeling group interactions, we remark
that state-of-the-art neural methods share a clear model structure: we present
a base group recommender ${\bm{R}}$ that includes three modules: a preference
encoder; a preference aggregator; and a joint user and group interaction loss.
Unifying these neural group recommenders within a single framework facilitates
deeper analysis into their shortcomings in addressing ephemeral groups.

The base neural group recommender ${\bm{R}}$ (Figure 6.1) first computes user
representations ${\bm{E}}\in{\mathbb{R}}^{|{\mathcal{U}}|\times D}$ from
individual user-item interactions ${\bm{X}}_{U}$ using a preference encoder
$f_{\textsc{enc}}(\cdot)$, followed by applying a neural preference aggregator
$f_{\textsc{agg}}(\cdot)$ to compute the group representation
${\bm{e}}_{g}\in{\mathbb{R}}^{D}$ for group $g$. Finally, the latent group
representation ${\bm{e}}_{g}$ is jointly trained over the group-item
${\bm{X}}_{G}$ and user-item ${\bm{X}}_{U}$ interactions, to make item
recommendations for group $g$.

##### User Preference Representations.

User embeddings ${\bm{E}}$ constitute a latent representation of their
personal preferences, indicated in the interaction matrix ${\bm{X}}_{U}$.
Since latent-factor collaborative filtering methods adopt a variety of
strategies (such as matrix factorization, autoencoders, etc.) to learn user
embeddings ${\bm{E}}$, we define the preference encoder
$f_{\textsc{enc}}:|{\mathcal{U}}|\times{\mathbb{Z}}_{2}^{|{\mathcal{I}}|}\mapsto{\mathbb{R}}^{D}$
with two inputs: user $u$ and associated binary personal preference vector
${\bm{x}}_{u}$, defined by:

${\bm{e}}_{u}=f_{\textsc{enc}}(u,{\bm{x}}_{u})\;\forall u\in{\mathcal{U}}$
(6.1)

We can augment the user representation ${\bm{e}}_{u}$ with additional inputs,
including user/item contextual attributes, item relationships, etc. via
customized encoders [241].

##### Group Preference Aggregation.

A preference aggregator models the interactions among group members to compute
an aggregate representation ${\bm{e}}_{g}\in{\mathbb{R}}^{D}$ for ephemeral
group $g\in{\mathcal{G}}$. Since groups are sets of users with no inherent
ordering, we consider the class of permutation-invariant functions (such as
summation or pooling operations) on sets [242]. Specifically,
$f_{\textsc{agg}}(\cdot)$ is permutation-invariant to the order of group
member embeddings $\\{e_{u_{1}},\dots,e_{u_{|g|}}\\}$. We compute the group
representation ${\bm{e}}_{g}$ for group $g$ using an arbitrary preference
aggregator $f_{\textsc{agg}}(\cdot)$ as:

${\bm{e}}_{g}=f_{\textsc{agg}}(\\{{\bm{e}}_{u}:u\in u^{g}\\})\;\forall
g\in{\mathcal{G}}$ (6.2)

##### Joint User and Group Loss.

The group representation ${\bm{e}}_{g}$ is trained over the group-item
interactions ${\bm{X}}_{G}$ with group-loss $L_{G}$. The framework supports
different recommendation objectives, including pairwise [87] and pointwise
[88] ranking losses. Here, we use a multinomial likelihood formulation owing
to its impressive results in user-based neural collaborative filtering [90].
The group representation ${\bm{e}}_{g}$ is transformed by a fully connected
layer and normalized by a softmax function to produce a probability vector
$\pi({\bm{e}}_{g})$ over ${\mathcal{I}}$. The group-interaction loss measures
the KL-divergence between the normalized purchase history
${\bm{x}}_{g}/|x_{g}|$ (${\bm{x}}_{g}$ indicates items interacted by group
$g$) and predicted item probabilities $\pi({\bm{e}}_{g})$, given by:

$L_{G}=-\sum\limits_{g\in{\mathcal{G}}}\frac{1}{|{\bm{x}}_{g}|}\sum\limits_{i\in{\mathcal{I}}}x_{gi}\log\pi_{i}({\bm{e}}_{g});\hskip
5.0pt\pi({\bm{e}}_{g})=\text{softmax}({\bm{W}}_{I}{\bm{e}}_{g})$ (6.3)

Next, we define the user-loss $L_{U}$ that regularizes the user
representations ${\bm{E}}$ with user-item interactions ${\bm{X}}_{U}$, thus
facilitating joint training with shared encoder $f_{\textsc{enc}}(\cdot)$ and
predictor (${\bm{W}}_{I}$) layers [131]. We use a similar multinomial
likelihood-based formation, given by:

$L_{U}=-\sum\limits_{u\in{\mathcal{U}}}\frac{1}{|{\bm{x}}_{u}|}\sum\limits_{i\in{\mathcal{I}}}x_{ui}\log\pi_{i}({\bm{e}}_{u});\hskip
5.0ptL_{R}=L_{G}+\lambda L_{U}$ (6.4)

where $L_{R}$ denotes the overall loss of the base recommender ${\bm{R}}$ with
balancing hyper-parameter $\lambda$. Prior work AGREE [131] trains an
attentional group preference aggregator with a pairwise regression loss over
both ${\bm{X}}_{U}$ and ${\bm{X}}_{G}$, while MoSAN [132] trains a collection
of sub-attentional aggregators with bayesian personalized ranking [87] loss on
just ${\bm{X}}_{G}$. Thus, state-of-the-art neural methods AGREE [131] and
MoSAN [132] are specific instances of the framework described by the base
neural group recommender ${\bm{R}}$.

####  6.3.3 Motivation

To address ephemeral groups, we focus on learning strategies that are
independent of the architectural choices in the base recommender ${\bm{R}}$.
With the rapid advances in neural methods, we envision future enhancements in
neural architectures for user representations and group preference
aggregation. Since ephemeral groups by definition purchase very few items
together, base group recommenders suffer from inadequate training data in
group interactions. Here, the group embedding ${\bm{e}}_{g}$ receives back-
propagation signals from sparse interacted items in ${\bm{x}}_{g}$, thus
lacking evidence to reliably estimate the role of each member. To address
group interaction sparsity towards robust ephemeral group recommendation, we
propose two data-driven model learning strategies that are independent of the
base recommendation mechanisms to generate individual and group
representations.

##### Self-supervised Representation Learning

We note that users’ preferences are group-dependent; and users occurring
together in groups typically exhibit covarying preferences (e.g., shared
cuisine tastes). Thus, group interactions reveal distinctions across groups
(e.g., close friends versus colleagues) and latent user associations (e.g.,
co-occurrence of users in similar groups), that are not directly evident when
the base group recommender ${\bm{R}}$ only predicts sparse group interactions.

We contrast the preference representations of group members against those of
non-member users with similar item histories, to effectively regularize the
latent space of user and group representations. This promotes the
representations to encode latent discriminative characteristics shared by
group members, that are not discernible from their sparse interactions.

Symbol | Description
---|---
${\bm{X}}_{U}$ | Binary $|{\mathcal{U}}|\times|{\mathcal{I}}|$ user-item interaction matrix
${\bm{X}}_{G}$ | Binary $|{\mathcal{G}}|\times|{\mathcal{I}}|$ group-item interaction matrix
${\bm{x}}_{u}$ |  $|{\mathcal{I}}|$-dimensional row for user $u$ in matrix ${\bm{X}}_{U}$
${\bm{x}}_{g}$ |  $|{\mathcal{I}}|$-dimensional row for group $g$ in matrix ${\bm{X}}_{G}$
${\bm{R}}$ | Base neural group recommender
$f_{\textsc{enc}}$ | Individual user preference encoder
$f_{\textsc{agg}}$ | Group preference aggregator
${\mathbf{e}}_{u}$ | Latent user preference representation for user $u\in{\mathcal{U}}$
${\mathbf{e}}_{g}$ | Latent group preference representation for group $g\in{\mathcal{G}}$
Table 6.1: Notation

##### Group-adaptive Preference Prioritization.

To overcome group interaction sparsity, we critically remark that while groups
are ephemeral with sparse interactions, the group members have comparatively
richer individual interaction histories. Thus, we propose to selectively
exploit the personal preferences of group members to enhance the quality of
group representations via contextual regularization.

The user-loss $L_{U}$ (equation 6.4) in base recommender ${\bm{R}}$ attempts
to regularize user embeddings ${\bm{E}}$ based on their individual activities
${\bm{X}}_{U}$. A key weakness is that $L_{U}$ forces ${\bm{e}}_{u}$ to
uniformly predict preferences ${\bm{x}}_{u}$ across all groups containing user
$u$. Since groups interact with items differently than individual members,
inaccurately utilizing ${\bm{X}}_{U}$ can become counter-productive. Fixed
regularization results in degenerate models that either over-fit or are over-
regularized, due to lack of flexibility in adapting preferences per group.

To overcome group interaction sparsity, we contextually identify members that
are highly relevant to the group and regularize the group representation to
reflect their personal preferences. To measure contextual relevance, we
introduce group-specific relevance weights $w(u,g)$ for each user $u$ where
$w(\cdot)$ is a learned weighting function of both the user and group
representations. This enhances the expressive power of the group recommender,
thus effectively alleviating the challenges imposed by group interaction
sparsity.

In this section, we defined ephemeral group recommendation, and presented a
base group recommender architecture with three modules: user representations,
group preference aggregation, and joint loss functions. Finally, we motivated
the need to: contrastively regularize the user-group space to capture member
associations and group distinctions; and learn group-specific weights $w(u,g)$
to regularize group representations with individual preferences.

###  6.4 GroupIM Framework

We first motivate mutual information towards achieving our two proposed model
learning strategies, followed by a detailed description of our proposed
framework GroupIM.

####  6.4.1 Mutual Information Maximization.

We introduce our user-group mutual information maximization approach through a
stylized example. We extend the introductory example to illustrate how to
regularize Alice’s latent representation based on her interactions in two
different groups. Consider Alice who first goes out for lunch to an Italian
restaurant with a visitor Bob, and later dines at a Mexican restaurant with
her friend Charlie.

First, Alice plays different roles across the two groups (i.e., stronger
influence among friends than with Bob) due to the differences in group context
(visitors versus friends). Thus, we require a measure to quantify the
contextual informativeness of user $u$ in group $g$.

Second, we require the latent representation of Alice to capture associations
with both visitor Bob and friend Charlie, yet express variations in her group
activities. Thus, it is necessary to not only differentiate the role of Alice
across groups, but also compute appropriate representations that make her
presence in each group more coherent.

Figure 6.2: Neural architecture diagram of GroupIM depicting the different
model components and loss terms appearing in Equation 6.13.

To achieve these two goals at once, we maximize user-group mutual information
(MI) to regularize the latent space of user and group representations, and set
group-specific relevance weights $w(u,g)$ in proportion to their estimated MI
scores. User-group MI measures the contextual informativeness of a member $u$
towards the group decision through the reduction in group decision uncertainty
when user $u$ is included in group $g$. Unlike correlation measures that
quantify monotonic linear associations, mutual information captures complex
non-linear statistical relationships between covarying random variables. Our
proposed user-group MI maximization strategy enables us to achieve our two-
fold motivation (Section 6.3.3):

  * •

Group Preference Co-variance Learning: Maximizing user-group MI encourages the
group embedding ${\bm{e}}_{g}$ to encode preference covariance across group
members, and regularizes the user embeddings ${\bm{E}}$ to capture social
associations in group interactions.

  * •

Group-specific User Relevance: By quantifying $w(u,g)$ through user-group
mutual information, we accurately capture the extent of informativeness for
user $u$ in group $g$, thus guiding group-adaptive personal preference
prioritization for regularization.

####  6.4.2 User-Group MI Maximization.

Neural MI estimation [164] has demonstrated feasibility to maximize MI by
training a classifier ${\bm{D}}$ (a.k.a, discriminator network) to accurately
separate positive samples drawn from their joint distribution from negative
samples drawn from the product of marginals.

We maximize user-group MI between group member representations
$\\{{\bm{e}}_{u}:u\in u^{g}\\}$ and the group representation ${\bm{e}}_{g}$
(computed in equations 6.1 and 6.2 respectively). We train a contrastive
discriminator network
${\bm{D}}:{\mathbb{R}}^{D}\times{\mathbb{R}}^{D}\mapsto{\mathbb{R}}^{+}$,
where ${\bm{D}}({\bm{e}}_{u},{\bm{e}}_{g})$ represents the probability score
assigned to this user-group pair (higher scores for users who are members of
group $g$). The positive samples $({\bm{e}}_{u},{\bm{e}}_{g})$ for ${\bm{D}}$
are the preference representations of $(u,g)$ pairs such that $u\in u^{g}$,
and negative samples are derived by pairing ${\bm{e}}_{g}$ with the
representations of non-member users sampled from a negative sampling
distribution $P_{{\mathcal{N}}}(u|g)$. The discriminator ${\bm{D}}$ is trained
on a noise-contrastive type objective with a binary cross-entropy (BCE) loss
between samples from the joint (positive pairs), and the product of marginals
(negative pairs), resulting in the following training objective:

$\displaystyle
L_{MI}=-\frac{1}{|{\mathcal{G}}|}\sum\limits_{g\in{\mathcal{G}}}\frac{1}{\alpha_{g}}\Big{[}$
$\displaystyle\sum\limits_{u\in
u^{g}}\log{\bm{D}}_{ug}+\sum\limits_{j=1}^{M_{g}}\mathbb{E}_{\tilde{u}\sim
P_{{\mathcal{N}}}}\log(1-{\bm{D}}_{\tilde{u}g})\Big{]}$ (6.5)

where $\alpha_{g}=|g|+M_{g}$, $M_{g}$ is the number of negative users sampled
for group $g$ and ${\bm{D}}_{ug}$ is a shorthand for
${\bm{D}}({\bm{e}}_{u},{\bm{e}}_{g})$. This objective maximizes MI between
${\bm{e}}_{u}$ and ${\bm{e}}_{g}$ based on the Jensen-Shannon divergence
between the joint and the product of marginals [44].

We employ a preference-biased negative sampling distribution
$P_{{\mathcal{N}}}(\tilde{u}|g)$, which assigns higher likelihoods to non-
member users who have purchased the group items ${\bm{x}}_{g}$. These hard
negative examples encourage the discriminator to learn latent aspects shared
by group members by contrasting against other users with similar individual
item histories. We define the negative sampling distribution
$P_{{\mathcal{N}}}(\tilde{u}|g)$ as:

$P_{{\mathcal{N}}}(\tilde{u}|g)\propto\eta{\mathcal{I}}({\bm{x}}_{\tilde{u}}^{T}\cdot{\bm{x}}_{g}>0\\})+(1-\eta)\frac{1}{|{\mathcal{U}}|}$
(6.6)

where ${\mathcal{I}}(\cdot)$ is an indicator function and $\eta$ controls the
sampling bias. We set $\eta=0.5$ across all our experiments. In comparison to
random negative sampling, our experiments indicate that preference-biased
negative user sampling exhibits better discriminative abilities.

When $L_{MI}$ is trained jointly with the base recommender loss $L_{R}$
(equation 6.4), maximizing user-group MI enhances the quality of user and
group representations computed by the encoder $f_{\textsc{enc}}(\cdot)$ and
aggregator $f_{\textsc{agg}}(\cdot)$. We now present our proposed approach to
overcome the limitations due to the fixed regularizer $L_{U}$ (Section 6.3.3).

####  6.4.3 Contextual User Preference Weighting

In this section, we describe a contextual weighting strategy to identify and
prioritize personal preferences of relevant group members, to overcome group
interaction sparsity. We avoid degenerate solutions by varying the extent of
regularization induced by each ${\bm{x}}_{u}$ (for user $u$) across groups
through group-specific relevance weights $w(u,g)$. Contextual weighting
accounts for user participation in diverse groups with different levels of
shared interests.

By maximizing user-group mutual information, the trained discriminator
${\bm{D}}$ outputs scores ${\bm{D}}({\bm{e}}_{u},{\bm{e}}_{g})$ that quantify
the contextual informativeness of each $(u,g)$ pair (higher scores for
informative users). Thus, we set the relevance weight $w(u,g)$ for group
member $u\in u^{g}$ to be proportional to the discriminator score
${\bm{D}}({\bm{e}}_{u},{\bm{e}}_{g})$. Instead of regularizing the user
representations ${\bm{E}}$ with ${\bm{x}}_{u}$ in each group ($L_{U}$ in eqn
6.4), we directly regularize the group representation ${\bm{e}}_{g}$ with
${\bm{x}}_{u}$ in proportion to ${\bm{D}}({\bm{e}}_{u},{\bm{e}}_{g})$ for each
group member $u$. Direct optimization of the group representation
${\bm{e}}_{g}$ (instead of ${\bm{e}}_{u}$) results in more effective model
regularization with faster flow of gradients, especially with sparse group
interactions in ephemeral groups. Thus, we now define the contextually
weighted user-loss $L_{UG}$ as:

$L_{UG}=-\sum\limits_{g\in{\mathcal{G}}}\frac{1}{|{\bm{x}}_{g}|}\sum\limits_{i\in{\mathcal{I}}}\sum\limits_{u\in
u^{g}}{\bm{D}}({\bm{e}}_{u},{\bm{e}}_{g})\;x_{ui}\log\pi_{i}({\bm{e}}_{g})$
(6.7)

where $L_{UG}$ effectively regularizes the latent group representation
${\bm{e}}_{g}$ with the individual activities of group member $u$ with
contextual relevance weight ${\bm{D}}({\bm{e}}_{u},{\bm{e}}_{g})$.

The overall model objective of our framework GroupIM includes three terms:
$L_{G}$, $L_{UG}$, and $L_{MI}$, which is described in Section 6.4.4 in
detail. GroupIM regularizes the latent representations computed by
$f_{\textsc{enc}}(\cdot)$ and $f_{\textsc{agg}}(\cdot)$ through user-group MI
maximization ($L_{MI}$) to contrastively capture group member associations;
and contextual MI-guided weighting ($L_{UG}$) to prioritize individual
preferences.

####  6.4.4 Model Details

We now describe the architectural details of preference encoder
$f_{\textsc{enc}}(\cdot)$, aggregator $f_{\textsc{agg}}(\cdot)$, discriminator
${\bm{D}}$, and an alternative optimization approach to train our framework
GroupIM.

##### User Preference Encoder

To encode individual user preferences ${\bm{X}}_{U}$ into preference
embeddings ${\bm{E}}$, we use a Multi-Layer Perceptron with two fully
connected dense layers, defined by:

${\bm{e}}_{u}=f_{\textsc{enc}}({\bm{x}}_{u})=\sigma({\bm{W}}_{2}^{T}(\sigma({\bm{W}}_{1}^{T}{\bm{x}}_{u}+b_{1})+b_{2})$
(6.8)

where ${\bm{e}}_{u}$ is the latent preference representation for user $u$,
with learnable weight matrices
${\bm{W}}_{1}\in{\mathbb{R}}^{|{\mathcal{I}}|\times D}$ and
${\bm{W}}_{2}\in{\mathbb{R}}^{D\times D}$, biases
$b_{1},b_{2}\in{\mathbb{R}}^{D}$, and $tanh(\cdot)$ activations for non-
linearity $\sigma$.

Pre-training: We pre-train the weights and biases of the first encoder layer
(${\bm{W}}_{1},{\bm{b}}_{1}$) on the user-item interaction matrix
${\bm{X}}_{U}$ with user-loss $L_{U}$ (equation 6.4). We use these pre-trained
parameters to initialize the first layer of $f_{\textsc{enc}}(\cdot)$ before
optimizing the overall objective of GroupIM. Our model ablation studies in
Section 6.5.5 indicate significant improvements owing to this pre-trained
initialization strategy.

##### Group Preference Aggregators

We consider three preference aggregators Maxpool, Meanpool and, Attention,
which are widely used in graph neural networks [243, 41, 68] and have close
ties to aggregators examined in prior work, i.e., Maxpool and Meanpool mirror
the heuristics of maximum satisfaction [125] and averaging [124], while
attentions learn varying member contributions [131, 132]. We define the three
preference aggregators below:

  * •

Maxpool: The preference embedding of each member is passed through MLP layers,
followed by element-wise max-pooling to aggregate group member embeddings,
given by:

${\bm{e}}_{g}=\textsc{max}(\\{\sigma({\bm{W}}_{\textsc{agg}}{\bm{e}}_{u}+b),\forall
u\in u_{g}\\})$ (6.9)

where max denotes the element-wise max operator and $\sigma(\cdot)$ is a
nonlinear activation. Intuitively, the MLP layers compute features for each
member, and max-pooling over each of the computed features effectively
captures different aspects of group members.

  * •

Meanpool: We similarly apply an element-wise mean-pooling operation after the
MLP layers, to compute the latent group representation ${\bm{e}}_{g}$ as:

${\bm{e}}_{g}=\textsc{mean}(\\{\sigma({\bm{W}}_{\textsc{agg}}{\bm{e}}_{u}+b),\forall
u\in u_{g}\\})$ (6.10)

  * •

Attention: To explicitly differentiate group members’ roles, we employ neural
attentions [130] to compute a weighted sum of members’ preference
representations, where the weights are learned by an attention network,
parameterized by a single MLP layer.

${\bm{e}}_{g}=\sum\limits_{u\in
u_{g}}\alpha_{u}{\bm{W}}_{\textsc{agg}}{\bm{e}}_{u}\hskip
10.0pt\alpha_{u}=\frac{\exp({\bm{h}}^{T}{\bm{W}}_{\textsc{agg}}{\bm{e}}_{u})}{\sum\limits_{u^{{}^{\prime}}\in
u_{g}}\exp({\bm{h}}^{T}{\bm{W}}_{\textsc{agg}}{\bm{e}}_{u^{{}^{\prime}}})}$
(6.11)

where $\alpha_{u}$ indicates the contribution of a group member $u$ towards
the group decision. We exclude bias terms for the sake of brevity. This
formulation can be trivially extended to item-conditioned weighting [131],
self-attention [133] and sub-attention networks [132].

##### Discriminator Architecture

The contrastive discriminator architecture learns a scoring function to assign
higher scores to observed $(u,g)$ pairs relative to negative examples, thus
parameterizing group-specific relevance weight $w(u,g)$. Similar to existing
work on neural mutual information estimation [44], we use a simple bilinear
function to score user-group representation pairs. The discriminator function
${\bm{D}}$ is defined by the following equation:

${\bm{D}}({\bm{e}}_{u},{\bm{e}}_{g})=\sigma({\bm{e}}_{u}^{T}{\bm{W}}{\bm{e}}_{g})$
(6.12)

where ${\bm{W}}$ is a learnable scoring matrix and $\sigma$ is the logistic
sigmoid non-linearity function to convert raw scores into probabilities of
$({\bm{e}}_{u},{\bm{e}}_{g})$ being a positive example. We leave investigation
of further architectural variants for the discriminator ${\bm{D}}$ to future
work.

##### Model Optimization

The overall objective of GroupIM is composed of three terms, the group-loss
$L_{G}$ (Equation 6.3), contextually weighted user-loss $L_{UG}$ (Equation
6.7), and MI maximization loss $L_{MI}$ (Equation 6.5). The combined model
training objective is given by:

$L=\underbrace{L_{G}}_{\text{Group Recommendation Loss}}+\overbrace{\lambda
L_{UG}}^{\text{Contextually Weighted User
Loss}}+\underbrace{L_{MI}}_{\text{User-Group MI Maximization Loss}}$ (6.13)

We train GroupIM using an alternating optimization schedule. In the first
step, the discriminator ${\bm{D}}$ is held constant, while optimizing the
group recommender on $L_{G}+\lambda L_{UG}$. The second step trains ${\bm{D}}$
on $L_{MI}$, resulting in gradient updates for both parameters of ${\bm{D}}$
as well as those of the user encoder $f_{\textsc{enc}}(\cdot)$ and preference
aggregator $f_{\textsc{agg}}(\cdot)$.

Thus, the discriminator ${\bm{D}}$ only seeks to regularize the model (i.e.,
encoder and aggregator) during training through loss terms $L_{MI}$ and
$L_{UG}$. During inference, we directly use the regularized encoder
$f_{\textsc{enc}}(\cdot)$ and aggregator $f_{\textsc{agg}}(\cdot)$ to make
group recommendations.

##### Model Objective Interpretation

To understand our model better, we examine a special case with fixed relevance
weights $w(u,g)=1$ without MI maximization. Here, the overall model objective
reduces to:

$\displaystyle
L^{{}^{\prime}}=-\sum\limits_{g\in{\mathcal{G}}}\Big{[}\frac{1}{|{\bm{x}}_{g}|}\sum\limits_{i\in{\mathcal{I}}}x_{gi}\log\pi_{i}({\bm{e}}_{g})+\lambda\sum\limits_{i\in{\mathcal{I}}}\bar{x}_{g}\log\pi_{i}({\bm{e}}_{g})\Big{]}$
(6.14)

where $\bar{x}_{gi}=\frac{1}{|{\bm{x}}_{g}|}\sum\limits_{u\in u^{g}}x_{ui}$ is
the mean preference vector of users in group $g$ for item $i\in{\mathcal{I}}$.
The first term is a supervised loss to predict label $x_{gi}$ from group
embedding ${\bm{e}}_{g}$. The second term can be viewed as an unsupervised
group reconstruction loss that encodes member preferences
$\\{{\bm{x}}_{u}:u\in u^{g}\\}$ into the group embedding ${\bm{e}}_{g}$ to
reconstruct their mean preference vector $\bar{x}_{g}$. In the special case of
$w(u,g)=1$, GroupIM reduces to a classical semi-supervised learning objective
(similar to Equation 6.4) for group recommendation.

Thus, our framework GroupIM can be overall regarded as jointly performing
self-supervised learning with user-group mutual information maximization (via
loss term $L_{MI}$) and semi-supervised learning with group-adaptive
regularization (via loss term $L_{UG}$).

###  6.5 Experiments

Dataset |  Yelp |  Weeplaces |  Gowalla |  Douban
---|---|---|---|---
# Users |  7,037 |  8,643 |  25,405 |  23,032
# Items |  7,671 |  25,081 |  38,306 |  10,039
# Groups |  10,214 |  22,733 |  44,565 |  58,983
# U-I interactions |  220,091 |  1,358,458 |  1,025,500 |  1,731,429
# G-I Interactions |  11,160 |  180,229 |  154,526 |  93,635
Average # items/user |  31.3 |  58.83 |  40.37 |  75.17
Average # items/group |  1.09 |  2.95 |  3.47 |  1.59
Average group size |  6.8 |  2.9 |  2.8 |  4.2
Table 6.2: Statistics of four real-world datasets with ephemeral groups.
Group-Item interactions are sparse: average number of interacted items per
group $<$ 3.5.

In this section, we present an extensive quantitative and qualitative analysis
of our model. We first introduce datasets, baselines, and experimental setup
(Section 6.5.1, 6.5.2, and 6.5.3), followed by our main group recommendation
results (Section 6.5.4). In Section 6.5.5, we conduct an ablation study to
understand our gains over the base recommender. In Section 6.5.6, we study how
key group characteristics (group size, coherence, and aggregate diversity)
impact recommendation results. In Section 6.5.7, we qualitatively visualize
the variation in discriminator scores assigned to group members, for different
kinds of groups. Finally, we discuss the limitations of our approach in
Section 6.5.9.

####  6.5.1 Datasets

First, we conduct experiments on large-scale POI (point-of-interest)
recommendation datasets extracted from three location-based social networks.
Since the POI datasets do not contain explicit group interactions, we
construct group interactions by jointly using the check-ins and social network
information: check-ins at the same POI within 15 minutes by an individual and
her subset of friends in the social network together constitutes a single
group interaction, while the remaining check-ins at the POI correspond to
individual interactions. We define the group recommendation task as
recommending POIs to ephemeral groups of users. The datasets were pre-
processed to retain users and items with five or more check-ins each. We
present dataset descriptions below:

  * •

Weeplaces 111https://www.yongliu.org/datasets/: we extract check-ins on POIs
over all major cities in the United States, across various categories
including Food, Nightlife, Outdoors, Entertainment and Travel.

  * •

Yelp 222https://www.yelp.com/dataset/challenge: user check-ins on local
restaurants located in the city of Los Angeles, California.

  * •

Gowalla [211]: we use restaurant check-ins across all cities in the United
States, in the time period upto June 2011.

Second, we evaluate event venue recommendation on Douban, which is the largest
online event-based social networking platform in China.

  * •

Douban [233]: users organize and participate in social events, where users
attend events together in groups and items correspond to event venues. During
pre-processing, we filter out users and venues with less than 10 interactions
each.

Groups across all datasets are ephemeral since group interactions are sparse
(average number of items per group $<3.5$ in Table 6.2)

####  6.5.2 Baselines

We present comparisons against state-of-the-art baselines in two categories:
score aggregation methods with predefined aggregators, and data-driven
preference aggregators.

  * •

Popularity [244]: recommends items to groups based on item popularity, which
is measured by its interaction count in the training set.

  * •

User-based CF + Score Aggregation: We utilize a state-of-the-art neural
recommendation model VAE-CF [90], followed by score aggregation via: averaging
(AVG), least-misery (LM), maximum satisfaction (MAX), and relevance-
disagreement (RD).

  * •

COM [123]: a probabilistic generative model that determines group decisions
based on group members’ individual preferences and topic-dependent influence.

  * •

CrowdRec [129]: a probabilistic generative model that extends COM through
item-specific latent variables capturing their global popularity for group
recommendation.

  * •

MoSAN [132]: a neural group recommender that employs a collection of sub-
attentional networks to model group member interactions. Since MoSAN
originally ignores individual activities ${\bm{X}}_{U}$, we include
${\bm{X}}_{U}$ into ${\bm{X}}_{G}$ as pseudo-groups with single users.

  * •

AGREE [131]: a neural group recommender that utilizes attentional group
preference aggregation to compute item-specific group member weights, for
joint model training over individual and group interactions.

We tested GroupIM by substituting three preference aggregators, Maxpool,
Meanpool, and Attention (Section 6.4.4). All experiments were conducted on a
single Nvidia Tesla V100 GPU with PyTorch [245] implementations on the Linux
platform. Our implementation of GroupIM and datasets are publicly
available333https://github.com/CrowdDynamicsLab/GroupIM.

Dataset | Yelp (LA) | Weeplaces | Gowalla | Douban
---|---|---|---|---
Metric |  N@20 |  N@50 |  N@20 |  N@50 |  N@20 |  N@50 |  N@20 |  N@50
Predefined Score Aggregators
Popularity [244] |  0.000 |  0.000 |  0.063 |  0.074 |  0.075 |  0.088 |  0.003 |  0.005
VAE-CF + AVG [90, 124] |  0.142 |  0.179 |  0.273 |  0.313 |  0.318 |  0.362 |  0.179 |  0.217
VAE-CF + LM [90, 122] |  0.097 |  0.120 |  0.277 |  0.311 |  0.375 |  0.409 |  0.221 |  0.252
VAE-CF + MAX [90, 125] |  0.099 |  0.133 |  0.229 |  0.270 |  0.267 |  0.316 |  0.156 |  0.194
VAE-CF + RD [90, 126] |  0.143 |  0.181 |  0.239 |  0.279 |  0.294 |  0.339 |  0.178 |  0.216
Data-driven Preference Aggregators
COM [123] |  0.143 |  0.154 |  0.329 |  0.348 |  0.223 |  0.234 |  0.283 |  0.288
Crowdrec [129] |  0.082 |  0.101 |  0.353 |  0.370 |  0.325 |  0.338 |  0.121 |  0.188
AGREE [131] |  0.123 |  0.168 |  0.242 |  0.292 |  0.160 |  0.223 |  0.126 |  0.173
MoSAN [132] |  0.470 |  0.494 |  0.287 |  0.334 |  0.323 |  0.372 |  0.193 |  0.239
Group Information Maximization Recommenders (GroupIM)
GroupIM-Maxpool |  0.488 |  0.501 |  0.479 |  0.505 |  0.433 |  0.463 |  0.291 |  0.313
GroupIM-Meanpool |  0.629 |  0.637 |  0.518 |  0.543 |  0.476 |  0.504 |  0.323 |  0.351
GroupIM-Attention |  0.633 |  0.647 |  0.521 |  0.546 |  0.477 |  0.505 |  0.325 |  0.356
Table 6.3: Group recommendation results on four datasets, N@K denotes NDCG@K
metric at $K=20$ and $50$. The GroupIM variants indicate maxpool, meanpool,
and attention as preference aggregators in our MI maximization framework.
GroupIM achieves significant performance gains of 31 to 62% NDCG@20 over
competing group recommenders. Notice that meanpool and attention variants of
our model GroupIM achieve comparable group recommendation performance across
all datasets.

####  6.5.3 Experimental Setup

We randomly split the set of all groups into training (70%), validation (10%),
and test (20%) sets, while utilizing the individual interactions of all users
for training. Note that each group appears only in one of the three sets. The
test set contains strict ephemeral groups (i.e., a specific combination of
users) that do not occur in the training set. Thus, we train on ephemeral
groups and test on strict ephemeral groups. We use NDCG@K and Recall@K as
evaluation metrics. Recall@K measures the percentage of relevant items in the
top-K rank list for each group, while NDCG@K is position-sensitive and assigns
higher scores for relevant items at the top. We report model performance at
$K=20,50$ in Tables 6.3 and 6.4.

We tune the latent dimension (number of topics for COM and Crowdrec) in the
range $\\{32,64,128\\}$ for all models and other baseline hyper-parameters in
ranges centered at author-provided valuess on the validation set. In GroupIM,
we use two fully connected layers of size 64 each in the MLP preference
encoder $f_{\textsc{enc}}(\cdot)$ and tune $\lambda$ in the range
$\\{2^{-4},2^{-3},\dots,2^{6}\\}$. We use 5 negative samples for each true
user-group pair to train the discriminator. Our experimental results are
averaged over 10 random runs with random initializations.

Dataset | Yelp (LA) | Weeplaces | Gowalla | Douban
---|---|---|---|---
Metric |  R@20 |  R@50 |  R@20 |  R@50 |  R@20 |  R@50 |  R@20 |  R@50
Predefined Score Aggregators
Popularity [244] |  0.001 |  0.001 |  0.126 |  0.176 |  0.143 |  0.203 |  0.009 |  0.018
VAE-CF + AVG [90, 124] |  0.322 |  0.513 |  0.502 |  0.666 |  0.580 |  0.758 |  0.381 |  0.558
VAE-CF + LM [90, 122] |  0.198 |  0.316 |  0.498 |  0.640 |  0.610 |  0.750 |  0.414 |  0.555
VAE-CF + MAX [90, 125] |  0.231 |  0.401 |  0.431 |  0.604 |  0.498 |  0.702 |  .339 |  0.517
VAE-CF + RD [90, 126] |  0.321 |  0.513 |  0.466 |  0.634 |  0.543 |  0.723 |  0.379 |  0.557
Data-driven Preference Aggregators
COM [123] |  0.232 |  0.286 |  0.472 |  0.557 |  0.326 |  0.365 |  0.417 |  0.436
Crowdrec [129] |  0.217 |  0.315 |  0.534 |  0.609 |  0.489 |  0.548 |  0.375 |  0.681
AGREE [131] |  0.332 |  0.545 |  0.484 |  0.711 |  0.351 |  0.605 |  0.310 |  0.536
MoSAN [132] |  0.757 |  0.875 |  0.548 |  0.738 |  0.584 |  0.779 |  0.424 |  0.639
Group Information Maximization Recommenders (GroupIM)
GroupIM-Maxpool |  0.676 |  0.769 |  0.676 |  0.776 |  0.628 |  0.747 |  0.524 |  0.637
GroupIM-Meanpool |  0.778 |  0.846 |  0.706 |  0.804 |  0.682 |  0.788 |  0.569 |  0.709
GroupIM-Attention |  0.782 |  0.851 |  0.716 |  0.813 |  0.686 |  0.796 |  0.575 |  0.714
Table 6.4: Group recommendation results on four datasets, R@K denotes the
Recall@K metric at $K=20$ and $50$. The GroupIM variants indicate maxpool,
meanpool, and attention as preference aggregators. GroupIM achieves
significant gains of 3 to 28% Recall@20 over competing group recommenders.

####  6.5.4 Experimental Results

We note the following key observations from our experimental results comparing
our framework GroupIM with its three aggregator variants, against competing
baselines on ephemeral group recommendation (Tables 6.3 and 6.4).

First, heuristic score aggregation with neural user-level recommenders (i.e.,
VAE-CF) performs comparable to (and often beats) probabilistic models (COM,
Crowdrec). Neural methods with multiple non-linear transformations, are
expressive enough to identify latent groups of similar users just from their
individual item interactions.

Second, there is no clear winner among the different pre-defined score
aggregation strategies, e.g., VAE-CF + LM (least misery) outperforms the rest
on Gowalla and Douban, while VAE-CF + LM (averaging) is superior on Yelp and
Weeplaces. This empirically validates the non-existence of a single optimal
strategy for all datasets.

Third, MoSAN [132] outperforms both probabilistic models and fixed score
aggregators on most datasets. MoSAN achieves better results owing to the
expressive power of neural preference aggregators (such as sub-attention
networks) to capture group member interactions, albeit not explicitly
differentiating personal and group activities. Notice that naive joint
training over personal and group activities via static regularization (as in
AGREE [131]) results in poor performance due to sparsity in group
interactions. Static regularizers on ${\bm{X}}_{U}$ cannot distinguish the
role of users across groups, resulting in models that lack generalization to
ephemeral groups with sparse interactions.

GroupIM variants outperform baselines significantly, with attention achieving
overall best results. In contrast to neural methods (i.e., MoSAN and AGREE),
GroupIM regularizes the latent representations by contextually weighting the
personal preferences of informative members, thus effectively tackling group
interaction sparsity. The maxpool variant is noticeably inferior, due to the
higher sensitivity of max operation to outlier members.

Note that Meanpool performs comparably to attention. This is because in
GroupIM, the discriminator ${\bm{D}}$ does the heavy-lifting of contextually
differentiating the role of users across groups to effectively regularize the
encoder $f_{\textsc{enc}}(\cdot)$ and aggregator $f_{\textsc{agg}}(\cdot)$
modules. If $f_{\textsc{enc}}(\cdot)$ and ${\bm{D}}$ are expressive enough,
efficient meanpool aggregation can achieve near state-of-the-art results
(Tables 6.3 and 6.4).

An important implication is the reduced inference complexity of our model,
i.e., once trained using our MI maximizing framework, simple aggregators (such
as meanpool) suffice to achieve state-of-the-art performance. This is
especially significant, considering that our closest baseline MoSAN [132]
utilizes sub-attentional preference aggregation networks that scale
quadratically with group size.

Figure 6.3: NDCG@K across size of rank list $K$. Variance bands indicate 95%
confidence intervals over 10 random runs. Existing methods underperform since
they either disregard member roles (VAE-CF variants) or overfit to the sparse
group activities. GroupIM contextually identifies informative members and
regularizes their representations, to show strong gains. Figure 6.4: Recall@K
across size of rank list $K$ (1 to 50). Variance bands indicate 95% confidence
intervals over 10 independent runs. GroupIM has larger recall gains for
smaller $K(<20)$, indicating more accurate recommendations within top ranks.

We compare the variation in NDCG and Recall scores of all models with size of
rank list $K$ (1 to 50) in figures 6.3 and 6.4. We only depict the best
aggregator for VAE-CF. GroupIM consistently generates more precise
recommendations across all datasets. We observe smaller gains in Douban, where
the user-item interactions exhibit substantial correlation with corresponding
group activities. GroupIM achieves significant performance gains in
characterizing diverse groups, evidenced by our results in section 6.5.6.

####  6.5.5 Model Analysis

In this section, we present an ablation study to analyze several variants of
GroupIM, guided by our motivations (Section 6.3.3). In our experiments, we
choose attention as the aggregator due to its consistently high performance.
We conduct empirical studies on Weeplaces and Gowalla datasets to report
NDCG@50 and Recall@50 in Table 6.5.

First, we examine the model performance of the base group recommender
${\bm{R}}$ (Section 6.3.2) which does not utilize user-group MI estimation or
maximization for model training. The ablation study results are shown in Table
6.5.

Dataset | Weeplaces | Gowalla
---|---|---
Metric |  N@50 |  R@50 |  N@50 |  R@50
Base Group Recommender Variants
(1) Base ($L_{G}$) |  0.420 |  0.621 |  0.369 |  0.572
(2) Base ($L_{G}+\lambda L_{U}$) |  0.427 |  0.653 |  0.401 |  0.647
GroupIM Variants
(3) GroupIM ($L_{G}+L_{MI}$) |  0.431 |  0.646 |  0.391 |  0.625
(4) GroupIM (Uniform weights) |  0.441 |  0.723 |  0.418 |  0.721
(5) GroupIM (Cosine similarity) |  0.488 |  0.757 |  0.445 |  0.739
(6) GroupIM (No pre-training) |  0.524 |  0.773 |  0.472 |  0.753
(7) GroupIM ($L_{G}+\lambda L_{UG}+L_{MI}$) |  0.543 |  0.804 |  0.505 |  0.796
Table 6.5:  GroupIM ablation study (NDCG and Recall at $K=50$). Contrastive
representation learning (row 3) improves the base recommender (row 1), but is
substantially more effective with group-adaptive preference weighting (row 7).

##### Base Group Recommender.

We examine two variants below:
(1) We train the base recommender ${\bm{R}}$ on group interactions
${\bm{X}}_{G}$ with loss $L_{G}$ (equation 6.3).
(2) We train the base recommender ${\bm{R}}$ jointly on individual
${\bm{X}}_{U}$ and group ${\bm{X}}_{G}$ interactions with static
regularization on ${\bm{X}}_{U}$ using joint loss $L_{R}$ (Equation 6.4).

In comparison to similar neural group preference aggregator MoSAN, our base
group recommender ${\bm{R}}$ is stronger on NDCG but inferior on Recall. The
difference is likely due to the multinomial likelihood used to train
${\bm{R}}$, in contrast to the ranking loss in MoSAN. Static regularization
via ${\bm{X}}_{U}$ (row 1) results in higher gains for Gowalla (richer user-
item interactions) with relatively larger margins for Recall than NDCG. We now
empirically examine the different model variants of GroupIM in two parts:

##### GroupIM: Self-supervised Representation Learning.

We analyze the benefits derived by just training the contrastive discriminator
${\bm{D}}$ to capture group member associations through user-group MI
maximization, i.e., we define a model variant (row 3) to optimize just
$L_{G}+L_{MI}$, without the $L_{UG}$ term. Direct user-group MI maximization
(row 3) improves over the base group recommender ${\bm{R}}$ (row 1), thus
validating the benefits of contrastive self-supervised learning via mutual
information maximization, however still suffers from lack of user preference
prioritization.

##### GroupIM: Group-adaptive Preference Prioritization

We analyze the utility of data-driven contextual weighting (via user-group
MI), by examining two alternate fixed strategies to define $w(u,g)$ in the
loss term $L_{UG}$ of GroupIM:
(4) Uniform weights: We assign the same relevance weight $w(u,g)=1$ for each
group member $u$ in group $g$, when optimizing the loss term $L_{UG}$.
(5) Cosine similarity: To model user-group correlation, we set the relevance
weight $w(u,g)$ as the cosine similarity between ${\bm{x}}_{u}$ and
${\bm{x}}_{g}$.

From table 6.5 (rows 4 and 5), the uniform weights variant of loss $L_{UG}$
(row 4) surpasses the statically regularized model (row 2), due to more direct
feedback from ${\bm{X}}_{U}$ to the group embedding ${\bm{e}}_{g}$ during
model training. Cosine similarity (row 5) achieves stronger gains owing to
more accurate correlation-guided user weighting across groups. Our model
GroupIM (row 7) has strong gains over the fixed weighting strategies as a
result of its regularization strategy to contextually identify informative
members across different groups.

##### GroupIM: Pre-training $f_{\textsc{ENC}}(\cdot)$ on ${\bm{X}}_{U}$

We depict model performance without pre-training (random initializations) in
row 6. Our model (row 7) achieves noticeable gains; pre-training identifies
good model initialization points for better convergence.

####  6.5.6 Impact of Group Characteristics

In this section, we examine our results closely to understand the reason for
GroupIM’s gains over baselines in two datasets, Weeplaces and Gowalla. We
study ephemeral groups along three facets: group size; group coherence; and
group aggregate diversity.

Figure 6.5: Performance (NDCG@50), across group sizes. GroupIM has larger
gains for larger groups due to accurate user associations learnt via MI
maximization.

##### Group Size

We partition test groups into bins based on five levels of group size (2-3,
4-5, 6-7, 8-9, and $\geq$10). Figure 6.5 depicts the variation in NDCG@50
scores on Weeplaces and Gowalla.

We make three key observations: methods that explicitly distinguish individual
and group activities (such as COM, CrowdRec, GroupIM), exhibit distinctive
trends wrt group size. In contrast, MoSAN [132] and AGREE [131], which either
uniformly mix both behaviors or apply static regularizers, show no noticeable
variation; Performance generally increases with group size. Although test
groups are previously unseen, for larger groups, subsets of inter-user
interactions are more likely to be seen during training, thus resulting in
better performance; GroupIM achieves higher (or steady) gains for groups of
larger sizes owing to its more accurate prioritization of personal preferences
for each member, e.g., GroupIM clearly has stronger gains for groups of sizes
8-9 and $\geq 10$ in Gowalla.

##### Group Coherence

We define group coherence as the mean pair-wise correlation of personal
activities (${\bm{x}}_{u}$) of group members, i.e., if a group has users who
frequently co-purchase items, it receives greater coherence. We separate test
groups into four quartiles by their coherence scores. Figure 6.6 depicts
NDCG@50 for groups under each quartile (Q1 - Lower values).

GroupIM has stronger gains for groups with low coherence (quartiles Q1 and
Q2), which empirically validates the efficacy of contextual user preference
weighting in regularizing the encoder and aggregator, for groups with
dissimilar member preferences.

Figure 6.6: Performance (NDCG@50), across group coherence quartiles (Q1:
lowest, Q4: highest). GroupIM has larger gains in Q1 & Q2 (low group
coherence). Figure 6.7: Performance (NDCG@50), across group aggregate
diversity quartiles (Q1: lowest, Q4: highest). GroupIM has larger gains in Q3
& Q4 (high diversity).

##### Group Aggregate Diversity

We adapt the classical aggregate diversity metric [246] to define group
aggregate diversity as the total number of distinct items interacted across
all group members, i.e., if the set of all purchases of group members covers a
wider range of items, then the group has higher aggregate diversity. We report
NDCG@50 across aggregate diversity quartiles in figure 6.7.

Model performance typically decays (and stabilizes), with increase in group
aggregate diversity. Diverse groups with large candidate item sets pose an
information overload for group recommenders, leading to worse results.
Contextual prioritization with contrastive self-supervised learning benefits
diverse groups, as evidenced by the higher relative gains of GroupIM highly
for diverse groups (quartiles Q3 and Q4).

####  6.5.7 Qualitative MI Discriminator Analysis

We examine the contextual relevance weights $w(u,g)$ estimated by GroupIM over
test ephemeral groups, across group size and coherence.

Figure 6.8: MI variation (std. deviation in discriminator scores over members)
per group coherence quartile across group sizes. For groups of a given size,
as coherence increases, MI variation decreases. As groups increase in size, MI
variation increases.

We divide groups into four bins based on group sizes (2-3, 4-6, 7-9, and $\geq
10$), and partition them into quartiles based on group coherence per bin. To
analyze the variation in contextual informativeness across group members, we
compute MI variation as the standard deviation of scores given by ${\bm{D}}$
over group members. Figure 6.8 depicts letter-value plots of MI variation for
groups in corresponding coherence quartiles across group sizes on Weeplaces.

MI variation increases with group size, since larger groups often comprise
users with divergent roles and interests. Thus, the learned discriminator
generalizes to unseen groups, to discern and estimate markedly different
relevance scores for each group member. To further examine the intuition
conveyed by the scores, we compare MI variation across group coherence
quartiles within each group size-range.

MI variation is negatively correlated with group coherence for groups of
similar sizes, e.g., MI variation is consistently higher for groups with low
coherence (quartiles Q1 and Q2). For highly coherent groups (quartile Q4), the
discriminator ${\bm{D}}$ assigns comparable scores across all members, which
is consistent with our intuitions and earlier empirical results on the
efficacy of simple averaging strategies for coherent groups.

####  6.5.8 Sensitivity Analysis

In this section, we analyze parameter sensitivity of our framework GroupIM
with respect to the user-preference weight $\lambda$. We show a plot depicting
model performance (NDCG@50) versus $\lambda$ in figure 6.9. Varying the hyper-
parameter $\lambda$ in different ranges of values results in performance drops
on either side, but for differing reasons. Low $\lambda$ values result in
overfitting to the group activities ${\bm{X}}_{G}$, while substantially larger
values result in degenerate representations that lack group distinctions.
Although we tune $\lambda$ for each dataset, we find recommendation
performance of our model to be stable in a broad range of values that transfer
across different datasets.

Figure 6.9:  $\lambda$ is varied in $\\{2^{-4},2^{-3},\dots,2^{6}\\}$. Larger
$\lambda$ values result in performance drops due to overfitting on
${\bm{X}}_{U}$. GroupIM is robust in a wide range of values.

####  6.5.9 Limitations

We identify two limitations of our work. Despite learning to contextually
prioritize users’ preferences across groups, $\lambda$ controls the overall
strength of preference regularization. Since optimal $\lambda$ varies across
datasets and applications, we plan to explore meta-learning approaches to
eliminate such hyper-parameters [247].

GroupIM relies on user-group MI estimation to contextually identify
informative members, which might become challenging when users have sparse
individual interaction histories. In such a scenario, side information (e.g.,
social network of users), or contextual factors (e.g., location, interaction
time) [105] can prove effective.

###  6.6 Conclusion

This chapter introduces a recommender architecture-agnostic framework GroupIM
that integrates arbitrary neural preference encoders and aggregators for
ephemeral group recommendation. To overcome group interaction sparsity,
GroupIM regularizes the user-group representation space by maximizing user-
group MI to contrastively capture preference covariance among group members.
Unlike prior work that incorporate individual preferences through static
regularizers, we dynamically prioritize the preferences of informative members
through MI-guided contextual preference weighting. Our extensive experiments
on four real-world datasets show significant gains for GroupIM over state-of-
the-art methods.

In this chapter, we explored our second inductive learning application of
generating item recommendations to ephemeral groups of users in a multipartite
interaction setting involving users, groups, and items. We designed a self-
supervised learning framework to specifically overcome group interaction
sparsity challenges. Finally, in the next chapter, we examine a practical
industrial pplication of generating friend suggestions in large-scale social
platforms, where we design inductive learning models with the capability to
seamlessly incorporate new users and dynamic user interactions.

##  Chapter 7 GraFRank: Multi-Faceted Friend Ranking in Social Platforms

###  7.1 Introduction

Learning latent user representations has become increasingly important in
advancing user understanding, and has seen widespread adoption in various
industrial settings, e.g., video recommendations on YouTube [3], related pin
recommendations on Pinterest [5] etc. The user representations learned using
deep models are effective at complementing, or even replacing conventional
collaborative filtering methods [248], and are versatile, e.g., the learned
user embeddings can be used to suggest new friendships and also to infer
profile attributes (e.g., age, gender) in social networks.

Learning latent representations of nodes in graphs has prominent applications
in multiple academic settings, such as link prediction [154], community
detection [249], and industrial recommender systems, including e-commerce
[250, 251], content discovery [5, 252], and food delivery [253]. Graph Neural
Networks (GNNs) [153] have emerged as a popular paradigm for graph
representation learning due to their ability to learn representations
combining graph structure and node/link attributes, without relying on
expensive feature engineering. GNNs can be formulated as a message passing
framework where node representations are learned by propagating features from
local graph neighborhoods via trainable aggregators. Recently, GNNs have
demonstrated promising results in a few industrial systems designed for item
recommendations in bipartite [5] or multipartite [252] user-to-item
interaction graphs.

Despite their rich representational ability, GNNs have been relatively
unexplored in large-scale user-user social interaction modeling applications,
like friend suggestion. Recommending new potential friends to encourage users
to expand their networks, is a cornerstone of social networking, and plays an
important role towards user retention, and promoting engagement within the
platform.

Prior efforts typically formulate friend suggestion as link prediction (or
matrix completion) with a rich literature of graph-based heuristics [254] to
quantify user-user affinity, e.g., two users are more likely to connect if
they have many common friends. A few GNN models target link prediction in
academic settings, learn aggregators over enclosing subgraphs around each
candidate link [255, 154, 256]; such models do not scale to industry-scale
social graphs with over millions of nodes and billions of edges. Still, GNNs
have enormous potential for learning expressive user representations in social
networks, due to their intuitive message-passing paradigm that enables
attention to social influence from friends in their ego-network.

Yet, designing GNNs for friend recommendations in large-scale social media
poses unique challenges. First, social networks are characterized by heavy-
tailed degree distributions, e.g., many networks approximately follow power-
law distributions [165]. This poses a key challenge of limited structural
information for a significant proportion of users with very few friends. A
related challenge is activity sparsity where a very small fraction of users
actively form new friendships at any given time. Secondly, contemporary social
networking platforms offer a multitude of avenues for users to interact, e.g.,
users can communicate with friends either by directly exchanging messages,
pictures, and videos, or indirectly through a variety of social actions,
including liking, sharing, and commenting on posts. Extracting knowledge from
such heterogeneous in-platform user actions is challenging, yet extremely
valuable to address sparsity challenges for a vast majority of inactive users.

Present Work: In this work, we overcome structural and interactional sparsity
by exploiting the rich knowledge of heterogeneous in-platform actions. We
formulate friend recommendation on social networks as multi-faceted friend
ranking on an evolving friendship graph, with multi-modal user features and
link communication features (Figure 7.1). We represent users with
heterogeneous feature sets spanning multiple modalities, that include a
collection of static profile attributes (e.g., demographic information) and
time-sensitive in-platform activities (e.g., content interests and
interactions). We also leverage pairwise link features on existing
friendships, which capture recent communication activities across multiple
direct (e.g., messages) and indirect (e.g., stories) channels within the
platform.

To understand the complexity of user interactions and gain insights into
various factors impacting friendship formation, we conduct an empirical
analysis to investigate attribute homophily with respect to different user
feature modalities. Our analysis reveals diverse homophily distributions
across modalities and users, and indicates non-trivial cross-modality
correlations. Motivated by these observations, we design an end-to-end GNN
architecture, GraFRank (Graph Attentional Friend Ranker) for multi-faceted
friend ranking.

GraFRank generates user representations by modality-specific neighbor
aggregation and cross-modality attention. Here, we handle heterogeneity in
modality homophily by learning modality-specific message-passing aggregators
to compute a set of latent representations for each user. Specifically, the
neighbor aggregator is driven by a friendship attention mechanism that
captures the influence of individual features and pairwise communications. We
introduce a cross-modality attention module to compute the effective user
representation by attending over the different modality-specific
representations for each user, thereby learning non-linear correlations across
modalities. We summarize our key contributions below:

Graph-Neural Friend Ranking: To our knowledge, ours is the first work to
investigate graph neural network usage and design for social user-user
interaction modeling applications. Unlike prior work that typically view
friend recommendation as structural link prediction, we present a novel
formulation with multi-modal user features and link features, to leverage
knowledge of rich heterogeneous user activities in social networking
platforms.

GraFRank Model: Motivated by our empirical study that reveals heterogeneity in
modality homophily and cross-modality correlations, we design a novel GNN
architecture, GraFRank, to learn multi-faceted user representations. Distinct
from conventional GNNs that operate on a single homogeneous feature space,
GraFRank encapsulates information from multiple correlated feature modalities
and user-user interactions.

Robust Experimental Results: Our experiments on two large-scale datasets from
a popular social networking platform Snapchat, indicate significant gains for
GraFRank over state-of-the-art baselines on friend candidate retrieval
(relative MRR gains of 30%) and ranking (relative MRR gains of 20%) tasks. Our
qualitative analysis indicates stronger gains for the crucial population of
less-active and low-degree users.

###  7.2 Related Work

We briefly review a few related lines of work on friend recommendations, graph
neural networks, and multi-modal learning.

Friend Recommendation: The earliest methods were carefully designed graph-
based heuristics of user-user proximity in social networks [254], e.g., path-
based Katz centrality [60] or common neighbor-based Adamic/Adar [61].
Supervised learning techniques exploited a collection of such extracted
pairwise features to train classifiers and ranking models [257, 258]. However,
computing pairwise heuristics on-the-fly for every potential link, is
infeasible in large-scale evolving social networks.

More recently, graph embedding methods to learn latent node representations in
graphs that capture the structural properties of a node and its neighborhoods,
gained popularity owing to the scalability and efficacy of skip-gram models
[259]. In particular, graph embedding models like node2vec [18] and Deepwalk
[17] learn unsupervised node embeddings to maximize the likelihood of co-
occurrence in fixed-length random walks, with applications in downstream tasks
such as node classification and link prediction.

Graph embedding methods cannot directly incorporate node/link features, and
more importantly learn latent embeddings for each node; this implies that
number of model parameters scales linearly with the size of the graph, which
is prohibitive for large-scale social networks with over multiple millions of
users.

Graph Neural Networks: GNNs learn node representations by recursively
propagating features (i.e., message passing) from local neighborhoods through
the use of aggregation and activation functions [153]. A key feature of GNNs
is their ability to learn node representations combining graph topology and
node/link attributes. Graph Convolutional Networks (GCNs) [39] learn degree-
weighted neighborhood aggregators by operating on the graph Laplacian. Many
models generalize GCN with a wide range of learnable aggregators, e.g., self-
attentions [40], mean and max pooling functions [41, 260]; these approaches
have consistently outperformed embedding techniques based upon random walks
[17, 18]. In contrast to most GNN models that store the entire graph in GPU
memory, GraphSAGE [41] is an inductive variant that reduces memory footprint
by sampling a fixed-size set of neighbors in each GNN layer. A few scalable
extensions include minibatch training with variance reduction [261, 262],
subgraph sampling [263], and graph clustering [264].

Despite the successes of GNNs in diverse applications, very few industrial
systems have developed large-scale GNN implementations. One recent system,
PinSage [5] extends GraphSAGE for Pinterest recommendation on a user-item
bipartite graph. MultiSage [252] extends PinSage to multipartite user-item
graphs.

However, GNNs remain unexplored for large-scale user-user social modeling
applications where users exhibit multifaceted behaviors by interacting with
different functionalities on social platforms. In our work, we design GNNs for
the important application of friend suggestion, through a novel multi-faceted
friend ranking formulation with multi-modal user features and link
communication features.

Multi-Modal Learning: Deep learning techniques for multi-modal feature fusion
over diverse modalities such as text, images, video, and graphs, have diverse
applications [265, 266]. Specifically, multi-modal extensions of GNNs have
been explored in micro-video recommendation [267] and urban computing [268]
applications. In contrast to prior work that regard modalities as largely
independent data sources, user feature modalities in social networks tend to
be correlated. Exploiting knowledge from multiple channels have been shown to
benefit recommendations in retail and e-commerce [269]. In this work, we
propose a novel cross-modality attention layer that captures non-linear
modality correlations, to learn multi-faceted user representations.

###  7.3 Preliminaries

We first formulate the problem of multi-faceted friend ranking in large-scale
social networking platforms (Section 7.3) and then briefly introduce relevant
background and notations on graph neural networks (Section 7.3.2).

####  7.3.1 Problem Formulation

In this section, we introduce the different information sources in a social
networking platform, that are relevant to friend recommendation. Each
registered individual in the platform is denoted by a user $u$ or $v$ and a
pair of users $(u,v)$ may be connected by a friendship, which is an undirected
relationship, i.e., if $u$ is a friend of $v$, $v$ is also a friend of $u$. We
assume that the social network has a set of $N$ users ${\mathcal{V}}$
introduced until our latest observation time of the platform. The friendship
graph ${\mathcal{G}}$ evolves when new friendships form and when new users
join the platform. In this work, we only consider the emerging of new users
and friendships while leaving the removal of existing users and edges as
future work.

Prior work typically represent a dynamic network as a sequence of static
snapshots, primarily due to scaling concerns. However, graph snapshots are
coarse approximations of the actual continuous-time network and rely on a
user-specified discrete time interval for snapshot creation [270, 271]. We
also assume multiple time-aware user-level features (across modalities) and
link(edge)-level features capturing pairwise user-user communications. In
industrial settings, such features are commonly extracted by routine batch
jobs and populated in an upstream database at regular time intervals (e.g.,
daily batch inference jobs), to facilitate efficient model training.

Thus, we adopt a hybrid data model that achieves the best of both worlds. We
formulate the friendship graph as a continuous-time dynamic graph (CTDG) with
the expressivity to record friendships at the finest possible temporal
granularity; and represent features as a sequence of daily snapshots where the
time-sensitive features (e.g., engagement activity) are recorded at different
time scales.

Friendship Graph: Let us consider an observation time window $(t_{s},t_{e})$
such that friendships created in this window specify the training data for the
friend ranking model. We divide this window $(t_{s},t_{e})$ into a sequence of
$S$ daily snapshots, denoted by $1,2,\dots,S$. Formally, we model the
friendship graph ${\mathcal{G}}$ as a timed sequence of friend creation events
over the entire time range $(0,t_{e})$, defined as:

######  Definition 7.1 (Friendship Graph).

Given a graph ${\mathcal{G}}=({\mathcal{V}},{\mathcal{E}},{\mathcal{T}})$, let
${\mathcal{V}}$ be the set of users,
${\mathcal{E}}\subseteq{\mathcal{V}}\times{\mathcal{V}}\times{\mathbb{R}}$ be
the set of friendship links between users in ${\mathcal{G}}$. At the finest
granularity, each link $e=(u,v,t)\in{\mathcal{E}}$ is assigned a unique
timestamp $t\in{\mathbb{R}}^{+};0<t<t_{e}$ that denotes the link creation
time, and ${\mathcal{T}}:{\mathbb{R}}^{+}\mapsto[0,S]$ is a function that maps
each timestamp $t$ to a corresponding snapshot.

Note that the time window $(t_{s},t_{e})$ corresponds to snapshots $[1,S]$,
and snapshot 0 is a placeholder for any $t<t_{s}$. However, friendship graph
${\mathcal{G}}$ includes the entire set of friendships created with all time-
stamped links in $(0,t_{e})$. The set of temporal neighbors of user $v$ at
time $t$ includes all friends created before $t$, defined as
$N_{t}(v)=\\{w:e=(v,w,t^{{}^{\prime}})\in{\mathcal{E}}\wedge
t^{{}^{\prime}}<t\\}$.

Multi-Modal Evolving User Features: In a social network, users typically use
multiple features, such as posting videos, exchanging messages or pictures
with friends, or liking and sharing posts and articles, which can be
indicative of both their stable traits and mutable interests. Since users with
shared interests tend to form social connections due to homophily [11], we
extract user attributes spanning a total of $K=4$ modalities, which include
profile attributes, in-app interests, friend creation activities, and user
engagement activities, described in detail below:

Profile Attributes: a set of (mostly) static demographic features describing
the user, including age, gender, languages, etc., that are listed or inferred
from their profile.

Content Interests: a real-valued feature vector describing the textual content
(e.g., posts, news) interacted by the user within the platform, e.g., topics
of posts liked by the user on Facebook.

Friending Activity: this feature modality records the aggregated number of
friend requests sent/received, reciprocated friendships, and viewed
suggestions made by the user in different time ranges (e.g., daily, weekly,
and monthly).

Engagement Activity: this modality records the aggregated number of in-app
direct and indirect engagements made by the user (e.g., direct messages,
pictures, and comments on posts) with other friends in different time ranges.

The user feature modalities include a combination of static and time-sensitive
features, i.e., the profile attributes are static while the rest of the
modalities are time-sensitive and often evolve at different scales across
users, e.g., a long-time active user may frequently communicate with a stable
set of friends, while a new user is more likely to quickly add new friends
before communicating.

The feature vector of a user $u\in{\mathcal{V}}$ in snapshot $s\in[1,S]$ is
defined by
${\mathbf{x}}_{v}^{s}=[{\mathbf{x}}_{v}^{s,1},\dots,{\mathbf{x}}_{v}^{s,K}]$
where ${\mathbf{x}}_{v}^{s,k}\in{\mathbb{R}}^{D}_{k}$ is the $k$-th user
feature modality and $[\cdot]$ denotes row-wise concatenation.

Figure 7.1: Desiderata for Multi-faceted Friend Ranking: temporally evolving
friendship graph with multi-modal user features and pairwise link
communication features.

Pairwise Link Communication Features: Social networks evolve centering around
communication artifacts, with two predominant types of online communication
channels: conversations and social actions. Conversations can include
exchanges of text messages, pictures, and video content with friends, which
indicate direct user-user interactions via communication. In contrast, social
actions facilitate indirect user interactions through a variety of actions,
e.g., posting a Snapchat Story (image or video content) or liking a Facebook
post, results in a passive broadcast to friends.

For conversational channels, we extract bidirectional edge features between
users and their friends reflecting the number of communications sent and
received by each pair of users. We capture indirect social actions by
recording the number of user actions associated with each friend. Similar to
user features, we extract edge features per snapshot by aggregating
communications at different time intervals. The link feature vector for a pair
of users $(u,v)$ at time $t$ (who became friends before $t$), is denoted by
${\mathbf{e}}^{s}_{uv}\in{\mathbb{R}}^{F}$ where $s={\mathcal{T}}(t)$ is the
snapshot corresponding to timestamp $t$, and $E$ is the number of extracted
link communication features.

We now formally define the problem of multi-faceted friend ranking in large-
scale social platforms, over friendship graph ${\mathcal{G}}$ with multi-modal
user features and pairwise link features.

######  Problem 7.1 (Multi-Faceted Friend Ranking).

Our objective is to leverage the multi-modal user features
$\\{{\mathbf{x}}^{s}_{v}:v\in{\mathcal{V}},1\leq s\leq S\\}$, link
communication features
$\\{{\mathbf{e}}^{s}_{uv}:s={\mathcal{T}}(t)\;,\;(u,v,t)\in{\mathcal{E}}\\}$
and the structure of the friendship graph ${\mathcal{G}}$, to generate
temporal user representations
$\\{{\mathbf{h}}_{v}(t)\in{\mathbb{R}}^{D}:v\in{\mathcal{V}}\\}$ at time $t$,
that can be used for friend ranking, e.g., via nearest-neighbor lookup for
candidate retrieval, or friend candidate re-ranking.

####  7.3.2 Background on GNNs

We briefly introduce a generic formulation of a graph neural network layer
with message-passing for neighborhood aggregation.

GNNs use multiple layers to learn node representations. At each layer $l>0$
($l=0$ is the input layer), GNNs compute a representation for node $v$ by
aggregating features from its neighborhood, through a learnable aggregator
$F_{\theta,l}$ per layer. Stacking $k$ layers allows the $k$-hop neighborhood
of a node to influence its representation.

$\mathbf{h}_{v,l}=F_{\theta,l}\Big{(}\mathbf{h}_{v,l-1},\\{\mathbf{h}_{u,l-1}\\}\Big{)},\quad
u\in N(v)$ (7.1)

Equation 7.1 indicates that the node embedding
$\mathbf{h}_{v,l}\in\mathbb{R}^{D}$ for node $v$ at the $l$-th layer is a non-
linear aggregation $F_{\theta,l}$ of the embeddings from layer $l-1$ of node
$v$ and the embeddings of immediate network neighbors $u\in\mathcal{N}(v)$ of
node $v$. The function $F_{\theta,l}$ defines the message passing function at
layer $l$ and can be instantiated using a variety of aggregator architectures,
including graph convolution [39], graph attention [40], and pooling [41]. The
node representation for $v$ at the input layer is $\mathbf{h}_{v,0}$, where
$\mathbf{h}_{v,0}={\mathbf{x}}_{v}$ and ${\mathbf{x}}_{v}\in\mathbb{R}^{D}$.
The representation of node $v$ at the final GNN layer is typically trained
using a supervised objective.

The above formulation (Equation 7.1) operates under the setting of a static
graph and assumes a single static input feature vector for each node. In
contrast, our social network has a time-evolving friendship graph
${\mathcal{G}}$ with pairwise link features and multi-modal node (user
features), which poses unique modeling challenges. In the next section, we
present a graph neural network model for multi-faceted friend ranking.

###  7.4 Graph Neural Friend Ranking

Figure 7.2: Users exhibit different extents of homophily across feature
modalities. (a) Overall modality homophily scores, with 95% confidence
interval bands (b) five representative cluster centroids identified by
clustering users based on their homophily distributions over the $K$
modalities.

In this section, we present our approach to inductively learn user
representations in a dynamic friendship graph with pairwise link features and
multi-modal node features.

We first conduct an empirical analysis on user feature modalities, to gain
insights into various factors impacting friendship formation (Section 7.4.1).
We then formulate the design choices of our model GraFRank for friend ranking
grounded on our acquired insights (Section 7.4.2), followed by model training
details (Section 7.4.3)

####  7.4.1 Motivating Insight: Modality Analysis

We conduct an empirical study that helps us formulate the design choices in
our model. We aim to validate the existence and understand the extent and
variance of attribute homophily with respect to the different user feature
modalities. We begin by analyzing users’ ego-networks to characterize modality
homophily, both overall and broken-down across different user segments. The
definition of modality homophily echoes the standard definition of attribute
homophily [11], but generalized to include a modality of attributes, i.e., the
tendency of users in a social graph to associate with others who are similar
to them along attributes of a certain modality.

We define a homophily measure ${\mathbf{m}}_{vu}^{k}$ between a user $v$ and
her friend $u$ on modality $k$ by the standard cosine similarity [272], which
is a normalized metric that accounts for heterogeneous activity across users.
We compute a modality homophily score ${\mathbf{m}}_{u}^{k}$ for user $v$ on
modality $k$ by the mean over all her neighbors, defined by:

${\mathbf{m}}_{v}^{k}=\frac{1}{|N_{v}|}\sum\limits_{u\in
N_{v}}{\mathbf{m}}_{vu}^{k}\hskip
15.0pt{\mathbf{m}}_{uv}^{k}=cos({\mathbf{x}}^{k}_{u},{\mathbf{x}}^{k}_{v})$
(7.2)

Note that, we omit the snapshot $s$ above since the discussion is restricted
to a single feature snapshot. Figure 7.2 (a) shows the overall modality
homophily scores (averaged across all users), for each of the $K$ modalities.
We observe differing extents of attribute homophily across modalities, with
higher variance for the time-sensitive modalities (e.g., friending and
engagement activities).

We further extend our analysis to examine the homophily distribution over
modalities at the granularity of individual users, to understand if modality
homophily varies across different users. To study this, we first represent
each user $u$ by a $K$-dimensional modality vector
${\mathbf{m}}_{u}=[{\mathbf{m}}_{u}^{1},\dots,{\mathbf{m}}_{u}^{K}]$ that
describes the homophily distribution over the $K$ feature modalities. We then
use $k$-means [273] to cluster the set of all users based on their computed
modality vectors; Figure 7.2 (b) shows the centroid vectors of five
representative clusters identified by $k$-means. We observe stark differences
in the modality vector centroids across the five clusters, indicating the
existence of user segments with diverse homophily distributions over the $K$
modalities. This motivates our first key observation:

######  Observation 7.1 (Heterogeneity in Modality Homophily).

Users exhibit different extents of homophily across feature modalities, and
the homophily distribution over modalities varies significantly across diverse
user segments.

Each modality enables identification of a subset of friends that exhibit
modality homophily. However, this poses a question: do the $K$ modalities
induce the same (or disparate) subsets of homophilous friends, or are the
friends that exhibit homophily in each modality, correlated? We investigate
this relationship in this section.

For every feature modality $k$, we cluster the ego-network (set of direct
friends) $N(v)$ of each user $v\in{\mathcal{V}}$ by representing it as a set
of modality-specific feature vectors
$\\{{\mathbf{x}}^{k}_{u}\in{\mathbb{R}}^{D_{k}}:u\in N(v)\\}$; this results in
ego-clustering assignments for each modality $k$. To quantify cross-modality
correlations, we compute a correlation score for each pair of modalities,
measured by the consensus between their corresponding ego-clustering
assignments.

We use two standard measures: Normalized Mutual Information (NMI) and Adjusted
Rand Index (ARI) to evaluate consensus between clusterings [274]. NMI measures
the statistical correlation between two clustering assignments; however, NMI
increases with the number of distinct clusters. ARI measures the percentage of
correct pairwise assignments, and is chance-corrected with an expected value
of zero. Note that NMI and ARI are symmetric metrics.

Figure 7.3 depicts average NMI and ARI scores for each pair of feature
modalities. We observe substantial correlation in cluster assignments across a
few modalities (e.g., time-sensitive modalities $M_{1}$ and $M_{3}$) while
some (e.g., static modality $M_{0}$) are quite distinct from the rest. Our key
takeaway regarding modality correlation is:

Figure 7.3: Cross-modality Correlation Study: NMI (a) and ARI (b) metrics for
each pair of modalities, quantifying pairwise correlation by consensus in ego-
clustering assignments (obtained independently with respect to each modality).
We observe substantial correlations across pairs of modalities.

######  Observation 7.2 (Cross-Modality Correlation).

There exist non-trivial correlations between pairs of feature modalities,
indicated by the consensus in their induced ego-network clustering of users.

####  7.4.2 GraFRank: Multi-Faceted Friend Ranking

In this section, we first introduce the key components of our model GraFRank
(Graph Attention Friend Ranker) for friend ranking. Our modeling choices in
designing a multi-modal GNN, follow from our observations. GraFRank has three
modules (Figure 7.5):

  * •

Modality-specific neighbor aggregation.

  * •

Cross-modality attention layer.

  * •

Pairwise ranking objective for model training.

Below, we present a detailed description of each module.

##### Modality-specific Neighbor Aggregation

Since the feature modalities vary in the extent of induced homophily
(Observation 7.1), we treat each modality individually as opposed to the
popular choice of directly combining multi-modal user features by
concatenation. Thus, our objective is to learn a modality-specific
representation ${\mathbf{z}}^{k}_{v}(t)\in{\mathbb{R}}^{D}$ for each user
$v\in{\mathcal{V}}$ at time $t\in{\mathbb{R}}^{+}$, that encapsulates
information exclusively from modality $k$. We endow each user $v$ with the
expressivity to flexibly prioritize different friends in her temporal
neighborhood $N_{t}(v)$, hence accounting for the variance in homophily
distribution across user segments.

To achieve this, we design a modality-specific neighbor aggregation module to
compute $K$ representations
$\\{{\mathbf{z}}_{v}^{1}(t),\dots,{\mathbf{z}}_{v}^{K}(t)\\},{\mathbf{z}}_{v}^{k}(t)\in{\mathbb{R}}^{D}$
for each user $v$ at time $t\in{\mathbb{R}}^{+}$, where each
${\mathbf{z}}_{v}^{k}(t)$ is obtained using an independent and unique message-
passing function per modality. We begin by describing a single layer, which
consists of two major operations: message propagation and message aggregation.
We then discuss generalization to multiple successive layers.

Figure 7.4: Friend communication rate in two direct channels (Chat, Snap) on
Snapchat. Most users communicate frequently only with a subset ($\leq 20$%) of
their friends, making influence modeling critical during neighbor aggregation.

Message Propagation: We define the message-passing mechanism to aggregate
information from the ego-network $N_{t}(v)$ of user $v$ at time $t$.
Specifically, the propagation step for modality $k$ aggregates the $k$-th
modality features $\\{{\mathbf{x}}^{s,k}_{u}:u\in
N_{t}(v),s={\mathcal{T}}(t)\\}$ from the corresponding snapshot
$s={\mathcal{T}}(t)$ of temporal neighbors $N_{t}(v)$. To quantify the
importance of each friend $u$ in the ego-network, we propose a friendship
attention [40] which takes embeddings ${\mathbf{x}}^{s,k}_{u}$ and
${\mathbf{x}}^{s,k}_{v}$ as input, and computes an attentional coefficient
$\alpha^{k}(v,u,t)$ to control the influence of friend $u$ on $v$ at time $t$.

$\alpha^{k}(v,u,t)=\text{LeakyRELU}\Big{(}{\mathbf{a}}_{k}^{T}\big{(}{\bm{W}}^{k}_{1}{\mathbf{x}}^{s,k}_{v}\;||\;{\bm{W}}^{k}_{1}{\mathbf{x}}^{s,k}_{u}\big{)}\Big{)}\hskip
10.0pts={\mathcal{T}}(t)$ (7.3)

where ${\bm{W}}^{k}_{1}\in{\mathbb{R}}^{D_{k}\times D}$ is a shared linear
transformation applied to each user, $||$ is the concatenation operation, and
the friendship attention is modeled as a single feed-forward layer
parameterized by weight vector ${\mathbf{a}}_{k}\in{\mathbb{R}}^{2D}$ followed
by the LeakyReLU nonlinearity. We then normalize the attentional coefficients
across all friends connected with $v$ at time $t$ by adopting the softmax
function:

$\alpha^{k}(v,u,t)=\frac{\exp\big{(}\alpha^{k}(v,u,t)\big{)}}{\sum_{w\in
N_{t}(v)}\exp\big{(}\alpha^{k}(v,w,t)\big{)}}$ (7.4)

Figure 7.5: Overall framework of GraFRank: a set of $K$ modality-specific
neighbor aggregators (parameterized by individual modality-specific user
features and link communication features) to compute $K$ intermediate user
representations; cross-modality attention layer to compute final user
representations by capturing discriminative facets of each modality.

We now define an ego-network representation
${\mathbf{z}}^{k}_{v}\big{(}t,N_{v}(t)\big{)}\in{\mathbb{R}}^{D_{k}}$ for user
$v$ in modality $k$ that captures messages propagated from first-order
neighbors in the ego-network $N_{v}(t)$. The message
${\mathbf{m}}^{k}_{v\leftarrow u}\in{\mathbb{R}}^{D}$ propagated from friend
$u$ to user $v$ at time $t$ is defined as the transformed friend embedding,
i.e., ${\mathbf{m}}^{k}_{v\leftarrow
u}={\bm{W}}^{k}_{1}{\mathbf{x}}^{s,k}_{v}$. We then compute
${\mathbf{z}}^{k}_{v}\big{(}t,N_{v}(t)\big{)}$ through a weighted average of
message embeddings from each friend $u\in N_{v}(t)$ and guided by normalized
friendship weights $\alpha^{k}(u,v,t)$, which is defined as:

${\mathbf{z}}^{k}_{v}\big{(}t,N_{v}(t)\big{)}=\sum_{u\in
N_{v}(t)}\alpha^{k}(v,u,t){\mathbf{m}}^{k}_{v\leftarrow u}$ (7.5)

While the above equation specifies different contributions to friends, these
weights are learnt merely based on the connectivity structure of the ego-
network. In reality, most users only have a few close friends, and users with
many friends only frequently communicate with a few of them. To empirically
validate this hypothesis, we compute the friend communication rate of all
users, defined by the percentage of friends that a user has directly
communicated with at least once (directly sent a Chat/Snap with on Snapchat)
in a one-month window. From Figure 7.4, we find that a vast majority of users
communicate primarily with a small percentage (10-20%) of their friends; thus,
we posit that friendship activeness is critical to precisely model user
affinity.

Towards this goal, we incorporate pairwise link communication features to
parameterize both the attentional coefficients and the message aggregated from
friends in the ego-network. Specifically, we formulate the message
${\mathbf{m}}^{k}_{v\leftarrow u}\in{\mathbb{R}}^{D}$ from friend $u$ to user
$v$ at time $t$ as a function of both friend feature ${\mathbf{x}}^{s,k}_{u}$
and link feature ${\mathbf{e}}^{s}_{vu}$.

${\mathbf{m}}^{k}_{v\leftarrow
u}={\bm{W}}^{k}_{2}{\mathbf{x}}^{s,k}_{u}+{\bm{W}}^{k}_{e}{\mathbf{e}}^{s}_{vu}+{\mathbf{b}}\qquad
s={\mathcal{T}}(t)$ (7.6)

where ${\bm{W}}^{k}_{2}\in{\mathbb{R}}^{D_{k}\times
D},{\bm{W}}^{k}_{e}\in{\mathbb{R}}^{E\times D}$ are trainable weight matrices
operating on the user and link features respectively, and
${\mathbf{b}}\in{\mathbb{R}}^{D}$ is a learnable bias vector. The attentional
co-efficient $\alpha^{k}(v,u,t)$ is then computed as a function of user
feature ${\mathbf{x}}_{v}^{s,k}$ and message embedding
${\mathbf{m}}^{k}_{v\leftarrow u}\in{\mathbb{R}}^{D}$ from friend $u$ to user
$v$, defined by:

$\alpha^{k}(v,u,t)=\sigma\bigg{(}{\mathbf{a}}_{k}^{T}\Big{(}{\bm{W}}^{k}_{1}{\mathbf{x}}^{s,k}_{v}\;||\;{\mathbf{m}}^{k}_{v\leftarrow
u}\Big{)}\bigg{)}\qquad s={\mathcal{T}}(t)$ (7.7)

where $\sigma$ is a non-linearity such as LeakyRELU. We similarly normalize
the attentional coefficients $\alpha^{k}(v,u,t)$ using Equation 7.4 and
compute the ego-network representation ${\mathbf{z}}^{k}_{v}(t,N_{v}(t))$ for
user $v$ on modality $k$ using Equation 7.5 with the message embedding
${\mathbf{m}}^{k}_{v\leftarrow u}$ from Equation 7.6 conditioned on link
features.

Distinct from conventional graph convolution and attention networks [39, 41,
5, 40] that only consider user ${\mathbf{x}}^{s,k}_{v}$ and friend
${\mathbf{x}}^{s,k}_{v}$ features to parameterize the neighbor aggregation, we
additionally encode the link communication features ${\mathbf{e}}^{s}_{vu}$
into the attentional co-efficient $\alpha^{k}(v,u,t)$ (Equation 7.7) and the
message ${\mathbf{m}}^{k}_{v\leftarrow u}$ (Equation 7.6) passed from friend
$u$; this encourages the aggregation to be cognizant of the pairwise
communications with friends, e.g., passing more messages from the active
friendships. In addition, we observe a boost in friend ranking performance
(Section 7.5.4) owing to our novel communication-aware message passing
strategy.

Message Aggregation: We refine the latent representation for user $v$ by
aggregating the messages propagated from friends in $N_{v}(t)$. In addition to
the messages from $N_{v}(t)$, we additionally consider the self-connection of
$v$, i.e., ${\mathbf{m}}^{k}_{v\leftarrow
v}={\bm{W}}^{k}_{1}{\mathbf{x}}_{v}^{s,k}$ to preserve knowledge of the
original features (${\bm{W}}_{1}$ is the same transformation used in Equation
7.3). Specifically, we concatenate the ego-network and self-representations of
user $v$, and further transform the concatenated embedding through a dense
layer $F_{\theta}^{k}$, defined by:

$\displaystyle{\mathbf{z}}^{k}_{v}(t)\quad$ $\displaystyle=\quad
F^{k}_{\theta}\Big{(}{\mathbf{m}}^{k}_{v\leftarrow
v}\;,{\mathbf{z}}^{k}_{v}\big{(}t,N_{v}(t)\big{)}\Big{)}$ (7.8)
$\displaystyle=\quad\sigma\bigg{(}{\bm{W}}_{a}^{k}\Big{(}{\mathbf{z}}^{k}_{v}(t,N_{v}(t))\;||\;{\mathbf{m}}^{k}_{v\leftarrow
v}\Big{)}+{\mathbf{b}}_{a}\bigg{)}\;\;$ (7.9)

where ${\bm{W}}_{a}^{k}\in{\mathbb{R}}^{D\times D}$,
${\mathbf{b}}_{a}\in{\mathbb{R}}^{D}$ are trainable weight parameters of the
message aggregator, and $\sigma$ denotes the ELU activation function, which
allows messages to encode both positive and small negative signals.
Empirically we observe significant gains owing to retaining knowledge of self-
representation via concatenation, instead of directly using the propagated
ego-network representation from the neighborhood, as in GCN [39], and GAT
[40].

Higher-order Propagation: We stack multiple aggregation layers to model high-
order connectivity information, i.e., propagate features from $l$-hop
neighbors. The inputs to layer $l$ depend on the user representations output
from layer $(l-1)$ where the initial (i.e., “layer 0”) representations are set
to the input user features in modality $k$. By stacking $l$ layers, we
recursively formulate the representation ${\mathbf{z}}^{k}_{v,l}$ of user $v$
at the end of layer $l$ by:

${\mathbf{z}}^{k}_{v,l}=F^{k}_{\theta,l}\Big{(}{\mathbf{m}}^{k}_{v\leftarrow
v,l-1},z^{k}_{v,l-1}\big{(}t,N_{v}(t)\big{)}\Big{)}\quad{\mathbf{m}}^{k}_{v\leftarrow
v,l-1}=z^{k}_{v,l-1}$ (7.10)

where $z^{k}_{v,l-1}$ is the representation of user $v$ in modality $k$ after
$(l-1)$ layers and $z^{k}_{v,l-1}(t,N_{v}(t))$ denotes the $(l-1)$-ego-network
representation of user $v$. We apply $L$ neighbor aggregation layers to
generate the layer-$L$ representation ${\mathbf{z}}^{k}_{v,L}$ of user $v$ in
modality $k$.

##### Cross Modality Attention

To learn complex non-linear correlations across feature modalities, we design
a cross-modality attention mechanism. Specifically, we learn modality
attention weights $\beta^{k}_{v}(t)$ to distinguish the influence of each
modality $k$ using a two-layer Multi-Layer Perceptron, by:

$\beta^{k}_{v}(t)=\frac{\exp\big{(}{\mathbf{a}}^{T}_{m}\;{\bm{W}}_{m}{\mathbf{z}}^{k}_{v,L}+b_{m}\big{)}}{\sum_{k^{\prime}=1}^{K}\exp\big{(}{\mathbf{a}}^{T}_{m}\;{\bm{W}}_{m}{\mathbf{z}}^{k^{\prime}}_{v,L}+b_{m}\big{)}}$
(7.11)

with weights ${\bm{W}}_{m}\in{\mathbb{R}}^{D\times
D},{\mathbf{a}}_{m}\in{\mathbb{R}}^{D}$ and scalar bias $b_{m}$. The final
representation ${\mathbf{h}}_{v}(t)\in{\mathbb{R}}^{D}$ of user $v$ is
computed by a weighted aggregation of the layer-$L$ modality-specific user
representations $\\{{\mathbf{z}}^{1}_{v,L},\dots,{\mathbf{z}}^{K}_{v,L}\\}$,
guided by modality weights $\beta^{k}_{v}(t)$, defined by:

${\mathbf{h}}_{v}(t)=\sum\limits_{k=1}^{K}\beta^{k}_{v}(t)\;{\bm{W}}_{m}{\mathbf{z}}^{k}_{v,l}$
(7.12)

####  7.4.3 Model Training

We train GraFRank using a pairwise ranking objective to differentiate positive
and negative neighborhoods. We assume access to training data described by a
set of timestamped links ${\mathcal{L}}$ created in a time span
$(t_{s},t_{e})$, where $(v,u,t)\in{\mathcal{L}}\;$ indicates a bi-directional
link between source user $u$ and target friend $v$ formed at time
$t\in(t_{s},t_{e})$. To learn the parameters of GraFRank, we define a triplet
like training objective based on max-margin ranking.

##### Pairwise Ranking Objective

We define a time-sensitive ranking loss over the user embeddings
(${\mathbf{h}}_{v}(t)$ for user $v$ at time $t$) to rank the inner product of
positive links $(v,u,t)\in{\mathcal{L}}$, higher than sampled negatives
$(v,n,t)$ by a margin $\Delta$, as:

$L=\sum\limits_{(u,v,t)\in{\mathcal{L}}}\mathbb{E}_{n\sim
P_{n}(v)}\max\\{0,{\mathbf{h}}_{v}(t)\cdot{\mathbf{h}}_{n}(t)-{\mathbf{h}}_{v}(t)\cdot{\mathbf{h}}_{u}(t)+\Delta\\}$
(7.13)

where $\Delta$ is a margin hyper-parameter and $P_{n}(v)$ is the negative
sampling distribution for user $v$. This generic contrastive learning
formulation enables usage of the same framework to learn user embeddings for
different recommendation tasks such as candidate retrieval and ranking, with
diverse negative sampling distributions.

##### Friend Candidate Retrieval and Ranking Tasks

We learn user embeddings towards two key use-cases in friend ranking:
candidate retrieval and candidate ranking. Candidate retrieval aims to
generate a list of top-$N$ (e.g., $N=100$) potential friend suggestions out of
a very large candidate pool (over millions of users), while candidate ranking
involves fine-grained re-ranking within a much smaller pool of the generated
candidates, to determine the top-$n$ (e.g., $n=10$) suggestions shown to end
users in the platform. We define different negative sampling distributions
$P_{n}(v)$ for each task owing to their different ranking granularties, as
follows:

Candidate Retrieval: For the coarse-grained task of candidate retrieval, we
uniformly sample five random negative users for each positive link, from the
entire user set ${\mathcal{V}}$. Generating random negatives is efficient and
effective at quickly training the model to identify potential friend
candidates for each user. However, random negatives are often too easy to
distinguish and may not provide the requisite resolution for the model to
learn fine-grained distinctions necessary for candidate friend ranking.

Candidate Ranking: To enhance model resolution for candidate ranking, we also
consider hard negative examples for each positive pair $(u,v)$ that are
related to the source $u$, but not as relevant as the target friend $v$. For
each user $u$, we generate five hard negatives based on graph proximity.
Specifically, users in $k$-hop ($3\leq k\leq 4$) graph neighborhoods of $u$
(that are not within the 2-hop neighborhood), are randomly sampled as hard
negatives. In practice, we pre-compute a set of hard negatives for each user,
to facilitate efficient negative sampling during model training.

We adopt a two-phase learning approach for candidate ranking. We pre-train the
model on random negatives (as in candidate retrieval), to identify good model
initialization points, followed by fine-tuning on hard negatives. Ranking hard
negatives is more challenging, hence encouraging the model to progressively
learn friend distinctions at a finer granularity. We empirically show notable
gains on candidate ranking due to our two-phase strategy, compared to training
individually on random or hard negatives.

##### Temporal Neighborhood Sampling

We learn a temporal user representation ${\mathbf{h}}_{v}(t)$ for user $v$ at
time $t$ by selecting a fixed number of friends from $N_{v}(t)$ for neighbor
aggregation at each layer; this controls the memory footprint during training
[41].

To efficiently identify and sample neighbors of $v$ at any time $t$, we
represent the time-evolving friendship graph ${\mathcal{G}}$ as a temporal
adjacency list at its latest time $t_{s}$ where each user $v$ has a list of
(friend, time) tuples sorted by link creation times. This data representation
enables $O(\log d)$ neighbor lookup at an arbitrary timestamp $t$ via binary
search where $d$ is the average user degree in the graph.

##### Multi-GPU Minibatch Training

We train GraFRank with minibatches of links from ${\mathcal{L}}$ using
multiple GPUs on a single shared memory machine. The temporal adjacency list
of ${\mathcal{G}}$ and feature matrices ${\bm{X}},{\bm{E}}$ are placed in
shared CPU memory to enable fast parallel neighborhood sampling and feature
lookup. We adopt a producer-consumer architecture [5] that alternates between
CPUs and GPUs for model training. A CPU-bound producer constructs friend
neighborhoods, looks up user and link features, and generates negative samples
for the links of a minibatch. We then partition each minibatch across multiple
GPUs, to compute forward passes and gradients with a PyTorch model over
dynamically constructed computation graphs. The gradients from different GPUs
are synchronized using PyTorch’s Distributed Data Parallel [275].

###  7.5 Experiments

To analyze the quality of user representations learned by GraFRank, we propose
five research questions to guide our experiments:

  1. ($\textbf{RQ}_{\textbf{1}}$)

Can GraFRank outperform feature-based models and state-of-the-art GNNs on
candidate retrieval and ranking tasks?

  2. ($\textbf{RQ}_{\textbf{2}}$)

How does GraFRank compare to prior work under alternative metrics of
reciprocated and communicated friendships?

  3. ($\textbf{RQ}_{\textbf{3}}$)

How do different architectural design choices and training strategies in
GraFRank impact performance?

  4. ($\textbf{RQ}_{\textbf{4}}$)

How do training strategies and hyper-parameters impact convergence and
performance of GraFRank?

  5. ($\textbf{RQ}_{\textbf{5}}$)

How do the learned user embeddings in GraFRank perform across diverse user
cohorts?

Dataset |  Region 1 |  Region 2
---|---|---
# users |  3.1 M |  17.1 M
# links |  286 M |  2.36 B
# user features |  79 |  79
# link features |  6 |  6
# test set friend requests |  46K |  340K
Table 7.1: Dataset statistics

####  7.5.1 Experimental Setup

We now present our experimental setup with a brief description of datasets,
evaluation metrics, and model training details.

##### Datasets

We evaluate the user representations learned by GraFRank for friend
recommendations, on two large-scale datasets from Snapchat. Each dataset is
constructed from the interactions among Snapchat users belonging to a specific
country (details obscured for privacy reasons). In total, we collect 79 user
features spanning four feature modalities and six pairwise link communication
features, as described in Section 7.3. All features are standardized using
zero-mean and unit-variance normalization before model training. We define the
set ${\mathcal{L}}$ of positive training examples (Equation 7.13) using all
timestamped friendships created over a span of one week. Since our objective
is to evaluate the quality of friendship recommendations, the test set
comprises all friend add requests generated by users over the next four days.
We use 10% of the labeled examples as a validation set for hyper-parameter
tuning. Table 7.1 shows the detailed statistics of the two datasets from
Snapchat.

##### Evaluation Metrics

We experiment on two friend recommendation tasks: candidate retrieval and
candidate ranking, as outlined in Section 7.4.3. To evaluate friend
recommendation, we use standard ranking metrics Hit-Rate (HR@K), Normalized
Discounted Cumulative Gain (NDCG@K) and Mean Reciprocal Rank (MRR). We adopt
the standard negative-sample evaluation [88] to generate $N$ negative samples
per positive pair of users $(u,v)$ in the test set (user $u$ has sent a friend
request to user $v$). We then compute ranking metrics for each test pair
$(u,v)$ by ranking $v$ among the $N$ negative samples via inner products in
the latent space.

To evaluate candidate retrieval, we use $N=10000$ randomly sampled negative
users for each positive test pair, to emulate retrieval from a large candidate
pool. Ideally, candidate ranking should operate over a shortlisted list of
potential friends identified by the retrieval system. However, we aim to
provide a fair benchmark comparison of different models for candidate ranking
that is agnostic to the biases of the upstream retrieval system. Thus, we
instead generate $N=500$ hard negatives samples per test pair based on $K$-hop
neighborhoods (Section 7.4.3), to ensure an unbiased comparison.

##### Training Details

We train GraFRank using $L=2$ message passing layers per feature modality with
a hidden dimension size of 64 and output embedding dimension $D=32$. In each
layer, we sample 15 first-order neighbors and 15 second-order neighbors for
each sampled first-order neighbor; each user receives messages propagated from
upto 225 friends. During model training, we apply dropout with rate of 0.3 in
the two aggregation layers. The model is trained for a maximum of 30 epochs
with a batch size of 256 positive pairs (apart from 5 random/hard negatives
per pair) and learning rate of 0.001 using Adam optimizer. We benchmark our
experiments using a machine with 32 cores, 200 GB shared CPU memory, and a
single Nvidia Tesla P100 GPU with PyTorch [275] implementations on the Linux
platform.

####  7.5.2 Baselines

We compare the performance of GraFRank on friend ranking against strong
feature-based machine learning models, and state-of-the-art graph neural
networks that learn user representations.

  * •

LogReg [276]: Logistic regression classifier for link prediction. The input
feature for each pair of users, is a concatenation of source and target
features across the $K$ user feature modalities.

  * •

XGBoost [277]: Tree boosting model for pairwise learning to rank, trained
using the same input features as LogReg. It is currently deployed at Snapchat
for quick-add friend suggestions.

  * •

MLP [276]: Two-layer Multi-Layer Perceptron with fully-connected layers and
ReLU activations to learn user representations. We train MLP using the same
ranking loss as our model (Equation 7.13).

  * •

GCN [39]: Graph convolutional networks with degree-weighted neighbor
aggregation and neighborhood sampling for scalable training. For each user, we
concatenate features across the $K$ user feature modalities into a new nodal
feature vector.

  * •

GAT [40]: Graph attention networks with self-attentional aggregation and
neighborhood sampling for scalable training.

  * •

SAGE + Max [41]: Element-wise max pooling for neighbor aggregation and self-
embedding concatenation at each layer.

  * •

SAGE + Mean [41]: Same as SAGE + Max with element-wise mean pooling function
for neighbor aggregation.

Note that graph autoencoders [278] and graph convolutional matrix completion
models [279] are not empirically comparable because they cannot scale to our
large-scale social network datasets.

For each baseline, we train separate models for candidate retrieval and
ranking tasks. We use random negatives for retrieval model training while
resorting to hard negatives for ranking; we empirically find training separate
models to be vastly superior to a training a single model using a mixture of
random and hard negatives, or even curriculum training [5].