# Understanding the Limitations of Diffusion Concept Algebra Through Food

E. Zhixuan Zeng Yuhao Chen Alexander Wong
University of Waterloo
{ezzeng, yuhao.chen1<EMAIL_ADDRESS>
###### Abstract

Image generation techniques, particularly latent diffusion models, have
exploded in popularity in recent years. Many techniques have been developed to
manipulate and clarify the semantic concepts these large-scale models learn,
offering crucial insights into biases and concept relationships. However,
these techniques are often only validated in conventional realms of human or
animal faces and artistic style transitions. The food domain offers unique
challenges through complex compositions and regional biases, which can shed
light on the limitations and opportunities within existing methods. Through
the lens of food imagery, we analyze both qualitative and quantitative
patterns within a concept traversal technique. We reveal measurable insights
into the model’s ability to capture and represent the nuances of culinary
diversity, while also identifying areas where the model’s biases and
limitations emerge.

##  1 Introduction

In recent years, latent diffusion models have gained significant traction,
heralding a new era of remarkably high-quality generated images. With this,
considerable effort has been made toward identifying better ways to examine,
disentangle, and traverse through higher-level concepts learned during
training. Recent works include those focused on image concept manipulation and
control [1, 2, 3, 4], model unlearning [5, 6], concept disentanglement [7, 8,
9, 10], along with ways to better understand the underlying concept
representation, such as through concept decomposition [11], or identifying
semantic directions [12, 13, 14, 15, 16, 17, 18]. These advancements not only
enhance control over the model’s outputs but also reveal hidden biases and
relationships within the concepts that are learned. However, their evaluation
is often limited to a narrow set of commonly studied areas like human and
animal faces [12, 13, 14, 15, 16, 17, 18] or artistic style transfer [18].
This narrow focus limits our understanding of their behaviour when applied to
more complex or less structured data.

Enter the domain of food imagery — a vastly under-explored area in the context
of latent diffusion models. Food presents an intriguing case study for several
reasons. First, it often involves compositions of multiple subjects, such as
several dishes on a table or plates arranged with a main dish, side dishes,
and even decorative arrangements. Second, some foods can exhibit strong
regional influences, with distinct appearances and dishes in different
cuisines. These influences can provide a convenient avenue for understanding
how the model deals with biases and entanglement between food dishes and
regions.

The diversity in food types — from regional specialties to various preparation
methods — also introduces a high degree of variability in visual
characteristics. These characteristics, along with the universal familiarity
with food, makes it an ideal candidate for exploring the limitations of
semantic directions and concept decomposition techniques.

In this paper, we explore one recent method, called Concept Algebra [18],
within the context of food image generation. We will focus on identifying
limitations, particularly in the context of causal separability [18] and
feature representation, and propose metrics useful for model understanding and
prompt engineering.

##  2 Background

Semantic directions refers to vectors on some learned latent space
corresponding to a semantic concept, where adding or subtracting that vector
would result in meaningful semantic change. This idea has existed for many
years, with the famous example being “King - male + female = Queen" [19].

Concept Algebra [18] is a recent technique applying semantic directions to
diffusion models for controlled image generation. To make it easier to find
these semantic directions, concepts are separated into different “sets", each
with a projection into a subspace. Traversal through this subspace maximizes
changes relevant to the target “set" of concepts, while minimizing changes in
other concepts.

For example, given two concept sets $\mathcal{Z}$ and $\mathcal{W}$ referring
to “style" and “content" respectively, an image may be composed of a “style"
concept Z and a “content" concept W (eg. “a mall in photorealistic style").
Identifying a direction in the “style" subspace corresponding to a change from
“photorealism" to “painting" would allow the user to generate a “painting"
styled image, while maintaining the same content as the original prompt (eg.
“a mall in painted style").

To identify the subspace for $\mathcal{Z}$, a set of prompts varying in
$\mathcal{Z}$ but constant in $\mathcal{W}$ is used. The constant content
concept ($\text{W}_{0}$) should be easy to generate for all $\mathcal{Z}$. The
model evaluates the latent vector conditioned on each prompt. The top $K$
eigenvectors are identified and used as the unit vectors for the “style"
subspace $\mathcal{R}_{Z}$. $\text{proj}_{z}$ defines projection from the
latent space into the “style" subspace. The image edit can then be performed
through:

$s_{edit}\leftarrow(\mathbb{I}-\text{proj}_{z})s[x_{\text{orig}}]+\text{proj}_{z}s[x_{\text{new}}]$
(1)

where $s[x]$ is the latent vector at each time step conditioned on a prompt
$x$, $x_{\text{orig}}$ is a prompt containing target “content" concept
($\text{W}_{t}$) and “style" concept $\text{Z}_{0}$ that pairs easily with
$\text{W}_{t}$, while $x_{\text{new}}$ contains some target “style" concept
($\text{Z}_{t}$) and the same $\text{W}_{0}$ used in calculating the “style"
subspace. $\mathbb{I}$ is the identity matrix. The edit ideally subtracts the
“style" in $x_{\text{orig}}$ and adds in the “style" in $x_{\text{new}}$,
without changing the “content" from $x_{\text{orig}}$.

##  3 Experimental Setup

Our goal is to explore Concept Algebra’s limitations in the context of food
imagery and discover if the concept subspace offers any benefits beyond image
editing. Our experiments primarily focus on two concept sets: “cuisine region"
as “style" concepts and “food ingredients" as “content" concepts. Other
“style" sets, such as “fast-food" vs “fine-dining" and food preparation
techniques (eg. fried, boiled) is also explored visually.

Subspace Visualization and Clustering: First, we visualize the projection of
many different prompt results into the “style" subspace (Figure 1c). These
prompts vary in both “content" and “style".

Figure 1: Prompts containing different food ingredients and cuisine regions
plotted in a “cuisine regions" subspace. (a) and (c) shows the vectors colored
by ingredients and region respectively. (b) and (d) shows the same vectors
after normalizing by the mean and variance of each ingredient distribution.

We define a cluster of subspace vectors as a collection of outputs conditioned
on the same “style". Because the subspace should maximize changes only in
“style", we hypothesize that intra-cluster distances should ideally be closer
together than inter-cluster distances. We also hypothesize that inter-cluster
distances may provide meaningful information on the perceived similarity
between each “style" (ie. Chinese cuisine may be closer to Japanese Cuisine
than American cuisine). Therefore, after collecting the subspace vectors, we
calculate the inter-cluster distances using the Jensen-Shannon distance [20],
a distribution distance measure based on KL divergence. We further evaluate
clustering quality with the silhouette score [21], a measure comparing intra-
cluster distance with inter-cluster distance. The score varies in range [-1,
1], where a higher score means tighter, more distinct clusters.

Causal Separability: The original work [18] states causal separability as a
prerequisite for the technique, meaning the way the two sets of concepts
effect the final output must not depend fundamentally on some interaction
between the two concepts. However, this definition is not truly clear-cut.
Concept entanglement is a well-observed phenomenon in diffusion models [7, 8,
9], and learned biases may cause concepts which are theoretically causally
separable difficult to separate in practice.

In our experiments, we observed that a change in ingredient may result in a
similar change in the region subspace vector across all regions (Fig. 1a). We
hypothesize that a larger, more consistent change in the “style" subspace
given “content" concepts suggests limited causal separability, while a weaker
correlation to changes in “content" concepts suggests stronger causal
separation. This can be visualized by normalizing the subspace vectors by the
mean and variance of each ingredient distribution (Fig. 1b and Fig. 1d). The
change can also be measured by the change in silhouette score before and after
normalization.

##  4 Observations

Figure 2: Jensen-Shannon distance, a distribution distance metric, between
different cuisine region clusters

###  4.1 Measurable Observations

Subspace Visualization and Clustering: An example visualization of the
projected subspace vectors can be seen in Fig. 1c, with the Jensen-Shannon
distances between each cluster visualized in Fig. 2. , where we see that the
cluster containing “Chinese cuisine" is very close to the cluster for “Asian
cuisine". This shows that, using these subspace vectors, we may be able to
easily identify some biases and relationships between categories in the
concept set.

Causal Separability: Through testing, we found that the difference in
Silhoutte Score before and after normalization can change significantly based
on prompt wording. Figure 3 shows two sets of experiments: one generated using
prompts formatted as “ingredient in Region cuisine" (eg. “food in Chinese
cuisine"), and the second using “food in region cuisine made with ingredient"

Figure 3: Comparing the result of different prompt wording. Visually, the
first row has limited variation in composition and the cut and shape of the
chicken compared to the second row. Numerically, silhouette scores in the
first row are 0.33 and 0.61 for non-normalized and normalized subspace vectors
respectively, while the second row is 0.61 and 0.63

With the same “content" and “style" concepts, the silhouette scores for the
first prompt format is 0.33 before normalization and 0.61 afterwards, while
the second prompt is 0.61 and 0.63 for non-normalized and normalized subspace
vectors respectively. This much smaller difference shows a significant
decrease in the influence “ingredients" have in this new “cuisine region"
subspace.

To further explore this phenomenon, instead of generic ingredients common to
all cuisines, we select foods with strong regional biases as “content"
concepts, including stirfry, sushi, bread, pasta, dimsum, curry, and tacos.
Given the strong regional biases, we expect that changes in “content" would
heavily influence the “cuisine region" subspace. This is verified, as when we
attempted to transform “bread" into “Japanese cuisine", we ended up with
silhouette scores of -0.17 and 0.54 for non-normalized and normalized subspace
vectors respectively.

We also observed examples where the silhouette scores were low both before and
after normalization. When we set the “style" concepts to be different cooking
techniques (eg. deep-fried, steamed, baked, raw), we saw very little variation
in the image outputs between the different categories. Correspondingly, the
clusters in the “cooking techniques" subspace were not very distinct, with
“Raw chicken" receiving a silhouette score of 0.11 both before and after
normalization. This suggests that the poor image results may not be due to
separability issues, but perhaps the space itself is not very expressive.

To summarize: Large difference between normalized and non-normalized
silhouette scores in the “style" subspace suggests issues with causal
separability. Tracking silhouette scores can aid in finding better prompt
wording. Low silhouette scores results in lower variation in generation
outputs and poor image results.

###  4.2 Visual Observations

Our visual analysis within the food imagery domain has also yielded insightful
observations. In the “cooking techniques" example (Fig. 4a, 4b), we observed
that oftentimes, the food itself was not transformed to fit its “boiled" or
“poached" characteristics. Instead, peripheral elements within the image
changed. This is not an isolated occurrence. For instance, in for Asian
cuisine, chopsticks are often added, along with small dishes of other common
“asian food" such as rice (Fig 4c).

(a) “Poached Chicken"

(b) “Boiled Chicken"

(c) “food in Chinese cuisine made using chicken"

Figure 4: Examples of style changes where peripheral elements rather than the
main element were changed significantly

This observation is difficult to uncover in other common domains of
exploration, as those domains either usually contain only a single subject
(eg. people), or requires changing the entire image to take effect (artistic
styles). In contrast, food images often contains multiple subjects, but does
not require changing the entire art style to move from one cuisine region to
another.

Furthermore, our exploration revealed a tendency of diffusion models to
prioritize alterations in the visual features of an image over the accuracy of
the subject’s transformation into a proper food item. A striking example of
this can be observed when Concept Algebra is used to transform “bread in
western cuisine" into Japanese cuisine. The result is an image where sushi
appears with a top texture reminiscent of bread (fig. 5b). Similarly, a
sandwich transformed into Japanese cuisine sees the bread being repurposed as
a plate for sushi (fig. 5c). Chicken transformed into fast-food has also
yielded a chicken-shaped burger (fig. 5a).

(a) “chicken in fast food"

(b) “bread in Japanese cuisine"

(c) “sandwich in Japanese cuisine"

Figure 5: Examples where the texture of the original ingredient was placed in
the form of some “default" food item in the target “style" concept

Two problems may be contributing to this phenomenon. First, this demonstrates
a higher focus on textural features as opposed to semantic meaning. This is
potentially due to using post-unet features as the latent space to construct
the “style" subspace. Future work may look into experimenting with vectors at
the middle of the U-net, as done in [14, 13, 12], or CLIP space [11, 22].

Second, the prompts used in constructing the “style" subspace use a single
prompt per region. As such, those regions may become overly represented by the
result of that single prompt. Existing biases (eg. Japanese cuisine = sushi)
may be amplified the use of the single exemplar. It is also possible that the
model has learned only a very limited representation of the concept and cannot
express greater variation.

##  5 Conclusion

Through applying concept algebra to food imagery, we’ve revealed patterns and
metrics unique to its complexities. We quantified distances between cuisine
clusters, identifying important biases and relationships between categories.
Through measuring cluster distinctiveness, we can estimated separability
between two concept sets of interest. By identifying improvements in those
metrics, we can aid in prompt engineering through quantitatively measuring
prompt performance.

Visually, we found a model bias towards textural over semantic accuracy,
leading to transformations that reveal strong biases in the model, potentially
emphasized by the concept algebra technique. Instances where peripheral
elements in images changed, instead of the food items themselves, reveal
another area of improvement.

Concept Algebra, and many other similar techniques exploring a model’s latent
representation, are vital to developing a more nuanced, accurate, and
culturally comprehensive understanding of the large-scale generative models in
use today. However, the limitations of these techniques can only be better
understood through exploring diverse testing domains. The nuanced exploration
of the food imagery domain promises to enrich future research, advancing the
depth and accuracy of generative models in capturing complex subjects.

## References

  * Gal et al. [2022] Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-or.  An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion.  In _The Eleventh International Conference on Learning Representations_ , September 2022.
  * Gandikota et al. [2023a] Rohit Gandikota, Joanna Materzynska, Tingrui Zhou, Antonio Torralba, and David Bau.  Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models, November 2023a.
  * Avrahami et al. [2023] Omri Avrahami, Kfir Aberman, Ohad Fried, Daniel Cohen-Or, and Dani Lischinski.  Break-A-Scene: Extracting Multiple Concepts from a Single Image.  In _SIGGRAPH Asia 2023 Conference Papers_ , pages 1–12, December 2023.  doi: 10.1145/3610548.3618154.
  * Liu et al. [2023] Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B. Tenenbaum.  Compositional Visual Generation with Composable Diffusion Models, January 2023.
  * Kumari et al. [2023] Nupur Kumari, Bingliang Zhang, Sheng-Yu Wang, Eli Shechtman, Richard Zhang, and Jun-Yan Zhu.  Ablating Concepts in Text-to-Image Diffusion Models.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 22691–22702, 2023.
  * Gandikota et al. [2023b] Rohit Gandikota, Joanna Materzynska, Jaden Fiotto-Kaufman, and David Bau.  Erasing Concepts from Diffusion Models.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 2426–2436, 2023b.
  * Jin et al. [2024] Xin Jin, Bohan Li, BAAO Xie, Wenyao Zhang, Jinming Liu, Ziqiang Li, Tao Yang, and Wenjun Zeng.  Closed-Loop Unsupervised Representation Disentanglement with $\beta$-VAE Distillation and Diffusion Probabilistic Feedback, February 2024.
  * Li et al. [2024] Senmao Li, Joost van de Weijer, Taihang Hu, Fahad Shahbaz Khan, Qibin Hou, Yaxing Wang, and Jian Yang.  Get What You Want, Not What You Don’t: Image Content Suppression for Text-to-Image Diffusion Models, February 2024.
  * Motamed et al. [2023] Saman Motamed, Danda Pani Paudel, and Luc Van Gool.  Lego: Learning to Disentangle and Invert Concepts Beyond Object Appearance in Text-to-Image Diffusion Models, November 2023.
  * Rassin et al. [2023] Royi Rassin, Eran Hirsch, Daniel Glickman, Shauli Ravfogel, Yoav Goldberg, and Gal Chechik.  Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment.  _Advances in Neural Information Processing Systems_ , 36, December 2023.
  * Chefer et al. [2023] Hila Chefer, Oran Lang, Mor Geva, Volodymyr Polosukhin, Assaf Shocher, Michal Irani, Inbar Mosseri, and Lior Wolf.  The Hidden Language of Diffusion Models, October 2023.
  * Park et al. [2023] Yong-Hyun Park, Mingi Kwon, Junghyo Jo, and Youngjung Uh.  Unsupervised Discovery of Semantic Latent Directions in Diffusion Models, February 2023.
  * Li et al. [2023] Hang Li, Chengzhi Shen, Philip Torr, Volker Tresp, and Jindong Gu.  Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation, November 2023.
  * Haas et al. [2023] René Haas, Inbar Huberman-Spiegelglas, Rotem Mulayoff, and Tomer Michaeli.  Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Models, March 2023.
  * Brack et al. [2023] Manuel Brack, Felix Friedrich, Dominik Hintersdorf, Lukas Struppek, Patrick Schramowski, and Kristian Kersting.  SEGA: Instructing Text-to-Image Models using Semantic Guidance, November 2023.
  * Dalva and Yanardag [2023] Yusuf Dalva and Pinar Yanardag.  NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models, December 2023.
  * Kwon et al. [2022] Mingi Kwon, Jaeseok Jeong, and Youngjung Uh.  Diffusion Models already have a Semantic Latent Space, 2022.
  * Wang et al. [2023] Zihao Wang, Lin Gui, Jeffrey Negrea, and Victor Veitch.  Concept Algebra for (Score-Based) Text-Controlled Generative Models.  _Advances in Neural Information Processing Systems_ , 36, December 2023.
  * Mikolov et al. [2013] Tomas Mikolov, Wen-tau Yih, and G. Zweig.  Linguistic Regularities in Continuous Space Word Representations.  In _North American Chapter of the Association for Computational Linguistics_ , May 2013.
  * Lin [1991] Jianhua Lin.  Divergence measures based on the shannon entropy.  _IEEE Transactions on Information theory_ , 37(1):145–151, 1991.
  * Rousseeuw [1987] Peter J Rousseeuw.  Silhouettes: a graphical aid to the interpretation and validation of cluster analysis.  _Journal of computational and applied mathematics_ , 20:53–65, 1987.
  * Oldfield et al. [2023] James Oldfield, Christos Tzelepis, Yannis Panagakis, Mihalis Nicolaou, and Ioannis Patras.  Parts of Speech–Grounded Subspaces in Vision-Language Models.  _Advances in Neural Information Processing Systems_ , 36, December 2023.


# Thinking inside The Box: Learning Hypercube Representations for Group
Recommendation

Tong Chen  The University of Queensland<EMAIL_ADDRESS>,  Hongzhi Yin
The University of Queensland<EMAIL_ADDRESS>,  Jing Long  The University of
Queensland<EMAIL_ADDRESS>,  Quoc Viet Hung Nguyen  Griffith University
<EMAIL_ADDRESS>,  Yang Wang  Hefei University of
Technology<EMAIL_ADDRESS>and  Meng Wang  Hefei University of Technology
<EMAIL_ADDRESS>
(2022)

###### Abstract.

As a step beyond traditional personalized recommendation, group recommendation
is the task of suggesting items that can satisfy a group of users. In group
recommendation, the core is to design preference aggregation functions to
obtain a quality summary of all group members’ preferences. Such user and
group preferences are commonly represented as points in the vector space
(i.e., embeddings), where multiple user embeddings are compressed into one to
facilitate ranking for group-item pairs. However, the resulted group
representations, as points, lack adequate flexibility and capacity to account
for the multi-faceted user preferences. Also, the point embedding-based
preference aggregation is a less faithful reflection of a group’s decision-
making process, where all users have to agree on a certain value in each
embedding dimension instead of a negotiable interval. In this paper, we
propose a novel representation of groups via the notion of hypercubes, which
are subspaces containing innumerable points in the vector space. Specifically,
we design the hypercube recommender (CubeRec) to adaptively learn group
hypercubes from user embeddings with minimal information loss during
preference aggregation, and to leverage a revamped distance metric to measure
the affinity between group hypercubes and item points. Moreover, to counteract
the long-standing issue of data sparsity in group recommendation, we make full
use of the geometric expressiveness of hypercubes and innovatively incorporate
self-supervision by intersecting two groups. Experiments on four real-world
datasets have validated the superiority of CubeRec over state-of-the-art
baselines.

Group Recommendation; Hypercube Representations; Self-supervised Learning

††journalyear: 2022††copyright: acmcopyright††conference: Proceedings of the
45th International ACM SIGIR Conference on Research and Development in
Information Retrieval; July 11–15, 2022; Madrid, Spain††booktitle: Proceedings
of the 45th International ACM SIGIR Conference on Research and Development in
Information Retrieval (SIGIR ’22), July 11–15, 2022, Madrid, Spain††price:
15.00††doi: 10.1145/3477495.3532066††isbn: 978-1-4503-8732-3/22/07††ccs:
Information systems Recommender systems

##  1\. Introduction

Group activities are natural habitats for humans, and there is no exception in
e-commerce. For example, friends can sign up for the same music festivals on
Facebook Events, and online platforms like Meetup allow users to form groups
and host activities (Yin et al., 2020). As traditional recommender systems
only target at suggesting relevant items to individuals, there is an urgent
demand in generating recommendations for a group of users, referred to as
group recommendation. With the increasingly strong synergy between social and
commercial functionalities in contemporary e-commerce platforms (e.g., Yelp,
Gowalla, Steam, etc.) (Yin and Cui, 2016), there has been a recent surge in
developing solutions to group recommendation (Hu et al., 2014; Cao et al.,
2018; Sankar et al., 2020).

Similar to conventional recommendation, latent factor models exhibit dominant
performance in group recommendation. In a general sense, groups and items are
mapped to points in the vector space (i.e., embeddings), then the pairwise
group-item affinity can be estimated via intuitive distance/similarity
metrics. As such, current group recommenders share a core motivation of
learning representative group-level embeddings to capture the joint
preferences of users. In rare cases where group-item interactions are dense,
group representations can be directly learned via standard collaborative
filtering models by substituting users with groups (Yin et al., 2019).
However, the real group activities are highly sparse (Yin et al., 2020; Sankar
et al., 2020), restricting the informativeness of the learned group
representations and hindering recommendations for new groups. Thus, a common
practice is to aggregate a group’s representation from its members’ user
embeddings, which are usually learned from the richer user-item interactions.
Such aggregation strategies have evolved from heuristics like the least misery
and maximum pleasure (Amer-Yahia et al., 2009), to the more recent attentive
sum (Cao et al., 2018) and social influence-based approach (Yin et al., 2019).

Despite the increasing complexity of these recent preference aggregation
approaches, the learned group representations still face a major bottleneck in
their expressiveness. This is mainly due to the amount of information lost
when compressing multiple user embeddings into one. In the context of
embeddings, a group/user is represented as a point in the vector space, and
each entry is the exact coordinate in the corresponding dimension. However,
while each individual user’s preference tend to be stable and can be denoted
by a certain value in each embedding dimension, the whole group’s overall
preference reflects a blend of multiplex personal interests, challenging the
capacity of point embeddings to accomodate such diversity. For example, if we
associate the embedding dimensions with explicit item features, a group of
users may land on different preferred values on the “price” dimension. But, it
quickly becomes problematic when using a single point for the entire group due
to the lack of considerations for such disparate and diverse user preferences,
which commonly exist in web applications (Nguyen et al., 2017; Li et al.,
2021).

In some cases, though higher dimensionality can be assigned to the group
embedding to encode more information and alleviate the problem, this does not
resolve the issue that a single embedding is unable to precisely reflect the
decision-making process within a user group. Essentially, aggregating user
preferences mimics the decision-making process (Guo et al., 2020) where all
group members reach a consensus, reflected by a group-level representation. In
reality, the group-level representation should involve synergies of common
interests as well as compromises on some personal tastes. For example, for two
users that have a large discrepancy on preferred prices, it is more realistic
for them to agree upon an approximate price range in between, rather than an
exact value. This, unfortunately, cannot be facilitated by existing
aggregation schemes where all user preferences are merged into a single point.
A point group embedding forces all users to agree upon a fixed decision in
each dimension, no matter its value is simply averaged from all users (Amer-
Yahia et al., 2009) or biased towards more influential group members (Yin et
al., 2019; Cao et al., 2018).

To this end, we aim to represent the group-level preferences in a more
expressive way, such that the multi-faceted user demands are thoroughly
encoded, and the intricate decision-making process among all group members are
faithfully preserved. Hence, we advocate addressing the restricted capacity of
learned group representations by seeking an alternative to the outgoing point
embeddings. Specifically, we introduce the notion of hypercubes as a new way
to represent the aggregated group-level preferences in the vector space. In
each dimension of the vector space, while a point embedding only has a fixed
value, a hypercube covers a range of values that span across a continuous
interval (Ren et al., 2020). Combining the intervals will shape a “cube”-like
subspace, enabling a hypercube to accommodate innumerable points in the vector
space (Zhang et al., 2021b). Compared with the traditional group embeddings,
hypercubes enable a higher tolerance of different user preferences.
Furthermore, as each dimension now encodes a preference interval, this
paradigm is a more realistic reflection of how a group of distinct users give
concessions to each other and eventually work out a combined criteria for
desired items.

However, further challenges are in place when unleashing the full potential of
hypercube group representations. Firstly, as traditional aggregation schemes
are dedicated to learning groups’ embedding representations, a new paradigm is
desired to effectively summarize all group members’ vectorized preferences
into a hypercube with minimal information loss in user interests. Secondly,
the utilization of group hypercubes voids the applicability of existing point-
wise distance measures like Euclidean distance or cosine similarity for group-
item pairs, highlighting the need for a revamped distance metric that can
precisely quantify and distinguish the affinity between different hypercubes
(i.e., groups) and points (i.e., items). Thirdly, the long-standing problem of
severe data sparsity with group-level interactions (Yin et al., 2020; Sankar
et al., 2020) still persists in our case, which will impede the quality of
learned group hypercube representations and amplify the risk of overfitting.

To address these issues, we present our solution to group recommendation,
namely the hypercube recommender (CubeRec). CubeRec inherits the classic point
embeddings for individual users/items whose preferences/attributes are
deterministic and can be learned from the relatively rich individual-level
interactions. Meanwhile, we innovatively represent each group with the more
expressive hypercube, which is first composed by the embeddings of its user
members, and then optimized through observed group-level interactions. In
CubeRec, we propose two alternative strategies to merge individual user
embeddings into group hypercubes, specifically a geometric bounding approach
that physically encapsulates all group members’ points, and an attentive
approach that composes hypercubes by investigating the semantics within user
embeddings. To allow for accurate item ranking, we put forward a distance
function to measure the distance between each item point and group hypercube.
The distance metric accounts for distances from both the inside and outside of
a hypercube, thus accomodating different scenarios and maintaining a
discriminative characteristic. Furthermore, we make full use of the geometry
of group hypercubes and propose a novel self-supervised learning paradigm to
tackle the data sparsity in group recommendation. In short, we define an
intersection operation between two group representations, and leverage the
common users shared between two groups as supervision signals to enrich the
information encoded within every hypercube.

In summary, the main contributions of our work are:

  * •

We define a new schema based on hypercubes to represent group-level
preferences for group recommendation. Hypercubes bypass the limited
expressiveness and flexibility of conventional point embeddings, allowing the
group representations to encode multi-faceted user preferences while fully
imitating a group’s decision-making process.

  * •

We propose CubeRec, a group recommender that comes with novel designs for
learning group-level hypercubes and measuring group-to-item distances. To
alleviate data sparsity, CubeRec further incorporates self-supervision by
innovatively utilizing the intersection between groups.

  * •

We conduct extensive experiments on four real-world benchmark datasets.
Experimental results show that CubeRec yields superior recommendation
performance compared with state-of-the-art baselines.

##  2\. Preliminaries

In this section, we mathematically define the hypercube representations of
groups, which are the key building block of CubeRec.

One can think of a hypercube as extending a rectangle into a higher-
dimensional space, where each edge of the hypercube corresponds to a real-
valued, closed interval on each dimension (Ren et al., 2020; Zhang et al.,
2021b). For group $\mathcal{G}$ which is a subset of users, we define its
representation $\mathbb{G}$ via the following center-offset format:

(1)
$\mathbb{G}\equiv\\{\textbf{g}\in\mathbb{R}^{d}:\textbf{c}-\textbf{o}\preceq\textbf{g}\preceq\textbf{c}+\textbf{o}\\},$

where $\textbf{c}\in\mathbb{R}^{d}$ is the geometric center of the hypercube,
and $\textbf{o}\in\mathbb{R}^{d}_{\geq 0}$ is the non-negative offset vector.
Both the center and offset are $d$-dimensional vectors learned within the same
embedding space. Essentially, in the $z$-th dimension of $\mathbb{R}^{d}$,
$\mathbb{G}$ will have its $z$-th edge spanning across
$[c_{z}-o_{z},c_{z}+o_{z}]$. For notation convenience, we define the hypercube
representation for the $n$-th group $\mathcal{G}_{n}$ as a tuple
$\mathbb{G}_{n}=(\textbf{c}_{n},\textbf{o}_{n})$ with its group-specific
center $\textbf{c}_{n}$ and offset $\textbf{o}_{n}$. We use this tuple as a
shorthand for Eq.(1) in the rest parts of our paper. Compared with point
embeddings, relaxing the fixed value in each dimension into a flexible range
brings an increased tolerance on the variety among group members. It also
reflects a more realistic decision process, as the group does not have to pre-
approve a specific value per dimension before ranking candidate items.

##  3\. The Hypercube Recommender

To provide an overview, CubeRec first learns user and item embeddings based on
user-level interactions with items. Then, we propose our solution to building
and learning hypercube representations at the group level. To further
compensate for the high sparsity of group-item interactions when learning
hypercube representations, we introduce a novel learning objective by
leveraging self-supervision signals from overlapping groups. In what follows,
we unfold the design details of CubeRec.

###  3.1. Learning User and Item Embeddings

In CubeRec, we employ LightGCN (He et al., 2020b) to learn user and item
embeddings. Compared with earlier collaborative filtering (CF) methods,
adopting LightGCN facilitates the modelling of higher-order collaborative
signals through message passing, and offers simplicity via the omission of
excessive nonlinear components (Wang et al., 2019). We keep this part brief as
it is not our core contribution and can actually be replaced by arbitrary
latent factor models.

Modelling User-Item Interactions. Treating each user/item as a distinct node,
the user-item interactions can be formulated as a bipartite graph, where we
let $\mathcal{N}(u_{i})$ and $\mathcal{N}(v_{j})$ denote the one-hop neighbor
sets of user $u_{i}$ and item $v_{j}$, respectively. The embedding of an
arbitrary user/item is updated by propagating its neighbors’ embeddings into
$u_{i}$/$v_{j}$:

(2)
$\textbf{u}^{(l)}_{i}=\\!\\!\\!\\!\sum_{j\in\mathcal{G}(u_{i})}\eta_{ij}\textbf{v}^{(l-1)}_{j},\,\,\,\textbf{v}^{(l)}_{j}=\\!\\!\\!\\!\sum_{i\in\mathcal{G}(v_{j})}\eta_{ij}\textbf{u}^{(l-1)}_{i},$

where $l\\!\leq\\!L$ denotes the propagation layer,
$\eta_{ij}\\!=\\!(\sqrt{|\mathcal{G}(u_{i})|\\!\cdot\\!|\mathcal{G}(v_{j})|})^{-1}$
is a graph Laplacian normalization term (He et al., 2020b; Kipf and Welling,
2017), while $\textbf{u}^{(0)}_{i}$ and $\textbf{v}^{(0)}_{j}$ are randomly
initialized and will be trained via back-propagation. Following (He et al.,
2020b), the final embeddings are obtained via mean pooling across all layers,
i.e.,
$\textbf{u}_{i}\\!=\\!\frac{1}{l+1}\sum_{l=0}^{L}\textbf{u}^{(l)}_{i}\\!\in\\!\mathbb{R}^{d}$
and
$\textbf{v}_{j}\\!=\\!\frac{1}{l+1}\sum_{l=0}^{L}\textbf{v}^{(l)}_{j}\\!\in\\!\mathbb{R}^{d}$,
which are used in the subsequent recommendation phases.

Handling Explicit Social Ties. Obviously, increasing $L$ can help capture
higher-order relationships including implicit social relationships like user-
item-user paths, as $L$ essentially controls the number of hops to be reached
during the neighborhood aggregation. As group recommendation services are
commonly provided on social platforms, the explicitly available social
connections between users (e.g., following relationship on social media) carry
strong signals for understanding a user’s preferences. However, the plain
LightGCN does not account for such explicit social ties. In CubeRec, we use
$\textbf{R}\in\\{0,1\\}^{|\mathcal{U}|\times|\mathcal{V}|}$ to denote the
interactions between users from set $\mathcal{U}$ and items from set
$\mathcal{V}$, and $\textbf{S}\in\\{0,1\\}^{|\mathcal{U}|\times|\mathcal{U}|}$
to denote the social relation adjacency matrix of all users. Then, we
formulate the adjacency matrix of the user-item graph as the following:

(3) $\textbf{A}=\begin{bmatrix}\textbf{S}&\textbf{R}\\\
\textbf{R}^{\top}&\textbf{0}\end{bmatrix},$

where 0 is a $|\mathcal{V}|\times|\mathcal{V}|$ matrix consisting of $0$s, and
the resulted adjacency matrix
$\textbf{A}\in\\{0,1\\}^{(|\mathcal{U}|+|\mathcal{V}|)\times(|\mathcal{U}|+|\mathcal{V}|)}$
now carries both user-item interactions and user-user social ties. With that,
we summarize the socially enhanced LightGCN in the matrix form:

(4)
$\textbf{E}^{(l)}=(\textbf{D}^{-\frac{1}{2}}\textbf{A}\textbf{D}^{-\frac{1}{2}})\textbf{E}^{(l-1)},$

where
$\textbf{D}\in\mathbb{N}^{(|\mathcal{U}|+|\mathcal{V}|)\times(|\mathcal{U}|+|\mathcal{V}|)}$
is a diagonal degree matrix in correspondence to $\mathbf{A}$, while
$\textbf{E}\in\mathbb{R}^{(|\mathcal{U}|+|\mathcal{V}|)\times d}$ stacks all
user and item embeddings.

User-level Recommendation Loss. We optimize all user and item embeddings with
a distance-based objective on user-level interactions, namely the hinge loss
(Chen et al., 2018; Hsieh et al., 2017):

(5)
$L_{user}=\sum_{(u_{i},v_{j^{+}},v_{j^{-}})\in\mathcal{D}_{user}}\\!\\!\\!\\!\\!\\!\max(0,\,\lambda+d_{ij^{+}}-d_{ij^{-}}),$

where $\mathcal{D}_{user}$ denotes the training set, $\lambda$ is the safety
margin size to be predefined, and
$d_{ij}=||\textbf{u}_{i}-\textbf{v}_{j}||_{2}^{2}$ is the squared Euclidean
distance between the embeddings of $u_{i}$ and $v_{j}$. The rationale of the
hinge loss is that, in the vector space, each user $u_{i}$ should aways be
closer to a visited item $v_{j^{+}}$ than an unvisited one $v_{j^{-}}$. As
such, a training sample is defined as a triple $(u_{i},v_{j^{+}},v_{j^{-}})$,
where every observed user-item pair $(u_{i},v_{j^{+}})$ is matched with a
fixed amount of uniformly sampled negative items $v_{j^{-}}$ to construct
$\mathcal{D}_{user}$.

(a) Geometric fusion and projection for hypercube learning |  (b) Attentive fusion and projection for hypercube learning |  (c) Group-to-item distances for different item points |  (d) Our goal with self-supervised learning via hypercube intersection
---|---|---|---

Figure 1. A schematic view of the key hypercube-based computations in CubeRec.
We use $\mathbb{R}^{2}$ for demonstration purpose. Corresponding details can
be found in Section 3.2 for (a) and (b), Section 3.3 for (c), and Section 3.4
for (d).

###  3.2. Composing Group-level Hypercubes

We perform pretraining with $L_{user}$ to acquire initial user and item
embeddings for making recommendations at the group level. As a common
practice, a group’s representation can be learned by merging all the
embeddings of its members via means like attention networks (Cao et al., 2018;
Huang et al., 2020) or graph convolution (Wang et al., 2020; Guo et al.,
2021), but they inevitably incur limited expressiveness. As discussed earlier,
compressing users’ multi-faceted preferences into a single point in the vector
space leads to significant information loss, and also misaligns with a group’s
decision-making process in the real world.

In this section, we present our solution to learning expressive and flexible
group representations with hypercubes. Specifically, we introduce two
different strategies for composing group-level hypercubes using individual
users’ point embeddings, of which both have their unique properties and are
further testified in Section 4.

Geometric Bounding and Projection. Given $s$ user embeddings
$\textbf{u}_{1},\textbf{u}_{2},...,\textbf{u}_{s}\in\mathbb{R}^{d}$ in group
$\mathcal{G}_{n}$, the geometric bounding operation is an intuitive way to
generate the corresponding group-level hypercube. As Figure 1(a) shows, we
first need to find the smallest hypercube covering all $s$ points (i.e., user
embeddings) from $\mathcal{G}_{n}$. Such a hypercube is computed as the
following:

(6)
$\widetilde{\mathbb{G}}_{n}=(\,\widetilde{\textbf{c}}_{n},\widetilde{\textbf{o}}_{n})=(\frac{\textbf{u}^{max}+\textbf{u}^{min}}{2},\frac{|\textbf{u}^{max}-\textbf{u}^{min}|}{2}),$

where $\textbf{u}^{max}$ and $\textbf{u}^{max}$ are boundary vectors computed
via:

(7)
$\vspace{-0.1cm}\begin{split}\textbf{u}^{max}&=\max(\textbf{u}_{1},\textbf{u}_{2},...,\textbf{u}_{s}),\\\
\textbf{u}^{min}&=\min(\textbf{u}_{1},\textbf{u}_{2},...,\textbf{u}_{s}),\end{split}$

where $\max(\cdot)$ and $\min(\cdot)$ both operate element-wise. However,
straightforwardly utilizing $\widetilde{\mathbb{G}}_{n}$ as the final group
representation is suboptimal due to its geometric property. That is, the more
diverse that the users in $\mathcal{G}_{n}$ are, the larger area that
$\widetilde{\mathbb{G}}_{n}$ will cover in the $d$-dimensional space. As the
hypercube shares the same vector space with items, the recommendation will
naturally prefer items that falls in the hypercube because it means those
items meet all the preference intervals on all latent dimensions.
Consequently, a highly diverse group will receive an excessively large
hypercube, covering more false positive items that eventually dilute the
recommendation accuracy. To alleviate this issue, we learn projection matrices
for both the center and offset that help rescale $\widetilde{\mathbb{G}}_{n}$
into the final group hypercube $\mathbb{G}_{n}$:

(8)
$\mathbb{G}_{n}=(\textbf{c}_{n},\textbf{o}_{n})=(\textbf{W}_{c}\widetilde{\textbf{c}}_{n},\textbf{W}_{o}\widetilde{\textbf{o}}_{n}),$

where $\textbf{W}_{c}\in\mathbb{R}^{d\times d}$ and
$\textbf{W}_{o}\in\mathbb{R}^{d\times d}_{\geq 0}$ are the corresponding
projection weights. Note that $\textbf{W}_{o}$ is non-negative such that
$\textbf{o}_{n}\geq 0$.

Attentive Fusion and Projection. Geometric bounding aims to let a hypercube
fairly cover all users’ preferences within a group. While this is reasonable
in many cases, recent studies also point out that each user in the group may
affect the final group decision to a different extent (Cao et al., 2018; Yin
et al., 2019) due to varied social influence and/or expertise. To make our
learned group representations able to account for every group member’s
importance discrepancy, we further propose an attentive approach (Vaswani et
al., 2017; Chen et al., 2020a) for composing group hypercubes:

(9)
$\vspace{-0.1cm}\widetilde{\textbf{c}}_{n}=\textbf{W}_{V}\textbf{U}_{n}\cdot\mathrm{softmax}\Big{(}\frac{(\textbf{W}_{K}\textbf{U}_{n})^{\top}\cdot\textbf{q}}{\sqrt{d}}\Big{)},$

where $\textbf{q}\in\mathbb{R}^{d}$ is the learnable query vector in the above
self-attention, $\textbf{W}_{K},\textbf{W}_{V}\in\mathbb{R}^{d\times d}$ are
respectively the key and value projection matrices, and
$\textbf{U}_{n}\in\mathbb{R}^{d\times s}$ stacks all $d$-dimensional user
embeddings in group $\mathcal{G}_{n}$. As each $\textbf{u}\in\textbf{U}_{n}$
reflects a user’s exact interest, the aggregation in Eq.(9) attentively
aggregates all group members’ interests, denoted by
$\widetilde{\textbf{c}}_{n}$. Then, we locate the centroid and offset of the
composed hypercube $\mathbb{G}_{n}$ via:

(10)
$\mathbb{G}_{n}=(\textbf{c}_{n},\textbf{o}_{n})=(\textbf{W}_{c}\widetilde{\textbf{c}}_{n},\,\textnormal{ReLU}(\textbf{W}_{o}\widetilde{\textbf{c}}_{n}+\textbf{b}_{o})\,),$

where we slightly abuse notations
$\textbf{W}_{c},\textbf{W}_{o}\in\mathbb{R}^{d\times d}$ to denote the
learnable matrices, and $\textbf{b}_{o}\in\mathbb{R}^{d}$ is the bias vector.
Since the attentively aggregated $\widetilde{\textbf{c}}_{n}$ summarizes the
core interest of the group, we directly use its linear projection as the
center $\textbf{c}_{n}$. Meanwhile, the process of obtaining
$\widetilde{\textbf{c}}_{n}$ sacrifices users’ preference diversity, which
should be preserved by the offset. Thus, we employ the more expressive
nonlinearity to partially infer and recover such missing information from
$\widetilde{\textbf{c}}_{n}$ and obtain the offset $\textbf{o}_{n}$. The
rectified linear unit (ReLU) is used to ensure non-negativity. This process is
depicted by Figure 1(b). In short, the attentive approach focuses more on the
semantics of user embeddings compared with its geometric counterpart that
retains more physical meanings, and is relatively more efficient.

###  3.3. Learning Hypercubes for Group Recommendation

To optimize the composed hypercube representations, we firstly put forward a
distance function that quantifies the affinity between each pair of group and
item, and then introduce how we define the loss function at the group level
for learning CubeRec.

Computing Item-to-Group Distances. Compared with conventional point group
embeddings, each edge of a hypercube defines a range on the corresponding
dimension in the latent feature space rather than a single value. Intuitively,
for any item point $\textbf{v}_{j}$ that falls in the hypercube
$\mathbb{G}_{n}$, it means each dimension of $\textbf{v}_{j}$ satisfies the
preference range specified by $\mathbb{G}_{n}$, making item $v_{j}$ a good fit
for group $\mathcal{G}_{n}$. However, it is not always the case in practice.
On one hand, a diverse group tend to have a relatively large coverage with the
generated hypercube, containing more irrelevant items. On the other hand, a
group with relatively narrow interests can have a rather small hypercube,
making it hard to cover any existing item embeddings, especially when $d$ is
large. In light of this, we propose an approach that measures the item-to-
group distance $d_{nj}$ via:

(11) $d_{nj}=f_{out}(\mathbb{G}_{n},\textbf{v}_{j})+\gamma
f_{in}(\mathbb{G}_{n},\textbf{v}_{j}),$

where functions $f_{out}(\cdot,\cdot)$ and $f_{in}(\cdot,\cdot)$ measure the
outer and inner item-to-group distances, respectively, which are balanced by a
fixed coefficient $\gamma\geq 0$. Taking the group hypercube $\mathbb{G}_{n}$
and item embedding $\textbf{v}_{j}$ as inputs, these two functions are
formulated as follows using squared Euclidean distance:

(12)
$\begin{split}f_{out}(\mathbb{G}_{n},\textbf{v}_{j})&=||\max(\textbf{v}_{j}-\textbf{g}^{\triangleright}_{n},\textbf{0})+\max(\textbf{g}^{\triangleleft}_{n}-\textbf{v}_{j},\textbf{0})||_{2}^{2},\\\
f_{in}(\mathbb{G}_{n},\textbf{v}_{j})&=||\textbf{c}_{n}-\min(\textbf{g}^{\triangleright}_{n},\max(\textbf{g}^{\triangleleft}_{n},\textbf{v}_{j}))||_{2}^{2},\\\
\end{split}$

where two points
$\textbf{g}^{\triangleleft}_{n},\textbf{g}^{\triangleright}_{n}\in\mathbb{R}^{d}$
are geometrically the lower-left and upper-right corners of hypercube
$\mathbb{G}_{n}$, respetively:

(13)
$\textbf{g}^{\triangleleft}_{n}=\textbf{c}_{n}-\textbf{o}_{n},\,\,\,\,\,\textbf{g}^{\triangleright}_{n}=\textbf{c}_{n}+\textbf{o}_{n}.$

As explained in Figure 1(c), assuming $\textbf{v}_{j}$ locates outside
$\mathbb{G}_{n}$, what Eq.(12) does is to: (1) locate the anchor point on
$\mathbb{G}_{n}$’s surface that is the closest to $\textbf{v}_{j}$; (2) use
$f_{out}(\cdot,\cdot)$ to measure the outer distance from $\textbf{v}_{j}$ to
this anchor point; and (3) use $f_{in}(\cdot,\cdot)$ to measure the inner
distance from the centroid $\textbf{c}_{n}$ to this anchor point. The outer
and inner distances are then combined into $d_{nj}$ via Eq.(11). When
$\textbf{v}_{j}$ is inside or on the surface of $\mathbb{G}_{n}$,
$f_{out}(\mathbb{G}_{n},\textbf{v}_{j})=0$, and
$f_{in}(\mathbb{G}_{n},\textbf{v}_{j})$ will reduce to the distance between
item and centroid because
$\min(\textbf{g}^{\triangleright}_{n},\max(\textbf{g}^{\triangleleft}_{n},\textbf{v}_{j}))=\textbf{v}_{j}$.
This is also shown in Figure 1(c).

Group-level Recommendation Loss. Following the definition of item-to-group
distance in Section 3.2, the higher the affinity between group
$\mathcal{G}_{n}$ and item $v_{j}$, the smaller the distance $d_{nj}$ will be.
Analogously to Eq.(5), we adopt the following hinge loss:

(14)
$L_{group}=\sum_{(\mathcal{G}_{n},v_{j^{+}},v_{j^{-}})\in\mathcal{D}_{group}}\\!\\!\\!\\!\\!\\!\max(0,\,\lambda^{\prime}+d_{nj^{+}}-d_{nj^{-}}),$

where $\lambda^{\prime}$ denotes the safety margin,
$(u_{i},v_{j^{+}},v_{j^{-}})\in\mathcal{D}_{group}$ are training instances
constructed by negative sampling. Essentially, for cube $\mathbb{G}_{n}$,
$L_{group}$ aims to pull a positive item’s embedding $\textbf{v}_{j^{+}}$ into
$\mathbb{G}_{n}$ and closer to its center, while pushing the negative item’s
embeddings $\textbf{v}_{j^{-}}$ away from it.

###  3.4. Self-supervision via Hypercube Intersections

Unfortunately, as pointed out by prior studies (Yin et al., 2019; Sankar et
al., 2020), a long-lasting obstacle for learning high-quality group
representations is that, the learning of group representations is purely
dependant on the interacted items, which are highly sparse at the group level
(see statistics in Section 4.1). As such, we propose to take advantage of the
rich self-supervision signals from user-group associations for learning
representative group hypercubes. Specifically, for group $\mathcal{G}_{n}$ we
firstly draw a different group $\mathcal{G}_{n^{\prime}}$, where
$\mathcal{G}_{n^{\prime}}$ shares at least one user in common with
$\mathcal{G}_{n}$ (details on how we deal with disjoint groups will follow).
Users shared between $\mathcal{G}_{n}$ and $\mathcal{G}_{n^{\prime}}$ are
termed relay users. Then intuitively, the intersection of two group hypercubes
denoted by $int(\mathbb{G}_{n},\mathbb{G}_{n^{\prime}})$, which is also a
hypercube, is supposed to reflect the properties of all the relay users.

With Figure 1(d), we illustrate our goal of self-supervised learning in
CubeRec. In a geometric sense, two hypercubes’ intersection is ideally the
subspace that hosts the point embeddings of relay users. On this basis, we
formulate the self-supervision loss below:

(15) $L_{self}=\sum_{\forall\mathcal{G}_{n}}\,\sum_{\begin{subarray}{c}\forall
u_{i^{+}}\in(\mathcal{G}_{n}\cap\mathcal{G}_{n^{\prime}})\\\ u_{i^{-}}\sim
p(u_{i^{-}}|\mathcal{G}_{n},\mathcal{G}_{n^{\prime}})\end{subarray}}\\!\\!\\!\\!\\!\\!\max(0,\,\lambda^{\prime\prime}+d_{nn^{\prime}i^{+}}-d_{nn^{\prime}i^{-}}),$

where $d_{nn^{\prime}i}$ is a shorthand for the distance between the
intersection and user $u_{i}$, i.e.,
$f_{out}(int(\mathbb{G}_{n},\mathbb{G}_{n^{\prime}}),\textbf{u}_{i})+\gamma
f_{in}(int(\mathbb{G}_{n},\mathbb{G}_{n^{\prime}}),\textbf{u}_{i})$. For each
positive user $u_{i^{+}}$ that falls in the group intersection
$\mathcal{G}_{n}\cap\mathcal{G}_{n^{\prime}}$, we sample a negative user
$u_{i^{-}}$ from
$\mathcal{U}\backslash(\mathcal{G}_{n}\\!\cap\\!\mathcal{G}_{n^{\prime}})$
with uniform distribution
$p(u_{i^{-}}|\mathcal{G}_{n},\mathcal{G}_{n^{\prime}})=|\mathcal{U}\backslash(\mathcal{G}_{n}\\!\cap\\!\mathcal{G}_{n^{\prime}})|^{-1}$.
By doing so, the representations of groups $\mathcal{G}_{n}$ and
$\mathcal{G}_{n^{\prime}}$ will be regularized by the relay users, so as to
capture the common interests between them. Correspondingly, the learned group
representations will be substantially more informative than those learned
purely with sparse item-level interactions.

Hypercube Intersection Operation. We hereby define how the hypercube
intersection $int(\mathbb{G}_{n},\mathbb{G}_{n^{\prime}})$ is calculated. For
convenience, we let
$int(\mathbb{G}_{n},\mathbb{G}_{n^{\prime}})=\mathbb{G}_{nn^{\prime}}=(\textbf{c}_{nn^{\prime}},\textbf{o}_{nn^{\prime}})$.
Because straightforwardly using the geometric intersection is only applicable
when two hypercubes have actual overlaps in each dimension of
$\mathbb{R}^{d}$, we resort to a relaxed intersection computation via a neural
approach. Firstly, center $\textbf{c}_{nn^{\prime}}$ is computed by performing
element-wise attention over the two group centers:

(16)
$\vspace{-0.1cm}\begin{split}\textbf{c}_{nn^{\prime}}&=\textbf{a}_{n}\odot\textbf{c}_{n}+\textbf{a}_{n^{\prime}}\odot\textbf{c}_{n^{\prime}},\\\
\textbf{a}_{*}&=\frac{\exp(\phi(\textbf{c}_{*}))}{\exp(\phi(\textbf{c}_{n}))+\exp(\phi(\textbf{c}_{n^{\prime}}))}\,\,\,\textnormal{for}\,*\in\\{n,n^{\prime}\\},\end{split}$

where $\odot$ denotes element-wise multiplication,
$\phi(\cdot):\mathbb{R}^{d}\mapsto\mathbb{R}^{d}$ is a multilayer perceptron
(MLP) with $d$ neurons throughout all layers. Meanwhile, the offset
$\textbf{o}_{nn^{\prime}}$ is determined via:

(17)
$\textbf{o}_{nn^{\prime}}=\min(\textbf{o}_{n},\textbf{o}_{n^{\prime}})\odot\textnormal{sigmoid}(\psi(\textbf{o}_{n}+\textbf{o}_{n^{\prime}})),$

where $\psi(\cdot):\mathbb{R}^{d}\mapsto\mathbb{R}^{d}$ is another MLP with a
distinct set of parameters. Essentially, Eq.(16) places the intersection
center $\textbf{c}_{nn^{\prime}}$ somewhere between $\textbf{c}_{n}$ and
$\textbf{c}_{n^{\prime}}$. Then, with the scaling effect of sigmoid function,
Eq.(17) shrinks the offsets of hypercubes $\mathbb{G}_{n}$ and
$\mathbb{G}_{n^{\prime}}$ to obtain the new offset of the intersection
$\textbf{o}_{nn^{\prime}}$.

Handling Isolated Groups. Despite the heavily overlapping nature of user
groups on social e-commerce platforms, we must take into account the situation
where a group $\mathcal{G}_{n}$ might be disjoint with all other groups in the
dataset. On this occasion, we propose the two following strategies to generate
a dummy user group $\widetilde{\mathcal{G}}_{n^{\prime}}$ having overlapping
users with $\mathcal{G}_{n}$:

  * I.

Proportional Swap (PS): With a specified proportion $\rho$, we swap
$\rho|\mathcal{G}_{n}|$ users in $\mathcal{G}_{n}$ with users uniformly
sampled from $\mathcal{U}\setminus\mathcal{G}_{n}$ with replacement. We denote
this as
$\widetilde{\mathcal{G}}_{n^{\prime}}=\textnormal{PS}(\mathcal{G}_{n})$.

  * II.

Proportional Imputation (PI): With a specified proportion $\rho$, we inject
$\rho|\mathcal{G}_{n}|$ uniformly sampled users from
$\mathcal{U}\setminus\mathcal{G}_{n}$ (with replacement) into
$\mathcal{G}_{n}$. We denote this as
$\widetilde{\mathcal{G}}_{n^{\prime}}=\textnormal{PI}(\mathcal{G}_{n})$.

The generated $\widetilde{\mathcal{G}}_{n^{\prime}}$ then serves as
$\mathcal{G}_{n^{\prime}}$ to facilitate self-supervision via Eq.(15). We
adopt a predefined value $\rho=0.5$ in our approach, where we use PS and PI
alternately during training.

Algorithm 1 Training Procedure of CubeRec

1:Input: $\mathcal{U}$, $\mathcal{G}$, $\mathcal{V}$

2:Output: All model parameters collectively referred to as $\Theta$

3:Randomly initialize $\Theta$;

4:repeat

5: Draw a mini-batch from $\mathcal{D}_{user}$;

6: Take a gradient step to update u and v w.r.t. $L_{user}$;

7:until convergence$\triangleright$ Pretraining user and item embeddings

8:repeat

9: Draw a mini-batch from $\mathcal{D}_{group}$ and compute $L_{group}$;

10: for each $\mathcal{G}_{n}$ in mini-batch do

11: if $\exists\mathcal{G}_{n^{\prime}}$ s.t.
$\mathcal{G}_{n}\cap\mathcal{G}_{n^{\prime}}\neq\varnothing$ then

12: Compute $L_{self}$ with a sampled $\mathcal{G}_{n^{\prime}}$;

13: else

14: Generate $\widetilde{\mathcal{G}}_{n^{\prime}}$ via either
$\textnormal{PS}(\mathcal{G}_{n})$ or $\textnormal{PI}(\mathcal{G}_{n})$ by
coin-flipping;

15: Compute $L_{self}$ with $\widetilde{\mathcal{G}}_{n^{\prime}}$;

16: Take a gradient step to update $\Theta$ w.r.t. $L_{group}+\mu L_{self}$ ;

17:until convergence

###  3.5. Optimizing CubeRec

As all components of CubeRec are end-to-end differentiable, we learn the model
parameters on multiple objectives with the mini-batch stochastic gradient
descent algorithm Adam (Kingma and Ba, 2015). As depicted by Algorithm 1, we
adopt a two-stage training strategy for CubeRec. To be specific, we first
obtain pretrained user and item embeddings by optimizing the user-level loss
$L_{user}$. Then, we fine-tune the pretrained embeddings and learn rest of the
parameters in CubeRec by optimizing towards the combined loss $L_{group}+\mu
L_{self}$ ($\mu\geq 0$). In both training stages, we update corresponding
model parameters in each iteration and repeat the entire training process
until the loss converges or is sufficiently small.

We tune the hyperparameters using grid search. Specifically, the latent
dimension $d$ is searched in $\\{16,32,64,128,256\\}$, and the distance
coefficient $\gamma$ and self-supervision weight $\mu$ are searched in
$\\{0.1,0.3,0.5,0.7,0.9\\}$. For optimizing the three losses $L_{user}$,
$L_{group}$, $L_{self}$, we draw 5 negative samples for each positive ground
truth cases to construct corresponding training sets. The safety margins on
those losses are set to $\lambda=\lambda^{\prime}=\lambda^{\prime\prime}=0.5$
following common practices for hinge loss (Zhang et al., 2021b; Chen et al.,
2018). We set the the learning rate to $1\times 10^{-3}$ and batch size to
$256$ according to device capacity. To prevent overfitting, we adopt a dropout
(Srivastava et al., 2014) ratio of $0.2$ on all deep layers of CubeRec during
training.

##  4\. Experiments

Table 1. Statistics of experimental datasets. | Meetup | Yelp | Gowalla | Douban
---|---|---|---|---
#users | 24,631 | 34,504 | 60,805 | 70,743
#groups | 13,552 | 24,103 | 78,453 | 109,538
#items | 19,031 | 22,611 | 8,984 | 60,028
#user-item interactions | 126,813 | 482,273 | 1,625,817 | 3,422,266
#group-item interactions | 19,031 | 26,883 | 208,336 | 164,153
average #items per user | 5.15 | 13.98 | 26.74 | 48.38
average #items per group | 1.40 | 1.12 | 2.66 | 1.50
average #groups per user | 4.83 | 3.11 | 3.23 | 7.53
average group size | 8.79 | 4.45 | 2.31 | 4.86
Table 2. Recommendation results. Numbers in bold face are the best results for corresponding metrics. Method | Yelp | Douban | Gowalla | Meetup
---|---|---|---|---
Recall$@K$ | NDCG$@K$ | Recall$@K$ | NDCG$@K$ | Recall$@K$ | NDCG$@K$ | Recall$@K$ | NDCG$@K$
$K=10$ | $K=20$ | $K=10$ | $K=20$ | $K=10$ | $K=20$ | $K=10$ | $K=20$ | $K=10$ | $K=20$ | $K=10$ | $K=20$ | $K=10$ | $K=20$ | $K=10$ | $K=20$
NCF-AVG (He et al., 2017) | 0.0042 | 0.0090 | 0.0018 | 0.0030 | 0.0023 | 0.0042 | 0.0010 | 0.0015 | 0.0146 | 0.0222 | 0.0088 | 0.0108 | 0.0087 | 0.0092 | 0.0074 | 0.0075
FM-AVG (Vinh Tran et al., 2019) | 0.0161 | 0.0249 | 0.0075 | 0.0097 | 0.0032 | 0.0065 | 0.0014 | 0.0023 | 0.0127 | 0.0212 | 0.0061 | 0.0082 | 0.0128 | 0.0232 | 0.0063 | 0.0089
AGREE (Cao et al., 2018) | 0.0168 | 0.0286 | 0.0073 | 0.0103 | 0.0031 | 0.0047 | 0.0013 | 0.0017 | 0.0125 | 0.0228 | 0.0063 | 0.0088 | 0.0081 | 0.0117 | 0.0033 | 0.0041
SIGR (Yin et al., 2019) | 0.0194 | 0.0311 | 0.0082 | 0.0114 | 0.0035 | 0.0056 | 0.0016 | 0.0020 | 0.0143 | 0.0266 | 0.0070 | 0.0099 | 0.0090 | 0.0139 | 0.0038 | 0.0049
CAGR (Yin et al., 2020) | 0.0215 | 0.0345 | 0.0106 | 0.0138 | 0.0026 | 0.0072 | 0.0010 | 0.0021 | 0.0169 | 0.0320 | 0.0090 | 0.0129 | 0.0028 | 0.0029 | 0.0012 | 0.0013
GroupIM (Sankar et al., 2020) | 0.0424 | 0.0535 | 0.0207 | 0.0235 | 0.0212 | 0.0316 | 0.0098 | 0.0124 | 0.0281 | 0.0384 | 0.0113 | 0.0136 | 0.0407 | 0.0525 | 0.0237 | 0.0266
CubeRec-G | 0.0455 | 0.0592 | 0.0241 | 0.0298 | 0.0243 | 0.0354 | 0.0114 | 0.0167 | 0.0361 | 0.0575 | 0.0171 | 0.0194 | 0.0437 | 0.0616 | 0.0261 | 0.0306
CubeRec-A | 0.0417 | 0.0520 | 0.0206 | 0.0224 | 0.0211 | 0.0309 | 0.0098 | 0.0158 | 0.0343 | 0.0495 | 0.0135 | 0.0164 | 0.0418 | 0.0525 | 0.0202 | 0.0211

In this section, we conduct experiments to verify the effectiveness of CubeRec
in group recommendation. Specifically, we aim to answer the following research
questions (RQs):

  * RQ1:

Is CubeRec the new state-of-the-art?

  * RQ2:

Are the major components proposed in CubeRec effective?

  * RQ3:

How does CubeRec perform w.r.t. different group sizes?

  * RQ4:

What is the impact of CubeRec’s key hyperparameters?

###  4.1. Datasets

We adopt four real-world datasets collected from different event-based social
networks, namely Meetup, Yelp, Gowalla, and Douban. Both Meetup and Douban
contain group events held in different venues respectively in New York and
Beijing, where event venues are items to be recommended. Meanwhile, Yelp and
Gowalla are typical check-in datasets on different restaurants (i.e., items in
our case) based in the US. Since Yelp and Gowalla do not originally contain
group information, we follow the widely adopted procedure (Sankar et al.,
2020; Yin et al., 2019) to construct group interactions by finding overlaps on
both check-in times and social relations. That is, we assume if a set of users
who are connected on the social network visit the same venue at the same time,
then they are regarded as members of a group, and the corresponding activities
are group activities (Yin et al., 2019). The use of explicit social
connections and spatiotemporal tags ensures the quality of discovered user
groups in Yelp and Gowalla. We provide the key statistics of the four datasets
in Table 1. Each dataset is split with a ratio of $8$:$1$:$1$ for training,
validation, and test, respectively.

###  4.2. Baselines and Evaluation Protocols

We compare CubeRec111We release our implementation at:
https://github.com/jinglong0407/CubeRec.git with the following state-of-the-
art baselines:

  * •

MF-AVG (Vinh Tran et al., 2019): This baseline takes the average of all group-
wise user representations learned via user-item matrix factorization (MF)
(Koren et al., 2009) to compose group embeddings.

  * •

NCF-AVG (He et al., 2017): It also represents groups with average pooling on
user embeddings, where user embeddings are learned via the neural
collaborative filtering instead of MF.

  * •

AGREE (Cao et al., 2018): This approach utilizes attention networks, thus
accounting for different importance of group members when learning group
embeddings.

  * •

SIGR (Yin et al., 2019): It exploits and integrates users’ global and local
social influence to improve the group recommendation.

  * •

CAGR (Yin et al., 2020): This method firstly learns centrality-aware user
representations, and then learns a group recommender via two-stage
optimization.

  * •

GroupIM (Sankar et al., 2020): This state-of-the-art group recommender adopts
mutual information maximization between users and groups to overcome the
sparse group-level interactions.

It is worth noting that in CubeRec, we implement both the geometric and
attentive approaches described in Section 3.2 for composing group hypercubes,
which are respectively marked as CubeRec-G and CubeRec-A. We set $L=3$ for
learning user and item embeddings with LightGCN, and implement the two MLPs
for hypercube intersection with a 3-layer setting and ReLU activation. Based
on the hyperparameter search described in Section 3.5, we fix $d=64$,
$\gamma=0.3$, and $\mu=0.7$ across all datasets. The impact of these
hyperparameters to the recommendation performance will be further discussed in
Section 4.6.

We leverage two metrics, namely recall at rank $K$ (Recall$@K$) (Chen et al.,
2019, 2021) and normalized discounted cumulative gain at rank $K$ (NDCG$@K$)
(Chen et al., 2020b; Zhang et al., 2021c) that are widely adopted in
recommendation research. We adopt $K=10,20$ where all items unvisited by each
group are taken as negative samples for evaluation. In short, Recall$@K$
measures the ratio of the ground truth items that are present on the top-$K$
list, and NDCG$@K$ evaluates whether the model can rank the ground truth items
as highly as possible.

|  |
---|---|---
Figure 2. Visualization of CubeRec-G (left) and CubeRec-A (right) for the same
user group and recommended positive item on Meetup. Note that only the first
two dimensions of $\mathbb{R}^{d}$ are used to facilitate visualization. The
amount of visualized negative items is different as all representations are
learned from two independent vector spaces, and embeddings out of the $[-3,3]$
range are clipped out to ensure clarity.

###  4.3. Recommendation Effectiveness (RQ1)

Table 2 summarizes the performance comparison among all the group
recommenders, where each method’s results are averaged over five runs on every
dataset. With the recommendation results, we discuss our key findings below.

Apparently, CubeRec-G consistently outperforms all baselines by a significant
margin. Compared with GroupIM which is the best baseline, the average
improvements on Recall@10 and NDCG@10 brought by CubeRec-G are 14.4% and
23.6%, respectively. Unlike the straightforward mean pooling from user
embeddings in NCF-AVG and FM-AVG, the selective preference aggregation schemes
in AGREE, SIGR, CAGR and GroupIM generally achieve better performance when
learning group-level representations for recommendation. However, these
baselines are still subject to limited flexibility and resilience of the point
embeddings used for representing groups. In contrast, our hypercube-based
group representations can bypass the inherent limitations of point embeddings,
thus achieving the best recommendation effectiveness.

Meanwhile, although CubeRec-A is not the best-performing variant of CubeRec,
it still yields highly competitive results which are on par or stronger than
the best baseline GroupIM. On one hand, its strong performance further
demonstrates the benefit of replacing point embeddings with the more
expressive hypercube representations. On the other hand, a possible reason
that CubeRec-G is more advantageous than CubeRec-A is that, the group
hypercubes learned via geometric bounding has a higher tolerance to the multi-
faceted user preferences, and are overall more comprehensive representations
of group-level interests. To qualitatively compare CubeRec-G and CubeRec-A, in
Figure 2, we visualize the learned hypercubes of a randomly picked group from
Meetup. For clarity, we only use the first two dimensions of $\mathbb{R}^{d}$.
The hypercubes and users in two plots belong to the same group in the dataset,
and the same positive item has been successfully recommended in the top-10
list in both evaluation cases. As the visualization suggests, the hypercube
learned by CubeRec-G is a tighter and more inclusive representation of all its
members’ preferences. Also, CubeRec-G appears to be more capable of pulling
the positive item into the group hypercube than CubeRec-A, leading to better
recommendation performance.

Another observation is the relatively inferior performance of all methods on
Douban compared with other datasets. Despite the dense user-item interactions
for learning individual users’ preferences, the recommendation performance is
largely impaired by the sparse group-item interactions and the largest item
set. It is worth mentioning that methods with self-supervised learning (i.e.,
GroupIM and CubeRec) are still able to leverage augmented supervision signals
to learn quality group representations, providing the best recommendation
results even for this challenging dataset.

Table 3. Ablation test with different model architectures (Recall@10 is demonstrated). Numbers in bold face are the best results from each model, and “$\downarrow$” marks a severe (over $5\%$) performance drop compared with the best results. Method | Architecture | Yelp | Douban | Gowalla | Meetup
---|---|---|---|---|---
CubeRec-G | Default | 0.0455 | 0.0243 | 0.0361 | 0.0437
Remove SR | 0.0440 | 0.0222$\downarrow$ | 0.0353 | 0.0414$\downarrow$
Point Distance | 0.0204$\downarrow$ | 0.0098$\downarrow$ | 0.0156$\downarrow$ | 0.0181$\downarrow$
Remove SSL | 0.0442 | 0.0232 | 0.0342$\downarrow$ | 0.0407$\downarrow$
CubeRec-A | Default | 0.0417 | 0.0211 | 0.0343 | 0.0418
Remove SR | 0.0391$\downarrow$ | 0.0209 | 0.0326 | 0.0371$\downarrow$
Point Distance | 0.0202$\downarrow$ | 0.0099$\downarrow$ | 0.0159$\downarrow$ | 0.0163$\downarrow$
Remove SSL | 0.0382$\downarrow$ | 0.0195$\downarrow$ | 0.0342 | 0.0399

###  4.4. Ablation Study (RQ2)

To better understand the performance gain from the core components proposed in
CubeRec, we perform ablation analysis on different degraded versions of
CubeRec. Table 3 summarizes the recommendation outcomes in terms of Recall@10.
In what follows, we introduce all variants and discuss the effectiveness of
corresponding model components.

|  |  |
---|---|---|---
Figure 3. Recommendation performance w.r.t. different group sizes.
---
|  |
Figure 4. Recommendation performance w.r.t. different hyperparameters.

Removing Social Relations (Remove SR). In CubeRec, we infuse social relations
when modelling user preferences via Eq.(3). To testify the usefulness of
social relations in CubeRec, we remove the social relation matrix S from the
adjacency matrix by replacing it with a $|\mathcal{U}|\times|\mathcal{U}|$
matrix filled with 0s. The resulted variant models, have experienced
noticeable performance drops for both CubeRec-G and CubeRec-A, especially on
Meetup dataset with the highest sparsity of user-item interactions. Hence, the
inclusion of social relations is important for CubeRec to learn representative
user embeddings, which are the building block for group hypercubes.

Using Only The Distance Between Item and Group Center Points (Point Distance).
A crucial difference between CubeRec and point embedding-based group
recommenders is the use of its hypercubes. Essentially, a group hypercube
specifies a subspace in $\mathbb{R}^{d}$, where the recommended item points
either locate in this subspace or maintain short distances with it. As a core
ranking mechanism for group-item pairs in CubeRec, we verify the efficacy of
such point-to-hypercube distance metric by replacing Eq.(11) with the
conventional point distance. That is,
$d_{nj}=||\textbf{c}_{n}-\textbf{v}_{j}||^{2}_{2}$ where squared Euclidean
distance between the group center and item points are used. As a result, there
is a significant decrease in recommendation for CubeRec across all datasets.
This reflects that our revamped distance metric in CubeRec is a highly
distinctive measurement for the affinity between item and groups.

Removing Self-supervised Learning (Remove SSL). This variant disables the
self-supervised loss when optimizing CubeRec, i.e., the model is only
optimized towards $L_{group}$ instead of $L_{group}+\mu L_{self}$. As CubeRec
can no longer leverage the common users shared between groups as a supervision
signal to counteract the group-level data sparsity and regularize the learned
group hypercubes, it suffers from inferior recommendation accuracy on all
datasets. Specifically, significant performance drops (over 5%) are observed
from CubeRec-G on Gowalla and Meetup, and from CubeRec-A on Yelp and Douban.
As such, the novel self-supervised learning scheme showcases its strong
contribution to the performance gain in CubeRec.

###  4.5. Effect of Different Group Sizes (RQ3)

In group recommendation, different group sizes will bring varied impact to the
recommendation accuracy, and it is crucial for a recommender to keep robust in
different circumstances. In this study, we further evaluate the performance of
CubeRec on various group sizes. Following (Sankar et al., 2020), we segment
all groups into 5 categories, namely groups with 2-3, 3-4, 6-7, 8-9, and 10 or
more users. We additionally choose GroupIM as the peer method for comparison
because it is the most performant baseline.

We summarize the recommendation results in Figure 3, where Recall@10 is used
for benchmarking. The first observation is that, CubeRec-G outperforms
CubeRec-A and GroupIM in almost all scenarios. This also supports the superior
performance of CubeRec-G given its capability to deal with different group
sizes. Secondly, CubeRec-G yields more performance gain compared with GroupIM
when the group sizes are relatively small (i.e., between 2 and 5). This is
especially important for group recommendation, as the group sizes follow the
typical long-tail distribution in all our datasets, considering it is more
natural and feasible for users to form smaller groups in real life. Thirdly,
we find that in general, group sizes are positively associated with the
recommendation accuracy. Though smaller groups may have weaker discrepancies
in user preferences during decision-making, the learning of group-level
representations are also restricted by the limited user-item interactions.
When the group size grows, the composed group representations tend to encode
richer predictive signals for making accurate recommendations. Meanwhile, in
some cases like Yelp and Gowalla, when the group size exceeds 9, the
recommendation performance starts decreasing. This is largely associated with
the excessive noise introduced by the diverse group members.

###  4.6. Hyperparameter Sensitivity (RQ4)

We answer RQ4 by investigating the performance fluctuations of CubeRec with
varying hyperparameters, in particular the latent dimension $d$, trade-off
$\gamma$ between inner and outer distances in Eq.(11), and the self-
supervision strength $\mu$. Based on the standard hyperparameter setup in
Section 4.2, we tune the value of one hyperparameter while keeping the others
unchanged, and record the new results achieved in Figure 4. Note that we only
showcase the hyperparameter sensitivity for CubeRec-G since it is the best-
performing of the two, and CubeRec-A exhibits a highly similar trend as
CubeRec-G.

Impact of $d$. The value of $d$ is examined in $\\{16,32,64,128,256\\}$. In
general, CubeRec benefits from a relatively larger $d$ on all four datasets.
But noticeably, the performance improvement stops when $d$ reaches a certain
size (64 and 128 in our case) due to overfitting.

Impact of $\gamma$. We also study the impact of
$\gamma\in\\{0.1,0.3,0.5,0.7,0.9\\}$ that weighs the contribution of inner
distance in our revamped group-to-item distance metric. As $\gamma$ increases
from 0.1 to 0.3, there is a generally upward trend in CubeRec’s performance.
However, when $\gamma$ exceeds 0.5, the performance starts to decrease. Hence,
in the revamped distance metric, accounting for slightly more outer distance
offers higher resolution when distinguishing a group’s preferences towards
different items.

Impact of $\mu$. The self-supervised learning loss is multiplied by $\mu$ to
control its regularization effect. When the value of $\mu$ varies in
$\\{0.1,0.3,0.5,0.7,0.9\\}$, the best performance of CubeRec is observed when
$\mu=0.7$ in most scenarios, suggesting the necessity of utilizing our
proposed self-supervision to counteract the sparsity of group-item
interactions. However, CubeRec achieves better performance when $\mu=0.3$ on
Yelp, which possibly attributes to the smallest average group size in this
dataset (3.11 users per group), hindering the construction of self-supervision
signals via relay users.

##  5\. Related Work

In general, group recommendation methods are designed in different ways to
suit two types of audiences (Yin et al., 2019; Sankar et al., 2020;
Quintarelli et al., 2016): persistent groups and occasional groups. While
persistent group recommendation assumes dense interaction records at the group
level, our work addresses occasional group recommendation, which is more
practical and widely studied as real user groups tend to be formed ad-hoc with
limited group-item interactions (Yin et al., 2019). Compared with persistent
group recommendation where traditional collaborative filtering can be directly
applied on group-item interactions (Hu et al., 2014; Seko et al., 2011), a
common practice in occasional group recommendation is to first learn
individuals’ preferences from user-level interactions, then perform preference
aggregation to infer the overall interest of a group. Besides, a widely
acknowledged challenge (Sankar et al., 2020; Yin et al., 2020) in occasional
group recommendation is the highly sparse group-level interactions. Hence, we
review two lines of research that contributes to group recommendation, namely
methods for preference aggregation and counteracting data sparsity.

###  5.1. Preference Aggregation

In group recommendation, all group members’ preferences can be aggregated via
either late aggregation or early aggregation. The aim of late aggregation
(Amer-Yahia et al., 2009) is to generate item recommendations or predicted
item-wise affinity scores for each group member, which are combined at the
output stage via predefined strategies to produce recommendation results for
the group. In this category, the most representative approaches are average
satisfaction, least misery and maximum pleasure (Amer-Yahia et al., 2009;
Baltrunas et al., 2010; Quintarelli et al., 2016). Unfortunately, these
predefined aggregation strategies heavily rely on heuristics, thus lacking the
desired expressiveness and flexibility for optimal performance (Guo et al.,
2020; Jia et al., 2021).

As such, recent group recommenders mostly rely on early aggregation methods
(Cao et al., 2018; Liu et al., 2012; Vinh Tran et al., 2019; Yin et al., 2020;
Yuan et al., 2014) for learning group-level preference representations. In
contrast to late aggregation, such methods first aggregate the preferences of
all group members into a group-level representation, and then make group
recommendations accordingly. Some early aggregation methods are built upon
probabilistic models (Gorla et al., 2013; Liu et al., 2012; Yuan et al., 2014)
by considering both a user’s own preferences and her/his impact to the group,
but the same type of probability distribution is assumed across users, which
is infeasible in real-world cases. To address this problem, latent factor
models (Sajjadi Ghaemmaghami and Salehi-Abari, 2021; Huang et al., 2020) are
adopted to map groups to embeddings in the vector space. For example,
attentive neural networks are proposed in (Cao et al., 2018; Guo et al., 2020)
to selectively aggregate user representations within a group, and (Vinh Tran
et al., 2019) further captures the fine-grained interactions between group
members via a sub-attention network. More recently, there are also studies to
incorporate additional information like social connections (Cao et al., 2021)
and knowledge graphs (Deng et al., 2021) into the learning process of group
representations. However, as discussed in Section 1, the point embeddings used
in those methods sacrifice the diversity of personal preferences. Besides,
compared with alternative solutions like capsule networks (Li et al., 2019;
Cen et al., 2020) that rank items by mapping the learned multi-interest
representations to a single embedding, CubeRec recommends items directly with
group hypercubes, thus retaining loyalty to the decision-making process in
real groups.

###  5.2. Alleviating Data Sparsity for Groups

When recommending items to occasional groups, the other crucial goal is to
mitigate the scarce interactions between groups and items which can heavily
hinder the quality of learned group representations. Some research account for
the social relationships among users to provide hints on group-level
preferences (Yin et al., 2019; Cao et al., 2021; Guo et al., 2020), such as
(Yin et al., 2019) that considers the impact of users’ social influences on a
group’s decision. Afterwards, (Guo et al., 2020) proposes an intra-group
voting mechanism based on self-attention (Vaswani et al., 2017) to
simultaneously model the social influence and the pairwise user interactions,
and (He et al., 2020a) further learns multi-view embeddings from the
interactions among groups, users and items with a heterogeneous graph. As
self-supervised learning (SSL) has shown its effectiveness in general
recommendation tasks (Wu et al., 2021; Zhou et al., 2021; Yu et al., 2022),
attempts are also made to design SSL objectives to counteract the data
sparsity in group recommendation. For example, (Zhang et al., 2021a) leverages
common users among groups and designs a double-scale contrastive learning to
enhance the learned user representations. Meanwhile, (Sankar et al., 2020)
proposes a user-group mutual information maximization scheme to jointly learn
informative user and group representations. Though SSL is proven to benefit
group recommendation, existing methods are only compatible to point user/group
embeddings in the vector space. In contrast, our design of SSL in CubeRec
takes advantage of the geometric properties of hypercubes, allowing for more
expressive group representations to be learned despite data sparsity.

##  6\. Conclusion

To learn high-quality group representations that have the capacity to encode
the multi-faceted user preferences and the flexibility to fully resemble a
group’s decision-making process, we move beyond the traditional point
embeddings, and propose to learn hypercube group representations in this
paper. The proposed solution CubeRec takes advantage of its more expressive
hypercubes in the vector space, and comes with a new distance metric and an
intersection-based self-supervision paradigm to respectively facilitate group-
item pairwise ranking and mitigation of data sparsity. Experimental results
have rigorously verified the efficacy of CubeRec, shedding light on its
applications in a wider range of recommendation tasks.

## Acknowledgement

T. Chen is supported by UQ New Staff Research Start-up Grant (NS-2103). This
work is partly sponsored by Australian Research Council under the streams of
Future Fellowship (FT210100624), Discovery Project (DP190101985) and Discovery
Early Career Researcher Award (DE200101465). Y. Wang and M. Wang are supported
by National Natural Science Foundation of China (U21A20470).

## References

  * (1)
  * Amer-Yahia et al. (2009) Sihem Amer-Yahia, Senjuti Roy, Ashish Chawlat, Gautam Das, and Cong Yu. 2009.  Group recommendation: Semantics and efficiency.  _PVLDB_ 2, 1 (2009), 754–765.
  * Baltrunas et al. (2010) Linas Baltrunas, Tadas Makcinskas, and Francesco Ricci. 2010\.  Group recommendations with rank aggregation and collaborative filtering. In _RecSys_. 119–126.
  * Cao et al. (2018) Da Cao, Xiangnan He, Lianhai Miao, Yahui An, Chao Yang, and Richang Hong. 2018\.  Attentive group recommendation. In _SIGIR_. 645–654.
  * Cao et al. (2021) Da Cao, Xiangnan He, Lianhai Miao, Guangyi Xiao, Hao Chen, and Jiao Xu. 2021\.  Social-Enhanced Attentive Group Recommendation.  _TKDE_ 33, 03 (2021), 1195–1209.
  * Cen et al. (2020) Yukuo Cen, Jianwei Zhang, Xu Zou, Chang Zhou, Hongxia Yang, and Jie Tang. 2020\.  Controllable multi-interest framework for recommendation. In _SIGKDD_. 2942–2951.
  * Chen et al. (2018) Hongxu Chen, Hongzhi Yin, Weiqing Wang, Hao Wang, Quoc Viet Hung Nguyen, and Xue Li. 2018\.  PME: projected metric embedding on heterogeneous networks for link prediction. In _SIGKDD_.
  * Chen et al. (2019) Tong Chen, Hongzhi Yin, Hongxu Chen, Rui Yan, Quoc Viet Hung Nguyen, and Xue Li. 2019\.  AIR: Attentional intention-aware recommender systems. In _ICDE_. 304–315.
  * Chen et al. (2020a) Tong Chen, Hongzhi Yin, Quoc Viet Hung Nguyen, Wen-Chih Peng, Xue Li, and Xiaofang Zhou. 2020a.  Sequence-Aware Factorization Machines for Temporal Predictive Analytics.  _ICDE_ (2020).
  * Chen et al. (2020b) Tong Chen, Hongzhi Yin, Guanhua Ye, Zi Huang, Yang Wang, and Meng Wang. 2020b.  Try this instead: Personalized and interpretable substitute recommendation. In _SIGIR_. 891–900.
  * Chen et al. (2021) Tong Chen, Hongzhi Yin, Yujia Zheng, Zi Huang, Yang Wang, and Meng Wang. 2021\.  Learning elastic embeddings for customizing on-device recommenders. In _SIGKDD_. 138–147.
  * Deng et al. (2021) Zhiyi Deng, Changyu Li, Shujin Liu, Waqar Ali, and Jie Shao. 2021.  Knowledge-Aware Group Representation Learning for Group Recommendation. In _ICDE_. 1571–1582.
  * Gorla et al. (2013) Jagadeesh Gorla, Neal Lathia, Stephen Robertson, and Jun Wang. 2013.  Probabilistic group recommendation via information matching. In _WWW_. 495–504.
  * Guo et al. (2021) Lei Guo, Hongzhi Yin, Tong Chen, Xiangliang Zhang, and Kai Zheng. 2021.  Hierarchical Hyperedge Embedding-based Representation Learning for Group Recommendation.  _TOIS_ (2021).
  * Guo et al. (2020) Lei Guo, Hongzhi Yin, Qinyong Wang, Bin Cui, Zi Huang, and Lizhen Cui. 2020\.  Group recommendation with latent voting mechanism. In _ICDE_. 121–132.
  * He et al. (2020b) Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020b.  LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation.  _SIGIR_ (2020).
  * He et al. (2017) Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017\.  Neural collaborative filtering. In _WWW_. 173–182.
  * He et al. (2020a) Zhixiang He, Chi-Yin Chow, and Jia-Dong Zhang. 2020a.  GAME: Learning Graphical and Attentive Multi-view Embeddings for Occasional Group Recommendation. In _SIGIR_. 649–658.
  * Hsieh et al. (2017) Cheng-Kang Hsieh, Longqi Yang, Yin Cui, Tsung-Yi Lin, Serge Belongie, and Deborah Estrin. 2017\.  Collaborative metric learning. In _WWW_. 193–201.
  * Hu et al. (2014) Liang Hu, Jian Cao, Guandong Xu, Longbing Cao, Zhiping Gu, and Wei Cao. 2014\.  Deep modeling of group preferences for group-based recommendation. In _AAAI_.
  * Huang et al. (2020) Zhenhua Huang, Xin Xu, Honghao Zhu, and MengChu Zhou. 2020\.  An efficient group recommendation model with multiattention-based neural networks.  _TNNLS_ 31, 11 (2020), 4461–4474.
  * Jia et al. (2021) Renqi Jia, Xiaofei Zhou, Linhua Dong, and Shirui Pan. 2021\.  Hypergraph Convolutional Network for Group Recommendation. In _ICDM_. 260–269.
  * Kingma and Ba (2015) Diederik P Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ (2015).
  * Kipf and Welling (2017) Thomas N Kipf and Max Welling. 2017.  Semi-supervised classification with graph convolutional networks.  _ICLR_ (2017).
  * Koren et al. (2009) Yehuda Koren, Robert Bell, and Chris Volinsky. 2009\.  Matrix factorization techniques for recommender systems.  _Computer_ (2009).
  * Li et al. (2019) Chao Li et al. 2019\.  Multi-interest network with dynamic routing for recommendation at Tmall. In _CIKM_. 2615–2623.
  * Li et al. (2021) Yunchuan Li, Yan Zhao, and Kai Zheng. 2021.  Preference-aware Group Task Assignment in Spatial Crowdsourcing: A Mutual Information-based Approach. In _ICDM_. 350–359.
  * Liu et al. (2012) Xingjie Liu, Yuan Tian, Mao Ye, and Wang-Chien Lee. 2012\.  Exploring personal impact for group recommendation. In _CIKM_. 674–683.
  * Nguyen et al. (2017) Quoc Viet Hung Nguyen, Chi Thang Duong, Thanh Tam Nguyen, Matthias Weidlich, Karl Aberer, Hongzhi Yin, and Xiaofang Zhou. 2017\.  Argument discovery via crowdsourcing.  _VLDBJ_ 26, 4 (2017), 511–535.
  * Quintarelli et al. (2016) Elisa Quintarelli, Emanuele Rabosio, and Letizia Tanca. 2016\.  Recommending new items to ephemeral groups using contextual user influence. In _RecSys_. 285–292.
  * Ren et al. (2020) Hongyu Ren, Weihua Hu, and Jure Leskovec. 2020.  Query2box: Reasoning over knowledge graphs in vector space using box embeddings.  _ICLR_ (2020).
  * Sajjadi Ghaemmaghami and Salehi-Abari (2021) Sarina Sajjadi Ghaemmaghami and Amirali Salehi-Abari. 2021.  DeepGroup: Group Recommendation with Implicit Feedback. In _CIKM_. 3408–3412.
  * Sankar et al. (2020) Aravind Sankar, Yanhong Wu, Yuhang Wu, Wei Zhang, Hao Yang, and Hari Sundaram. 2020\.  Groupim: A mutual information maximization framework for neural group recommendation. In _SIGIR_. 1279–1288.
  * Seko et al. (2011) Shunichi Seko, Takashi Yagi, Manabu Motegi, and Shinyo Muto. 2011\.  Group recommendation using feature space representing behavioral tendency and power balance among members. In _RecSys_. 101–108.
  * Srivastava et al. (2014) Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014.  Dropout: a simple way to prevent neural networks from overfitting.  _JMLR_ (2014), 1929–1958.
  * Vaswani et al. (2017) Ashish Vaswani et al. 2017\.  Attention is all you need. In _NIPS_. 5998–6008.
  * Vinh Tran et al. (2019) Lucas Vinh Tran, Tuan-Anh Nguyen Pham, Yi Tay, Yiding Liu, Gao Cong, and Xiaoli Li. 2019\.  Interact and decide: Medley of sub-attention networks for effective group recommendation. In _SIGIR_. 255–264.
  * Wang et al. (2020) Wen Wang, Wei Zhang, Jun Rao, Zhijie Qiu, Bo Zhang, Leyu Lin, and Hongyuan Zha. 2020\.  Group-Aware Long-and Short-Term Graph Representation Learning for Sequential Group Recommendation. In _SIGIR_. 1449–1458.
  * Wang et al. (2019) Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019.  Neural graph collaborative filtering. In _SIGIR_. 165–174.
  * Wu et al. (2021) Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. 2021.  Self-supervised graph learning for recommendation. In _SIGIR_. 726–735.
  * Yin and Cui (2016) Hongzhi Yin and Bin Cui. 2016.  _Spatio-temporal recommendation in social media_.  Springer.
  * Yin et al. (2019) Hongzhi Yin, Qinyong Wang, Kai Zheng, Zhixu Li, Jiali Yang, and Xiaofang Zhou. 2019\.  Social influence-based group representation learning for group recommendation. In _ICDE_. 566–577.
  * Yin et al. (2020) Hongzhi Yin, Qinyong Wang, Kai Zheng, Zhixu Li, and Xiaofang Zhou. 2020.  Overcoming data sparsity in group recommendation.  _TKDE_ (2020).
  * Yu et al. (2022) Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Jundong Li, and Zi Huang. 2022\.  Self-Supervised Learning for Recommender Systems: A Survey.  _arXiv preprint arXiv:2203.15876_ (2022).
  * Yuan et al. (2014) Quan Yuan, Gao Cong, and Chin-Yew Lin. 2014.  COM: a generative model for group recommendation. In _SIGKDD_. 163–172.
  * Zhang et al. (2021a) Junwei Zhang, Min Gao, Junliang Yu, Lei Guo, Jundong Li, and Hongzhi Yin. 2021a.  Double-Scale Self-Supervised Hypergraph Learning for Group Recommendation. In _CIKM_. 2557–2567.
  * Zhang et al. (2021b) Shuai Zhang, Huoyu Liu, Aston Zhang, Yue Hu, Ce Zhang, Yumeng Li, Tanchao Zhu, Shaojian He, and Wenwu Ou. 2021b.  Learning User Representations with Hypercuboids for Recommender Systems. In _WSDM_. 716–724.
  * Zhang et al. (2021c) Shijie Zhang, Hongzhi Yin, Tong Chen, Zi Huang, Lizhen Cui, and Xiangliang Zhang. 2021c.  Graph embedding for recommendation against attribute inference attacks. In _The Web Conference_. 3002–3014.
  * Zhou et al. (2021) Yujia Zhou, Zhicheng Dou, Yutao Zhu, and Ji-Rong Wen. 2021\.  PSSL: Self-supervised Learning for Personalized Search with Contrastive Sampling. In _CIKM_. 2749–2758.