# Cycle Generative Adversarial Networks for Complementary Item Recommendations

Koby Bibas<EMAIL_ADDRESS>MetaTel AvivIsrael ,  Oren Sar Shalom
<EMAIL_ADDRESS>AmazonTel AvivIsrael and  Dietmar Jannach
<EMAIL_ADDRESS>University of KlagenfurtKlagenfurtAustria

(2023)

# Complementary Item Recommendation with Cycle Generative Adversarial Networks

Koby Bibas<EMAIL_ADDRESS>MetaTel AvivIsrael ,  Oren Sar Shalom
<EMAIL_ADDRESS>AmazonTel AvivIsrael and  Dietmar Jannach
<EMAIL_ADDRESS>University of KlagenfurtKlagenfurtAustria

(2023)

# Semi-supervised Adversarial Learning for Complementary Item Recommendation

Koby Bibas<EMAIL_ADDRESS>MetaTel AvivIsrael ,  Oren Sar Shalom
<EMAIL_ADDRESS>AmazonTel AvivIsrael and  Dietmar Jannach
<EMAIL_ADDRESS>University of KlagenfurtKlagenfurtAustria

(2023)

###### Abstract.

Complementary item recommendations are a ubiquitous feature of modern
e-commerce sites. Such recommendations are highly effective when they are
based on collaborative signals like co-purchase statistics. In certain online
marketplaces, however, e.g., on online auction sites, constantly new items are
added to the catalog. In such cases, complementary item recommendations are
often based on item side-information due to a lack of interaction data. In
this work, we propose a novel approach that can leverage both item side-
information and labeled complementary item pairs to generate effective
complementary recommendations for cold items, i.e., for items for which no co-
purchase statistics yet exist. Given that complementary items typically have
to be of a different category than the seed item, we technically maintain a
latent space for each item category. Simultaneously, we learn to project
distributed item representations into these category spaces to determine
suitable recommendations. The main learning process in our architecture
utilizes labeled pairs of complementary items. In addition, we adopt ideas
from Cycle Generative Adversarial Networks (CycleGAN) to leverage available
item information even in case no labeled data exists for a given item and
category. Experiments on three e-commerce datasets show that our method is
highly effective.

Recommender systems, Complementary Items, CycleGAN

††journalyear: 2023††copyright: acmlicensed††conference: Proceedings of the
ACM Web Conference 2023; May 1–5, 2023; Austin, TX, USA††booktitle:
Proceedings of the ACM Web Conference 2023 (WWW ’23), May 1–5, 2023, Austin,
TX, USA††price: 15.00††doi: 10.1145/3543507.3583462††isbn:
978-1-4503-9416-1/23/04††conference: The Web Conference; April 30–May 4, 2023;
Austin, TX††ccs: Information systems Recommender systems

##  1\. Introduction

Many modern e-commerce sites provide _complementary_ item recommendations for
their customers. For example, when online users shop for shirts, they may
receive suggestions for suitable pairs of pants. In practice, such
recommendations often appear under labels such as “Frequently bought
together”, and these recommendations are an important driver for cross-selling
on e-commerce platforms (Brovman et al., 2016; Jannach and Jugovac, 2019;
Jannach et al., 2019). Given the typically large product assortments in
e-commerce, the ranking of complementary item recommendations is commonly
based on behavioral patterns observed in the consumer community, for instance
on co-purchase patterns. In addition, domain knowledge may be incorporated,
for example, that the complementary item recommendations have to come from a
pre-defined _target_ category (e.g., pants), or from a category that is
different from the category of the _seed_ item, i.e., the item for which
complementary suggestions are sought for.

In some domains, however, item catalogs can be very volatile (Sar Shalom et
al., 2015). On the eBay (_ebay.com_) auction platform, for example, constantly
new items become available and often there is only one exemplar of a certain
product available (Galron et al., 2018; Wang et al., 2021). Consequently,
suitable methods are needed for _cold_ seed items, i.e., items for which no
transaction data is available. In this setting it is impossible to model the
seed item based on collaborative signals like item co-purchases. Thus, one may
have to rely on alternative approaches, like processing side-information about
the items, e.g., natural language descriptions of the items or structured item
attributes.

Although we focus on cold items, it is still possible to leverage aggregate
usage data. Specifically, such data allows us to infer a set of _complementary
categories_ , i.e., categories that include complementary items, like the
categories pants and shirts. Recommendations for an item from a certain
category can then be constrained to items that belong to complementary
categories. The advantages of this approach are threefold: (1) it helps rule
out irrelevant recommendations; (2) it significantly reduces inference time;
and (3) it can be used to design bundles (Zhu et al., 2014; Avny Brosh et al.,
2022) (e.g., by recommending a single item per complementary category), and
thereby allow users to understand how they were formed. Given these
advantages, category-level constraints on complementary item recommendation is
therefore a common practice at large e-commerce sites like Facebook Shops and
Instagram Shops111https://research.facebook.com/blog/2021/12/shops-on-
facebook-and-instagram-understanding-relationships-between-products-to-
improve-buyer-and-seller-experience/.

In this work, we propose a novel deep learning model to address this
challenging problem setting. The main technical idea of our approach is to
maintain a latent space for each item category. Simultaneously, we learn to
translate representations of the items, which we derive with the help of side-
information, into these latent spaces. This process finally allows us to
locate items in a given category space that are close to the seed item. The
basic learning process in our architecture is based on _supervised learning_
using labeled pairs of complementary items. We note that these pairs of items,
which can for example be obtained by analyzing existing shopping baskets of a
site, do _not_ include the seed item, as we supply recommendations for cold
items. Instead, they help us learn how categories are related. Since these
labeled pairs can be sparse for certain categories, we furthermore incorporate
ideas from Cycle Generative Adversarial Networks (CycleGAN) (Zhu et al., 2017)
in our architecture. This allows us to leverage available item information
even when no labeled data exists for a given item and category.

Overall, we therefore suggest a novel _semi-supervised_ model for
complementary item recommendations. An evaluation of the model on three
e-commerce datasets confirms that it is favorable over alternative approaches
under different experimental settings. Moreover, the experiments clearly
demonstrate the benefits of extending the architecture with CycleGAN elements.
To our knowledge, utilizing CycleGAN concepts for improved complementary item
recommendation has not been explored in the literature before.

##  2\. Background and Related Work

A common technical approach in real-world e-commerce is to determine related
items, e.g., on item detail pages222Such recommendations are often not
personalized according to long-term user profiles, which is also one
assumption of our present work. See (Wang et al., 2021) for a personalized
approach at eBay., is to rely on co-purchasing patterns. Recommendations based
on such simple shopping basket analyses can be surprisingly effective (Lee and
Hosanagar, 2019; Ludewig and Jannach, 2018). We note, however, that the items
returned by such approaches are not necessarily complementary items, and it
may happen that two particular shirts are frequently bought together (Hao et
al., 2020). In our work, we therefore assume that a complementary item belongs
to one among the pre-defined complementary categories, which are different
from the seed item’s category (e.g., pants in the case of shirts).

Another type of recommendations commonly seen in practice are often shown
under labels such as “Similar Items” or “Related Items”, some of which may
serve as substitutes (McAuley et al., 2015b; Yao and Harper, 2018; Trattner
and Jannach, 2019; Wang et al., 2011; Brovman et al., 2016). Finding similar
items is another relevant problem, but almost the opposite of our research
focus. _Related items_ , see, e.g., (Wang et al., 2011), on the other hand,
can in theory include both complements (e.g., accessories) and substitutes,
i.e., alternatives. Several works (Ahsan et al., 2020; Mane et al., 2019;
Angelovska et al., 2021) aim to find complementary products, but these often
cannot distinguish between similar and complementary items, since the seed and
the recommended items are encoded by the exact same network.

In terms of the addressed research challenge, the work by Galron et al.
(Galron et al., 2018) at eBay is closest to our research, and we also adopt a
similar evaluation approach. Given the sparsity of user-item interaction data,
which hampers the use of traditional collaborative filtering approaches, the
item-based DCF (Deep Collaborative Filtering) method proposed in (Galron et
al., 2018) learns a similarity function between items based on their side-
information. This similarity function is then used to retrieve recommendations
for a given seed item, i.e., the neural network takes a pair of seed and
target items as inputs, and returns a similarity prediction as an output.
Internally, the network encodes the items as sparse feature vectors based on
characteristics such as the title or category. To train the model, purchase
data from eBay in the form of pairs of items that were co-purchased by the
same users is used. Computational experiments and an A/B test indicate that
DCF performed significantly better than the existing system at eBay. Given the
proven performance of DCF in a real-world setting, we use this method as a
baseline in our research. We note that in the evaluation of DCF
recommendations from the same category were considered as ground truth.

_P-Companion_ (Hao et al., 2020) is another neural framework for complementary
item recommendation. This model focuses on diversifying the recommendations
and as it outperformed several other methods (McAuley et al., 2015a; Wang et
al., 2018; Hao et al., 2019), we also use it as baseline. While this model has
some commonalities with our approach, it does not allow semi-supervised
learning. Furthermore, it does not maintain a separate latent space per
category. Instead, the various dimensions in the item embeddings are weighted
differently, according to their category.

In terms of applications, various previous works focus on fashion
recommendation (Zhao et al., 2017a; De Divitiis et al., 2023; Wu et al., 2019;
Li et al., 2021). In many cases, _visual_ approaches are highly effective
(Cohen et al., 2021). Such approaches typically consider domain-specifics and
strongly rely on item images to establish relationships between different
clothing items, e.g., by projecting items in a shared visual style space
(McAuley et al., 2015a). In a recent work (De Divitiis et al., 2023), the
authors for example use shape and color information to learn which shapes and
colors are compatible. Compatibility estimation is also the focus in (Song et
al., 2018; He et al., 2016a). A session-based recommendation approach, which
also considers visual information is presented in (Wu et al., 2019). However,
such visual approaches can also have their limitations. Specifically, purely
visual approaches may return items that are stylistically similar, but not
complementary. In (Zhao et al., 2017a), the authors therefore propose a
complementary item recommendation approach that relies on textual information,
and can be extended using recent methods (Shalom et al., 2021). These works
are different from our approach in that our framework is not domain-specific.
Furthermore, our approach considers both visual and textual side-information
and in addition allows us to specify target categories for the complementary
items. Explicitly making compatibility predictions between items is not in the
focus of our present work, but an interesting area for future investigations.
Similarly, works that aim to automatically discern if two items rather
represent alternatives or complements, e.g., (Liu et al., 2020; Wang et al.,
2018), may be integrated in our framework in the future as well to assess if
they help further improve accuracy.

A number of other works exist that use datasets that contain information about
co-purchased or co-viewed items like we do in our computational experiments,
e.g., (Trofimov, 2018; Zhao et al., 2017b). The focus of such works however is
often not primarily on making complementary item recommendations. Instead, the
goal is to improve prediction accuracy in general, which is achieved by
exploiting co-occurrence information in the data. No distinction is however
made if the resulting recommendations actually contain substitutes or
complements.

The use of adversarial training in recommender systems is widespread (Krishnan
et al., 2018, 2019; Resheff et al., 2019). Specifically, using GANs for
complementary recommendations was proposed in (Kumar and Gupta, 2019), where a
generator learns how to create the image of a complementary item. However, the
goal of this methods is to generate an image and not an item. Another
generative adversarial learning approach is proposed in (Huynh et al., 2018).
Given a seed item, a generator network creates recommendations and a
discriminator needs to distinguish between real labeled recommendations and
generated ones. However, this model requires full supervision and, unlike our
approach, it does not allow to control important traits of the
recommendations, like the category.

A _quality-aware_ method for complementary item recommendation is proposed in
(Zhang et al., 2018). The underlying intuition is that not only the
compatibility of the items matters, but also the quality of the recommended
items. The quality in their approach is based on explicit item ratings, and
the proposed model jointly considers user preferences and compatibility
aspects. In our work, in contrast, we do not assume the existence of long-term
user preference profiles and explicit ratings. Nonetheless, personalizing the
recommendations (Yan et al., 2022; Wang et al., 2022; Zhang et al., 2021) or
considering the sequential nature of user-sessions (Xu et al., 2020) may be an
aspect to explore in the future.

Figure 1. The proposed model architecture. Elements with dashed lines are part
of the adversarial learning network.

##  3\. Overview of Proposed Approach

##### Summary of Problem Setting

We recall the specifics of our problem setting. Given a _seed_ item and a
_target_ category, the task is to recommend complementary items from the
target category333In practice, there can be multiple complementary categories
and the recommendation process may thus be repeated for each category. . The
main challenge is that no interaction data yet exists for the seed item.
However, we assume that each item has a known category and there is some
additional side-information available for each item. Moreover, we make the
assumption that there exists a labeled set of pairs of complementary
items444Note that the pairs in the labeled set do not contain the seed item..
Such a set may be created manually or automatically derived, e.g., by
analyzing co-purchase patterns of pairs of items with sufficient purchase
signals. Generally, however, paucity of labeled data is an inherent problem,
and a model that can cope with it is preferable.

##### Technical Approach

Let $\mathcal{C}$ denote the set of all item categories in the catalog. As
mentioned, one key idea of our approach is to maintain a latent space for each
item category $c\in C$. To make a recommendation for a given seed item, we
first create a distributed representation (i.e., embedding) of it, and we then
translate this embedding into the target category’s latent space. The
translation process can be understood as creating a pseudo-item by converting
the seed item’s representation into the latent space of the target category.
The main challenge in our approach now is to learn based on sparse data how to
translate items to the target latent space in a way that the important traits
of the seed item are maintained. Once the seed item is positioned in the
target category’s latent space, plausible recommendations can be derived by
selecting items that are close to it. More formally, let $I_{c}$ be the set of
items in category $c$. To find plausible recommendations in category $c$ for
seed item $s$, the translated representation $v^{c}_{s}$ is generated. Then
items are recommended according to their distance from $v^{c}_{s}$:
$\operatorname{argmax}_{i\in I_{c}}\operatorname{cos}(v^{c}_{s},v_{i})$, where
$\operatorname{cos}$ is the cosine similarity and $v_{i}$ is a distributed
representation of item $i$.

We recall that in our approach we aim to concentrate on a subset of _relevant_
categories for each item. Therefore, given an item’s category, we determine
complementary categories from item-level labeled data. To find the set of
complementary categories for category $c\in\mathcal{C}$ we aggregate the item-
level labeled pairs to the category-level to determine the set of other
relevant categories555In our work we do not allow complementary item
recommendations from the same category, as done in (Wu et al., 2019). This
constraint can however be trivially dismissed as needed.

##### Architecture Components

The overall architecture has two main elements. First, the architecture
includes a _supervised_ learning component, which uses item side-information
and the labeled set of pairs of complementary items to learn the translation
between the category latent spaces. Since this labeled set of pairs may suffer
from data paucity in particular for rare categories, we extend the
architecture to include a sub-network that supports _unsupervised_ adversarial
learning from all items in the catalog. I.e., also for items for which there
is no labeled complementary item in a given category in the dataset. Thus, the
two elements combined together constitute a semi-supervised learning approach.
Figure 1 shows the overall architecture of the model.

##  4\. Supervised Learning Component

The supervised learning component has two elements. The _item encoder_ creates
an embedding of the items. The _category translator_ then translates the
encoded item into the latent space of any given target category.

##### Item encoder

Given an item $i$, the model first generates its representation $v_{i}$ based
on its side-information. In many application scenarios, including fashion as
discussed above, the image of the item is a highly important piece of side-
information. To incorporate this information, we first feed the item image
through a pre-trained image processing model. Then, we pass its output through
a learned multilayer perceptron (MLP). For categorical features, the model
fits an embedding per each distinct value. We note that continuous features,
such as an item’s price, can be dealt with by converting them to categorical
features through discretization. To obtain the overall item representation,
the representations of all features are concatenated and passed through
another MLP. We acknowledge that other, more sophisticated approaches could be
used instead of a simple MLP. Yet, we defer these potential improvements to
future work. We note that the category itself is used here as another feature,
which makes the _item encoder_ category-aware.

##### Category translator

The task of this element is to learn how to transfer the item representations
from one latent space to another. For example, let $s$ be a specific shirt and
$p$ and $n$ (stand for positive and negative) are two pairs of pants, with
embeddings $v_{s}$, $v_{p}$ and $v_{n}$, respectively. Let us assume that $s$
is complemented by $p$, but not by $n$. Then, we would like to learn a
transformation such that projecting the shirt’s representation $v_{s}$ into
the pants domain will yield a representation that is more similar to $v_{p}$
than to $v_{n}$. We stress that there are many plausible architectures to
combine the item and the category representations, some of them allow
heterogeneous dimensionalities of the latent spaces of the various categories.
However, our model of choice is a simple concatenation of these embeddings,
followed by an MLP.

##### Model training

Training in the supervised learning component is based on the given set of
labeled pairs of items. Each labeled pair consist of a seed item $s$ and a
positive complementary item $p$ of category $c$. For each such pair, we apply
negative sampling and draw a negative item $n$ (from category $c$ as well).
The _item encoder_ is then invoked on each of the items to obtain the
representations $v_{s}$, $v_{p}$ and $v_{n}$. Then, applying the _category
translator_ on $v_{s}$ yields $v^{c}_{s}$, the representation of the seed item
in the desired latent space. As a loss function, we use the triplet loss
(Schroff et al., 2015):
$\mathcal{L}\left(v^{c}_{s},v_{p},v_{n}\right)=\operatorname{max}\left({\operatorname{d}\left(v^{c}_{s},v_{p}\right)}-{\operatorname{d}\left(v^{c}_{s},v_{n}\right)}+\alpha,0\right)$,
where $\operatorname{d}$ is a distance function; the negative cosine
similarity performed well in our experiments. $\alpha$ is a hyperparameter
that sets the margin by which the positive sample should be more similar than
the negative one.

##  5\. Adversarial Learning Component

The effectiveness of any _supervised_ learning approach is bound by the
available labeled data. In complementary item recommendation, data paucity, as
mentioned, is a common problem. In particular for more rare categories there
may not be a sufficient amount of labeled pairs available for effective
learning. Therefore, our architecture includes an adversarial learning
component which implements _unsupervised_ learning. Thereby, we aim to
leverage information from any item in the catalog, even if there are no
labeled complementary items for it in a given category. In our example, this
would be the case if we are given a particular shirt and there is no
complementary item in the pants category in the training data. Ultimately, the
combination of both components leads to a _semi-supervised_ learning approach.
We emphasize that the adversarial component is only needed to further improve
the training process. At inference time, only the outputs of the _item
encoder_ and the _category translator_ are used to determine the
recommendations.

###  5.1. Architecture Elements

The main goal of this component is to further improve the effectiveness of the
_category translator_. To ensure that the _category translator_ is not
misguided when incorporating information about items without labels, the
adversarial component takes inspiration from CycleGAN (Zhu et al., 2017).
CycleGAN was designed in the context image-to-image translation problems,
e.g., to translate a summer landscape into winter. A central idea in these
networks is the concept of _cycle consistency_. For the mentioned example, a
translation would be cycle consistent if we end up close to the original image
if we translate the winter landscape back. Ultimately, cycle consistency
ensures that inputs and outputs are paired in a meaningful way.

Technically, our adversarial learning component has two main sub-networks, the
_classifier_ and the _reconstructor_. The _classifier_ receives an item
representation $v_{i}$ and a category $c$ as an input and returns the
probability that item $i$ belongs to category $c$. To this end, the
_classifier_ assigns a score for each category by passing $v_{i}$ to an MLP,
where the last layer is of size $|\mathcal{C}|$. Then, these scores are turned
into probabilities using _softmax_ and the probability of the item belonging
to category $c$ is extracted. Finally, the translated representation
$v^{c}_{i}$ is sent to the _classifier_ to affirm it indeed looks like a
representation of an item in category $c$. This auxiliary component motivates
the _category translator_ to produce reliable embeddings. However, it might be
incapable to train a good recommender, because it is not guaranteed that
important traits of the seed item are preserved. In our running example, the
translated representation may seem like it belongs to a pair of pants, but it
might not lead to good recommendations, as it is not constrained to retain
either concrete or abstract properties of the original shirt like style, age
group, or price.

To address this issue, the _reconstructor_ is introduced, which encourages the
_category translator_ to be _cycle consistent_ and to retain the traits of the
seed item. Specifically, this network receives a translated representation
$v^{c}_{i}$ and the original category of the seed item. It then aims to
reconstruct the original representation of the seed item $v_{i}$ by returning
a vector $v^{\prime}_{i}$ such that $v_{i}\approx v^{\prime}_{i}$.

###  5.2. Model training

The adversarial learning component incorporates two types of loss functions:
for the _reconstructor_ and for the _classifier_. To allow end-to-end learning
of the entire architecture, the final loss is given by a weighted sum of these
two losses and the loss function of the supervised learning component. The
weights for the losses constitute a convex combination and are determined by
hyperparameters.

##### Cycle consistency loss

The loss of the _reconstructor_ , in our approach is the Euclidean distance
$\|v_{i}-v^{\prime}_{i}\|^{2}$. Since both terms in the loss, $v_{i}$ and
$v^{\prime}_{i}$, are outputs of learned networks, a degenerated, yet optimal
loss can be achieved. For instance, if the _item encoder_ and the _category
translator_ return the same fixed value, regardless of the input, then the
loss would be 0. To overcome this issue, we recall that the purpose of this
loss is to motivate the _category translator_ to not dismiss important traits
in the seed item, represented by $v_{i}$. Therefore, $v_{i}$ serves as a label
for this loss. Consequently, we stop the gradients backpropagate from $v_{i}$
due to this loss. We point out that this procedure allows to further improve
the _item encoder_ , due to the Jacobian obtained from $v^{\prime}_{i}$.

##### Classifier loss

As mentioned above, given an item representation and a category, the
_classifier_ outputs the probability $p$ that the represented item belongs to
this category. Conventionally, its loss is cross-entropy and since it is
supplied with a single true category, the loss is contracted to
$-\\!\operatorname{log}p$. We should bear in mind that only a well trained
classifier can challenge the _category translator_ and thereby allow it to
generate suitable representations. This poses several difficulties and rules
out the feasibility of a naive implementation of the architecture. We discuss
these challenges and how we addressed them as follows. To optimally train the
_classifier_ we supply it with two types of training instances.

The first type consists of genuine outputs of the _item encoder_. Namely, we
invoke the _classifier_ twice, with the embeddings of the seed item $v_{s}$
and the positive item $v_{p}$666For running time considerations the negative
item’s embedding is not fed to the _classifier_ , although it could be
trivially added., where the label categories are the ones from the catalog.
However, we note that a standard implementation would also affect the _item
encoder_. Specifically, it can ultimately lead to a degenerated model, as the
category is an input to the _item encoder_. That way, the _item encoder_ is
motivated to cooperate with the _category translator_ so as to excel at this
loss at the expense of the true objective of the model, which is supplying
recommendations. For example, the _item encoder_ may output the category
embedding, while ignoring the rest of its input features. We therefore
implemented the model in a way that we prevent the gradients of the
_classifier_ of flowing through the parameters of the _item encoder_. Our
experiments revealed that this led to significantly improved performance.

As for the second type of input for the _classifier_ , we recall that the
_category translator_ aims to create an embedding $v^{c}_{s}$, which makes the
_classifier_ classify $v^{c}_{s}$ as if it belongs to category $c$. In order
to avert a situation where the _category translator_ finds edge cases
(adversarial examples) that only deceive the _classifier_ , but do not look
like real vectors of items in category $c$, it is important that the
_classifier_ is trained on the output of the _category translator_
$v^{c}_{s}$, with label category $c$. If done this way, $v^{c}_{s}$ is used to
train both the _category translator_ and the _classifier_ , but each of them
has a different objective. The _category translator_ aims to fool the
_classifier_ , while the latter needs to challenge the _category translator_.
To address this issue, we use _adversarial training_ with a gradient reversal
layer. This means that during backpropagation the _category translator_
obtains the original gradients, while the _classifier_ is directed by the
additive inverse of the gradients.

We note that this model is scalable since its training time is linear in the
size of the labeled set and item-category pairs; also, the amount of
parameters grows linearly with the number of categories.

###  5.3. Specific CycleGAN Adaptations

Here, we lay out the commonalities and differences of CycleGAN and our
approach in more detail. Specifically, this exposition will clarify how we
transferred ideas from CycleGAN to the complementary item recommendation
problem in an innovative way.

CycleGAN is a variant of Generative Adversarial Networks (GAN) (Goodfellow et
al., 2020), which includes two generators and two discriminators that are
trained simultaneously. Let $X$ be the source domain and $Y$ the target domain
for a given translation problem (e.g., summer to winter). Generator G learns
to transform images from $X$ to $Y$. Generator F learns to transform images
from $Y$ to $X$. Discriminator D_X learns to differentiate between genuine
images of $X$ and generated images $F(Y)$. That is, its objective is to return
a high probability value for $x\in X$ and a low probability value for
$\texttt{F}(y)$, for $y\in Y$. Similarly, discriminator D_Y operates in domain
$Y$. For simplicity, we collectively refer to D_X and D_Y as D, which
represents a discriminator that distinguishes between genuine and generated
images. Training of these networks is done using adversarial training. It is
generally desired that $\texttt{G}(x)$ does not dismiss important
characteristics of the original image. This would guarantee that
$\texttt{G}(x)$ is a translation of $x$, rather than just an image that seems
to belong to $Y$, but has no affinity to $x$. To this end, the cycle
consistency loss is introduced. Namely, this loss asks to minimize the
difference between $x$ and $\texttt{F}(\texttt{G}(x))$.

We first explain that in a sense, the _item encoder_ serves as the real
distribution, the _category translator_ as G, the _classifier_ as D, and the
_reconstructor_ as F. Given an input representation $v$ and a category $c$,
the _classifier_ works as follows. If $v$ is a was generated directly by the
_item encoder_ , then it should confirm that the item belongs to the declared
category $c$; otherwise, $v$ was further translated by the _category
translator_ , and the _classifier_ should reject an assumed affinity between
the item and the category. Therefore, similar to GANs, the _classifier_ (D)
aims to output a probability close to one for inputs drawn from the _item
encoder_ (real distribution) and probability of zero for inputs from the
_category translator_ (G). Like in CycleGAN, our approach applies the cycle
consistency loss using the _reconstructor_ , which translates the item
representation back to its original category and therefore serves as F.

However, there are some notable differences between the two models. First, the
instances drawn from the real distribution in CycleGAN are genuine images from
the source domain. Therefore, the cycle consistency loss is well defined, as
the original image $x$ serves as the label for $\texttt{F}(\texttt{G}(x))$. In
contrast, our approach works on the feature space. That is, the “real
distribution” is actually the output of the _item encoder_ , which is a
_learned_ network. Consequently, the outputs of the _image encoder_ also serve
as labels, which might hamper its training. As mentioned before, we overcome
this problem by stopping the gradients that arise from the seed item’s vector.

Another difference stems from the complexity of the _category translator_.
While in CycleGAN there are only two domains, in our problem the number of
domains (categories) can reach to hundreds. Therefore we resort to training a
unified translator for all categories. Given the unified translator it might
seem like the _reconstructor_ is redundant, as also the _category translator_
can translate between any pair of categories. However, in our experiments we
noticed a crucial advantage in separating these two networks. We conjecture
that it is due to their different objectives. The _category translator_ aims
to find good recommendations, while the _reconstructor_ wishes to recover the
original latent traits of the item. Therefore, the _category translator_ may
generate vectors that are similar to those of popular or high quality items,
since they usually make good recommendations, while the _reconstructor_ is not
bound to this need. Therefore, the _category translator_ , which generates the
recommendations, should preferably be distinct from the _reconstructor_.

##  6\. Experimental Evaluation

We conducted an in-depth experimental evaluation of our approach. All our code
is shared online for reproducibility
777https://fb.me/cgan_ccomplementary_item_recommendation.

###  6.1. Experiment Design

##### Datasets & Preprocessing

We rely on real-world data from _Amazon_ (He and McAuley, 2016), as used in
previous related works (Liu et al., 2020; Hao et al., 2020). From these
datasets, we first extracted the items’ side-information, which include image,
price, and category. Furthermore, each item $i$ in the dataset can be
associated with a list of recommended items $\\{r_{i}\\}$ that are _frequently
bought together_. By removing items from $\\{r_{i}\\}$ that belong to the same
category as $i$, we create a labeled set of complementary item pairs
$\\{(s,r_{i})\\}$, which we use for model training and evaluation in the
experiments. We note that these pairs may be asymmetric, e.g., a laptop may be
accompanied by a recommendation for a charger, but not vice versa.

We considered three subsets of the _Amazon_ datasets: “Clothing, Shoes and
Jewelry” (dubbed as Clothing), “Home and Kitchen” (Home), and “Toys and Games”
(Toys). These subsets were chosen because complementary items are of key
importance in these domains. At the same time, they differ in key aspects like
their verticals, target users, attributes that affect the notion of
complementary, and number of items and categories. The main statistics of
these datasets are shown in Table 1.

Table 1. Dataset statistics Statistic | Clothing | Toys | Home
---|---|---|---
#items | 14,591 | 20,510 | 29,258
#item pairs | 53,375 | 112,964 | 162,497
#categories | 126 | 124 | 286
#category pairs | 4,621 | 5,734 | 18,999
Avg. items per category | 115.8 | 165.4 | 102.3
Max items per category | 808 | 2,558 | 796
Min items per category | 12 | 16 | 11

In terms of pre-processing, we excluded categories with less than five items
to reduce noise. As a pre-trained image processing model, we used ResNet152
(He et al., 2016b). If an item had multiple images, we consider only the first
one. We furthermore discretized the continuous price to twenty bins using
equal-depth binning. We used a random sample of 80% of the data for training
and the rest for validation and testing. To mimic the cold start problem, we
ensured that the seed items in the validation and test sets do not appear in
the training set.

##### Evaluation Procedure & Metrics

The output of our model is a ranked list of complementary items, given a seed
item. We therefore apply standard list-based accuracy metrics, namely Hit Rate
($HR@k$) and NDCG (Normalized Discounted Cumulative Gain), see (Shani and
Gunawardana, 2011). Moreover, since the labeled set is highly skewed to
popular items, some models may focus on a narrow set of such popular items in
their recommendations, thus leading to limited coverage and diversity.
Therefore, we also report _catalog coverage_ (Herlocker et al., 2004), which
is defined as the fraction of the items of the catalog that appear in the
top-$k$ recommendation lists for the seed items in the test set. The value of
$k$ in this measure was 10.

Following the described problem setting, in the main evaluation protocol the
desired category of the recommendations is given. To showcase the robustness
of the methods, we also experiment in a setting where the recommendations can
come from any category. We recall that our model requires a target category as
an input. To create a recommendation list that considers all relevant
categories, we go over all complementary categories of the seed item in a
round-robin fashion and iteratively select the next recommendation.

##### Baselines

We compare our model against these baselines.

  * •

Popularity is a simple yet effective baseline, which utilizes the labeled set
to count the popularity of each item. For each item $i$ it records the number
of seed items for which $i$ is labeled as their complementary item in the
labeled set.

  * •

DCF (Galron et al., 2018) is a recent neural model optimized for cold items.
Since its code was not released, we implemented it based on the the original
paper and make it publicly available.

  * •

DCF-Hard is a variant of _DCF_ that we propose here. It leverages category
information to apply _hard negative mining_. Specifically, negative samples
are not drawn from the entire catalog, but only from the items in the category
of the labeled target item.

  * •

P-Companion (Hao et al., 2020) is another recent neural model. Since its code
is not publicly available, we implemented it and publish the code in our
repository as well. To make it compatible with our problem setting we made two
modifications. First, we control the target categories and second, we omit
collaborative information to support the cold start problem.

We carefully tuned the hyperparameters both of our model and of the baselines
for each of the datasets in a manual process. The final hyperparameters can be
found in the code repository.

###  6.2. Results

####  6.2.1. Main Results

Table 2 reports the main results of our experiments, where the category is
given to the recommenders. We name our proposed model ALCIR, which stands for
_A dversarial Learning for Complementary Item Recommendation_. To analyze the
contribution of the individual elements of our architecture, we report results
for (a) _ALCIR-Sup_ , which only consists of the supervised learning component
from Section 4 and (b) _ALCIR_ , which corresponds to the _full_ model as
described in Section 5.

Table 2. Category-aware recommendation performance Method | NDCG | HR@1 | HR@5 | HR@10 | Cov. (%)
---|---|---|---|---|---
Clothing Dataset
Popularity | 0.238 | 0.051 | 0.153 | 0.234 | 6.99
DCF | 0.211 | 0.011 | 0.059 | 0.111 | 7.24
DCF-Hard | 0.226 | 0.019 | 0.083 | 0.147 | 40.49
P-Companion | 0.238 | 0.030 | 0.102 | 0.169 | 84.59
ALCIR-Sup | 0.298 | 0.074 | 0.203 | 0.302 | 89.92
ALCIR | 0.316 | 0.092 | 0.233 | 0.332 | 88.5
Toys Dataset
Popularity | 0.231 | 0.038 | 0.126 | 0.200 | 5.13
DCF | 0.193 | 0.009 | 0.042 | 0.081 | 5.96
DCF-Hard | 0.208 | 0.017 | 0.064 | 0.111 | 36.72
P-Companion | 0.235 | 0.028 | 0.104 | 0.172 | 93.78
ALCIR-Sup | 0.297 | 0.073 | 0.205 | 0.303 | 90.46
ALCIR | 0.308 | 0.078 | 0.224 | 0.330 | 82.62
Home Dataset
Popularity | 0.251 | 0.048 | 0.160 | 0.255 | 8.2
DCF | 0.211 | 0.011 | 0.051 | 0.102 | 8.32
DCF-Hard | 0.221 | 0.014 | 0.046 | 0.077 | 12.82
P-Companion | 0.245 | 0.032 | 0.111 | 0.187 | 94.45
ALCIR-Sup | 0.296 | 0.068 | 0.197 | 0.293 | 93.44
ALCIR | 0.304 | 0.077 | 0.210 | 0.312 | 94.38

The results in Table 2 show that _ALCIR_ consistently outperforms all
baselines in terms of the accuracy measures on all three datasets, usually
with a large margin888We report results at additional cut-off thresholds in
the code repository.. We additionally observe that already the supervised
component of our model (_ALCIR-Supervised_) performs better than the
baselines. The adversarial component is then successful in even further
increasing these already strong results.

Considering the ranking of the other models, we notice that the popularity-
based approach represents a baseline that can be difficult to beat. The _DCF_
model and even the improved _DCF-Hard_ model never reach the accuracy levels
of the popularity-based method. _P-Companion_ works better, but still does not
reach the Hit Rate values of popular-item recommendations. Only in terms of
the NDCG, _P-Companion_ reaches a similar performance level. The strong
performance of the popularity-based method is not surprising, though. An
inspection of the datasets revealed that the ten most popular items cover
between a fifth and a quarter of the labeled complementary items. Moreover,
also the evaluation of a “Co-Purchase” method in (Hao et al., 2020) showed
that _P-Companion_ did not outperform such a popularity-based approach in a
user-centric study999Popularity-based recommendations were not examined in
(Galron et al., 2018)..

Our proposed model, in contrast, outperforms the popularity-based model
consistently. We also observe that _ALCIR_ leads to high _coverage_ values,
ranging from 82% to 94%. Only the _P-Companion_ method reaches an even
slightly higher catalog coverage. The popularity-based method by design only
recommends very popular items, leading to the lowest coverage. Interestingly,
also the _DCF_ method has very low catalog coverage. The improved DCF version
(_DCF-Hard_) helps to address the coverage problem to a good extent.

Table 3 finally shows the performance results when we consider all relevant
categories for complementary item recommendations as described above.
Naturally, across all models, the performance results for this experiment are
lower compared to a case where the category is known. Nevertheless, we observe
that the proposed model ALCIR is superior also in this problem setting.

Table 3. Recommendation performance w/o target category Method | NDCG | HR@1 | HR@5 | HR@10 | Cov. (%)
---|---|---|---|---|---
Clothing Dataset
Popularity | 0.099 | 0.001 | 0.005 | 0.007 | 0.06
DCF | 0.111 | 0.000 | 0.001 | 0.002 | 0.09
DCF-Hard | 0.118 | 0.001 | 0.003 | 0.007 | 1.82
P-Companion | 0.111 | 0.000 | 0.002 | 0.003 | 0.62
ALCIR-Sup | 0.150 | 0.009 | 0.036 | 0.060 | 59.36
ALCIR | 0.170 | 0.012 | 0.060 | 0.098 | 60.55
Toys Dataset
Popularity | 0.102 | 0.001 | 0.003 | 0.005 | 0.04
DCF | 0.124 | 0.000 | 0.001 | 0.002 | 0.06
DCF-Hard | 0.148 | 0.003 | 0.010 | 0.016 | 5.67
P-Companion | 0.123 | 0.001 | 0.001 | 0.002 | 5.12
ALCIR-Sup | 0.172 | 0.010 | 0.044 | 0.021 | 64.56
ALCIR | 0.184 | 0.009 | 0.046 | 0.071 | 56.03
Home Dataset
Popularity | 0.093 | 0.000 | 0.001 | 0.002 | 0.03
DCF | 0.117 | 0.000 | 0.001 | 0.002 | 0.05
DCF-Hard | 0.117 | 0.000 | 0.002 | 0.003 | 0.13
P-Companion | 0.117 | 0.000 | 0.001 | 0.001 | 1.61
ALCIR-Sup | 0.143 | 0.005 | 0.021 | 0.033 | 53.42
ALCIR | 0.150 | 0.008 | 0.030 | 0.049 | 55.53
Figure 2. Performance with respect to the amount of labeled data

####  6.2.2. Additional Analyses

##### Impact of Data Paucity

One main assumption when we introduced the unsupervised learning component to
our model was that it will be particularly beneficial for rare categories. To
validate this assumption, we conducted the following analysis to assess the
relative contribution of the full _ALCIR_ method when we vary the amount of
labeled data for each pair of categories. To this end, we count the number of
labeled instances per each pair of complementary categories and discretize
these counts to ten equally-sized bins. We order the bins by the amount of
labeled data for each pair of categories in ascending order. The first bins
therefore represent pairs of complementary categories for which little
supervision is available in the training set.

Figure 2 shows the performance of each model for the ten bins. We note that
performance measures between different bins cannot be trivially compared,
because each bin holds different target categories, with a different amount of
items from which the recommender should select. Therefore, only the
performance of the different models within the same bin and dataset can be
compared, since they refer to the exact same test items. Considering this
aspect, we can observe the same trend in all datasets, where the relative
advantage of _ALCIR_ is higher for the rare category pairs, i.e, in the left
parts of the figures. In contrast, the advantage of the full model becomes
smaller for the very popular category pairs, and it is particularly pronounced
for the Toys and Home datasets. Overall, the analysis confirms our assumption
regarding the usefulness of the full model in particular for rare categories.

##### Ablation Study

Besides consisting of a supervised and an unsupervised component, one central
feature of our architecture is that it makes use of a number of specific loss
functions (triplet loss, cycle consistency loss and classifier loss), all of
which are aimed to increase the performance of the model. To validate that
these architecture elements contribute to the overall model performance, we
ran an ablation study to assess the performance when only some of the
components are utilized. Table 4 shows the outcomes of this analysis in
absolute numbers and relative to the performance of the complete model (shown
in parentheses). The results provide evidence that indeed all components
contribute to the performance of the model. Most noteworthy is that a model
that does not utilize labeled data, but only the unsupervised losses of the
classifier and the cycle consistency performs the worst. This, however, comes
as no surprise, given the well-known importance of labeled data.

Table 4. Ablation study for category aware recommendation Method |  NDCG |  HR@1 |  HR@5 |  HR@10
---|---|---|---|---
Clothing Dataset
Classifier+Cycle |  0.21
(-33.5%) |  0.015
(-83.9%) |  0.059
(-74.7%) |  0.114
(-65.7%)
ALCIR-Sup |  0.298
(-5.7%) |  0.074
(-19.4%) |  0.203
(-13.1%) |  0.302
(-8.9%)
Triplet+Cycle |  0.311
(-1.8%) |  0.087
(-5.6%) |  0.226
(-3.1%) |  0.319
(-3.8%)
Triplet+Classifier |  0.305
(-3.5%) |  0.082
(-11.3%) |  0.213
(-8.8%) |  0.306
(-7.5%)
ALCIR |  0.316 |  0.092 |  0.233 |  0.332
Toys Dataset
Classifier+Cycle |  0.207
(-32.9%) |  0.016
(-79.5%) |  0.07
(-68.7%) |  0.124
(-62.5%)
ALCIR-Sup |  0.297
(-3.7%) |  0.073
(-6.4%) |  0.205
(-8.7%) |  0.303
(-8.2%)
Triplet+Cycle |  0.294
(-4.4%) |  0.072
(-6.8%) |  0.205
(-8.3%) |  0.303
(-8.4%)
Triplet+Classifier |  0.298
(-3.3%) |  0.073
(-6.6%) |  0.209
(-6.6%) |  0.309
(-6.5%)
ALCIR |  0.308 |  0.078 |  0.224 |  0.330
Home Dataset
Classifier+Cycle |  0.218
(-28.1%) |  0.014
(-81.2%) |  0.068
(-67.5%) |  0.127
(-59.3%)
ALCIR-Sup |  0.296
(-2.6%) |  0.068
(-11.4%) |  0.197
(-6.1%) |  0.293
(-6.1%)
Triplet+Cycle |  0.298
(-2%) |  0.069
(-9.9%) |  0.201
(-4.1%) |  0.303
(-2.9%)
Triplet+Classifier |  0.297
(-2.1%) |  0.069
(-10.7%) |  0.199
(-4.9%) |  0.297
(-4.7%)
ALCIR |  0.304 |  0.077 |  0.210 |  0.312

##  7\. Future Work

In this work we have proposed a novel semi-supervised approach for the highly
relevant problem of complementary item recommendation, and an in-depth
empirical evaluation clearly demonstrates the benefits of the approach. Our
insights point to a number of future research directions. Our work focused on
complementary item recommendations for cold items, and we assume that existing
data (e.g., about co-purchases) can be used for the warm items. In future
work, we plan to extend our model to also support warm items, using the same
framework. That way, the input features for the item encoder would include
also collaborative data by applying e.g., the method suggested in (Bibas et
al., 2022). In addition to such extensions and continuing related research in
(Wang et al., 2021; Zhang et al., 2018), other promising approaches to further
improve the effectiveness of the model could lie in the _personalization_ of
the complementary item recommendations and to consider aspects of item
_quality_. Finally, our problem setting can be thought as a special case of
domain adaptation, where we transfer representations from one category to
another. An interesting future work would be to extend our method to other
recommendation scenarios, like context-aware recommendations, where we
generate item representations in different contexts.

## References

  * (1)
  * Ahsan et al. (2020) Unaiza Ahsan, Xiquan Cui, Rebecca West, Mingming Guo, Khalifeh Al Jadda, et al. 2020\.  Complementary Recommendations Using Deep Multi-modal Embeddings For Online Retail. In _2020 IEEE International Conference on Big Data (Big Data)_. IEEE, 1774–1779.
  * Angelovska et al. (2021) Marina Angelovska, Sina Sheikholeslami, Bas Dunn, and Amir H Payberah. 2021.  Siamese Neural Networks for Detecting Complementary Products. In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop_. 65–70.
  * Avny Brosh et al. (2022) Tzoof Avny Brosh, Amit Livne, Oren Sar Shalom, Bracha Shapira, and Mark Last. 2022.  BRUCE: Bundle Recommendation Using Contextualized item Embeddings. In _Proceedings of the 16th ACM Conference on Recommender Systems_. 237–245.
  * Bibas et al. (2022) Koby Bibas, Oren Sar Shalom, and Dietmar Jannach. 2022\.  Collaborative Image Understanding. In _The 31st ACM International Conference on Information and Knowledge Management_.
  * Brovman et al. (2016) Yuri M Brovman, Marie Jacob, Natraj Srinivasan, Stephen Neola, Daniel Galron, Ryan Snyder, and Paul Wang. 2016.  Optimizing similar item recommendations in a semi-structured marketplace to maximize conversion. In _Proceedings of the 10th ACM Conference on Recommender Systems_. 199–202.
  * Cohen et al. (2021) Rami Cohen, Oren Sar Shalom, Dietmar Jannach, and Amihood Amir. 2021.  A black-box attack model for visually-aware recommender systems. In _Proceedings of the 14th ACM International Conference on Web Search and Data Mining_. 94–102.
  * De Divitiis et al. (2023) Lavinia De Divitiis, Federico Becattini, Claudio Baecchi, and Alberto Del Bimbo. 2023.  Disentangling Features for Fashion Recommendation.  _ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM)_ 19 (2023).
  * Galron et al. (2018) Daniel A Galron, Yuri M Brovman, Jin Chung, Michal Wieja, and Paul Wang. 2018.  Deep item-based collaborative filtering for sparse implicit feedback.  _arXiv preprint arXiv:1812.10546_ (2018).
  * Goodfellow et al. (2020) Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020\.  Generative adversarial networks.  _Commun. ACM_ 63, 11 (2020), 139–144.
  * Hao et al. (2019) Junheng Hao, Muhao Chen, Wenchao Yu, Yizhou Sun, and Wei Wang. 2019.  Universal representation learning of knowledge bases by jointly embedding instances and ontological concepts. In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 1709–1719.
  * Hao et al. (2020) Junheng Hao, Tong Zhao, Jin Li, Xin Luna Dong, Christos Faloutsos, Yizhou Sun, and Wei Wang. 2020.  P-Companion: A Principled Framework for Diversified Complementary Product Recommendation. In _Proceedings of the 29th ACM International Conference on Information & Knowledge Management_. 2517–2524.
  * He et al. (2016b) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016b.  Deep residual learning for image recognition. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 770–778.
  * He and McAuley (2016) Ruining He and Julian McAuley. 2016.  Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In _Proceedings of the 25th International Conference on World Wide Web_. 507–517.
  * He et al. (2016a) Ruining He, Charles Packer, and Julian McAuley. 2016a.  Learning Compatibility Across Categories for Heterogeneous Item Recommendation. In _2016 IEEE 16th International Conference on Data Mining (ICDM)_. 937–942.
  * Herlocker et al. (2004) Jonathan L. Herlocker, Joseph A. Konstan, Loren G. Terveen, and John T. Riedl. 2004.  Evaluating Collaborative Filtering Recommender Systems.  _Transactions on Information Systems_ 22, 1 (2004), 5–53.
  * Huynh et al. (2018) Cong Phuoc Huynh, Arridhana Ciptadi, Ambrish Tyagi, and Amit Agrawal. 2018.  CRAFT: Complementary Recommendation by Adversarial Feature Transform. In _ECCV Workshops (3)_. 54–66.
  * Jannach and Jugovac (2019) Dietmar Jannach and Michael Jugovac. 2019.  Measuring the Business Value of Recommender Systems.  _ACM Transactions on Management Information Systems_ 10, 4 (2019).
  * Jannach et al. (2019) Dietmar Jannach, Oren Sar Shalom, and Joseph A Konstan. 2019\.  Towards More Impactful Recommender Systems Research.. In _ImpactRS@ RecSys_.
  * Krishnan et al. (2019) Adit Krishnan, Hari Cheruvu, Cheng Tao, and Hari Sundaram. 2019.  A modular adversarial approach to social recommendation. In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management_. 1753–1762.
  * Krishnan et al. (2018) Adit Krishnan, Ashish Sharma, Aravind Sankar, and Hari Sundaram. 2018.  An adversarial approach to improve long-tail performance in neural collaborative filtering. In _Proceedings of the 27th ACM International Conference on Information and Knowledge Management_. 1491–1494.
  * Kumar and Gupta (2019) Sudhir Kumar and Mithun Das Gupta. 2019.  c+ GAN: Complementary Fashion Item Recommendation.  _arXiv preprint arXiv:1906.05596_ (2019).
  * Lee and Hosanagar (2019) Dokyun Lee and Kartik Hosanagar. 2019.  How Do Recommender Systems Affect Sales Diversity? A Cross-Category Investigation via Randomized Field Experiment.  _Information Systems Research_ 30, 1 (2019), 239–259.
  * Li et al. (2021) Zhi Li, Bo Wu, Qi Liu, Likang Wu, Hongke Zhao, and Tao Mei. 2021\.  Learning the Compositional Visual Coherence for Complementary Recommendations. In _the Twenty-Ninth International Joint Conference on Artificial Intelligence_.
  * Liu et al. (2020) Yiding Liu, Yulong Gu, Zhuoye Ding, Junchao Gao, Ziyi Guo, Yongjun Bao, and Weipeng Yan. 2020.  Decoupled Graph Convolution Network for Inferring Substitutable and Complementary Items. In _Proceedings of the 29th ACM International Conference on Information & Knowledge Management_ _(CIKM ’20)_. 2621–2628.
  * Ludewig and Jannach (2018) Malte Ludewig and Dietmar Jannach. 2018.  Evaluation of Session-based Recommendation Algorithms.  _User-Modeling and User-Adapted Interaction_ 28, 4–5 (2018), 331–390.
  * Mane et al. (2019) Mansi Ranjit Mane, Stephen Guo, and Kannan Achan. 2019.  Complementary-similarity learning using quadruplet network.  _arXiv preprint arXiv:1908.09928_ (2019).
  * McAuley et al. (2015a) Julian McAuley, Rahul Pandey, and Jure Leskovec. 2015a.  Inferring Networks of Substitutable and Complementary Products. In _Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ _(KDD ’15)_. 785–794.
  * McAuley et al. (2015b) Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. 2015b.  Image-Based Recommendations on Styles and Substitutes. In _Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval_ _(SIGIR ’15)_. 43–52.
  * Resheff et al. (2019) Yehezkel S. Resheff, Yanai Elazar, Moni Shahar, and Oren Sar Shalom. 2019.  Privacy and Fairness in Recommender Systems via Adversarial Training of User Representations. In _Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods - ICPRAM_. 476–482.
  * Sar Shalom et al. (2015) Oren Sar Shalom, Shlomo Berkovsky, Royi Ronen, Elad Ziklik, and Amir Amihood. 2015.  Data quality matters in recommender systems. In _Proceedings of the 9th ACM Conference on Recommender Systems_. 257–260.
  * Schroff et al. (2015) Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015\.  Facenet: A unified embedding for face recognition and clustering. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 815–823.
  * Shalom et al. (2021) Oren Sar Shalom, Haggai Roitman, and Pigi Kouki. 2021\.  Natural language processing for recommender systems.  In _Recommender Systems Handbook_. Springer.
  * Shani and Gunawardana (2011) Guy Shani and Asela Gunawardana. 2011.  Evaluating recommendation systems.  In _Recommender Systems Handbook_. Springer, 257–297.
  * Song et al. (2018) Xuemeng Song, Fuli Feng, Xianjing Han, Xin Yang, Wei Liu, and Liqiang Nie. 2018\.  Neural Compatibility Modeling with Attentive Knowledge Distillation.  _CoRR_ abs/1805.00313 (2018).  http://arxiv.org/abs/1805.00313
  * Trattner and Jannach (2019) Christoph Trattner and Dietmar Jannach. 2019.  Learning to Recommend Similar Items from Human Judgements.  _User Modeling and User-Adapted Interaction_ 30 (2019), 1–49.
  * Trofimov (2018) Ilya Trofimov. 2018\.  Inferring Complementary Products from Baskets and Browsing Sessions.  _CoRR_ abs/1809.09621 (2018).  http://arxiv.org/abs/1809.09621
  * Wang et al. (2011) Jian Wang, Badrul Sarwar, and Neel Sundaresan. 2011.  Utilizing Related Products for Post-Purchase Recommendation in e-Commerce. In _Proceedings of the Fifth ACM Conference on Recommender Systems_ _(RecSys ’11)_. 329–332.
  * Wang et al. (2022) Rui Wang, Ning Yang, and S Yu Philip. 2022.  Learning aspect-level complementarity for intent-aware complementary recommendation.  _Knowledge-Based Systems_ (2022), 109936.
  * Wang et al. (2021) Tian Wang, Yuri M Brovman, and Sriganesh Madhvanath. 2021\.  Personalized embedding-based e-commerce recommendations at ebay.  _arXiv preprint arXiv:2102.06156_ (2021).
  * Wang et al. (2018) Zihan Wang, Ziheng Jiang, Zhaochun Ren, Jiliang Tang, and Dawei Yin. 2018.  A Path-Constrained Framework for Discriminating Substitutable and Complementary Products in E-Commerce. In _Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining_ _(WSDM ’18)_. 619–627.
  * Wu et al. (2019) Jui-Chieh Wu, José Antonio Sánchez Rodríguez, and Humberto Jesús Corona Pampín. 2019.  Session-based complementary fashion recommendations.  _arXiv preprint arXiv:1908.08327_ (2019).
  * Xu et al. (2020) Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. 2020.  Product knowledge graph embedding for e-commerce. In _Proceedings of the 13th International Conference on Web Search and Data Mining_. 672–680.
  * Yan et al. (2022) An Yan, Chaosheng Dong, Yan Gao, Jinmiao Fu, Tong Zhao, Yi Sun, and Julian McAuley. 2022.  Personalized complementary product recommendation. In _Companion Proceedings of the Web Conference 2022_. 146–151.
  * Yao and Harper (2018) Yuan Yao and F. Maxwell Harper. 2018.  Judging Similarity: A User-Centric Study of Related Item Recommendations. In _Proceedings of the 12th ACM Conference on Recommender Systems_ _(RecSys ’18)_. 288–296.
  * Zhang et al. (2021) Wei Zhang, Zeyuan Chen, Hongyuan Zha, and Jianyong Wang. 2021\.  Learning from substitutable and complementary relations for graph-based sequential product recommendation.  _ACM Transactions on Information Systems(TOIS)_ (2021).
  * Zhang et al. (2018) Yin Zhang, Haokai Lu, Wei Niu, and James Caverlee. 2018\.  Quality-aware neural complementary item recommendation. In _Proceedings of the 12th ACM Conference on Recommender Systems_. 77–85.
  * Zhao et al. (2017a) Kui Zhao, Xia Hu, Jiajun Bu, and Can Wang. 2017a.  Deep style match for complementary recommendation.  _arXiv preprint arXiv:1708.07938_ (2017).
  * Zhao et al. (2017b) Tong Zhao, Julian McAuley, Mengya Li, and Irwin King. 2017b.  Improving recommendation accuracy using networks of substitutable and complementary products. In _2017 International Joint Conference on Neural Networks (IJCNN)_.
  * Zhu et al. (2017) Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017.  Unpaired image-to-image translation using cycle-consistent adversarial networks. In _Proceedings of the IEEE International on Computer Vision_. 2223–2232.
  * Zhu et al. (2014) Tao Zhu, Patrick Harrington, Junjun Li, and Lei Tang. 2014\.  Bundle recommendation in ecommerce. In _Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval_. 657–666.