# Multi-grained Hypergraph Interest Modeling for
Conversational Recommendation

Chenzhan Shang<EMAIL_ADDRESS>School of InformationRenmin University of
China ,  Yupeng Hou<EMAIL_ADDRESS>Gaoling School of Artificial
IntelligenceRenmin University of China ,  Wayne Xin Zhao†🖂<EMAIL_ADDRESS>Gaoling School of Artificial IntelligenceRenmin University of China ,  Yaliang
Li<EMAIL_ADDRESS>Alibaba Group and  Jing Zhang  zhang-
<EMAIL_ADDRESS>School of InformationRenmin University of China

(2022)

###### Abstract.

Conversational recommender system (CRS) interacts with users through multi-
turn dialogues in natural language, which aims to provide high-quality
recommendations for user’s instant information need. Although great efforts
have been made to develop effective CRS, most of them still focus on the
contextual information from the current dialogue, usually suffering from the
data scarcity issue. Therefore, we consider leveraging historical dialogue
data to enrich the limited contexts of the current dialogue session.

In this paper, we propose a novel multi-grained hypergraph interest modeling
approach to capture user interest beneath intricate historical data from
different perspectives. As the core idea, we employ _hypergraph_ to represent
complicated semantic relations underlying historical dialogues. In our
approach, we first employ the hypergraph structure to model users’ historical
dialogue sessions and form a session-based hypergraph, which captures _coarse-
grained, session-level_ relations. Second, to alleviate the issue of data
scarcity, we use an external knowledge graph and construct a knowledge-based
hypergraph considering _fine-grained, entity-level_ semantics. We further
conduct multi-grained hypergraph convolution on the two kinds of hypergraphs,
and utilize the enhanced representations to develop interest-aware CRS.
Extensive experiments on two benchmarks ReDial and TG-ReDial validate the
effectiveness of our approach on both recommendation and conversation tasks.
Code is available at: https://github.com/RUCAIBox/MHIM.

Conversational Recommender System, Hypergraph Learning

$\dagger$ Beijing Key Laboratory of Big Data Management and Analysis Methods.

🖂 Corresponding author.

††copyright: acmcopyright††journalyear: 2022††doi:
xx.xxxx/xxxxxxx.xxxxxxx††conference: Make sure to enter the correct conference
title from your rights confirmation emai; June 03–05, 2018; Woodstock,
NY††booktitle: Woodstock ’18: ACM Symposium on Neural Gaze Detection, June
03–05, 2018, Woodstock, NY††price: 15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs:
Information systems Personalization††ccs: Information systems Recommender
systems

##  1\. INTRODUCTION

Conversational Recommender System (CRS) has become a trending research topic
in recent years, with the goal of capturing users’ instant preferences through
multi-turn dialogues and providing high-quality recommendations. Compared to
traditional recommender systems, CRS can leverage explicit feedback signals
from natural language conversations in an interactive way for more precise
modeling of user preferences.

In terms of methodology, a typical CRS consists of a conversation module and a
recommendation module. The conversation module aims to understand user
utterances and generate informative responses. The recommendation module
focuses on capturing user preferences from textual signals and recommending
appropriate items. Several methods (Lei et al., 2020a, b; Zhang et al., 2018)
propose to utilize item attributes to progressively narrow down candidate item
set and generate responses using pre-defined templates. Other studies (Li et
al., 2018; Chen et al., 2019) construct end-to-end frameworks for both
recommending and generating human-like responses. To further improve the
performance, researchers integrate multi-type external data (Zhou et al.,
2020a; Lu et al., 2021; Zhou et al., 2022) and devise a more controllable
conversation module (Liang et al., 2021).

Though great efforts have been made to develop effective CRSs, most of them
focus on utilizing the limited contextual information from the current
dialogue session (_i.e.,_ the ongoing conversation), usually suffering from
the data sparsity problem. To address this issue, our solution is inspired by
the observation that a user probably has engaged with the CRS several times
before the current conversation. These historical dialogue sessions contain
crucial evidence for capturing the preference of a user, which are easy to
collect in a practical system. Therefore, we consider comprehensively
capturing user interest that lies beneath intricate historical data to enrich
the limited contexts of the current dialogue session.

However, it is non-trivial to leverage historical dialogues for improving CRS,
and there still exist two major challenges. First, the intra- and inter-
session correlations among historical dialogues are complicated, where each
dialogue that a user invokes is called a session. A session typically
concentrates on a specific topic, involving a small number of important
entities (_e.g.,_ actor or director for movie recommendation). Due to the
limited dialogue context, it is difficult to accurately capture the
relatedness among intra-session entities and establish inter-session
relations. Second, historical data remains scarce in real-world conversational
recommendation scenarios. The number of users’ historical dialogue sessions
and items’ interaction records both follow a long-tail distribution.
Specifically, most of the users only interact with the system a few times, and
thus the historical dialogues may not be sufficient for accurate user
modeling. In addition, most of the items probably only appear a few times,
resulting in difficulty of learning informative entity representations for
recommendation.

In light of these challenges, and inspired by the work about hypergraph
learning (Feng et al., 2019; Yadati et al., 2019) and history-aware dialogue
system (Qu et al., 2019; Gupta and Sharma, 2021), our core idea is to employ
hypergraphs to represent complicated semantic relations among historical
dialogue sessions. Different from standard graph, in a hypergraph, a hyperedge
connects more than two vertices (Benson et al., 2016), which is particularly
suitable to model the interrelations among multiple objects (_e.g.,_
entities). When applied to our setting, hypergraph can better represent a
dialogue by directly associating multiple involving entities in an explicit
way. Besides, different hyperedges also share common vertices, which may be
useful to model vital attributes for reflecting intrinsic user interest.

To this end, in this paper, we propose a novel Multi-grained Hypergraph
Interest Modeling approach for conversational recommendation, named as MHIM.
We consider two major ways to construct hypergraphs for improving CRS. First,
to capture complicated relations in historical data, we model each session as
a hyperedge of items and construct a session-based hypergraph. Second, to
alleviate data scarcity, we incorporate an external knowledge graph (KG) and
construct a knowledge-based hypergraph. In order to model the above two kinds
of hypergraphs, we introduce multi-grained hypergraph convolution to model
historical user interest, with the session-based hypergraph capturing coarse-
grained (_session-level_) user preferences beneath historical data, and the
knowledge-based hypergraph capturing fine-grained (_entity-level_) user
interest on the KG. Besides, to learn informative entity representations, we
propose to pre-train the KG encoder by contrastive subgraph discrimination. To
make accurate recommendations, we devise a hypergraph-aware attention module
to derive user representation, which considers multi-grained user preferences
learned from historical data. For the conversation task, historical dialogue
sessions are further leveraged to build an interest-aware response generator.

To our knowledge, it is the first time that historical data is leveraged to
model user interest for CRS via hypergraph modeling. We firstly construct
multi-grained hypergraphs to model user preferences from dialogues, and then
propose a hypergraph convolution layer to learn informative entity and user
representations for conversational recommendation. Extensive experiments on
two public CRS datasets have demonstrated the effectiveness of our approach in
both recommendation and conversation tasks, by comparing a number of
competitive baselines.

##  2\. PRELIMINARIES

In this section, we first formulate the task of conversational recommendation,
and then introduce the definition of hypergraph.

Task Description. Conversational recommendation aims to provide high-quality
recommendations through a multi-turn dialogue with users. During the dialogue,
the system generates responses for clarification or makes relevant
recommendations, until the user accepts the results or exits. Typically, a CRS
consists of two major components, namely the recommendation module and the
conversation module. These two modules should be integrated seamlessly to
fulfill the recommendation goal (Chen et al., 2019; Zhou et al., 2020a).
Formally, let $\mathcal{I}$ denote the item set and $\mathcal{U}$ denote the
user set. A conversation $\mathcal{C}$ is a sequence composed of utterances
(_i.e.,_ a text sentence) occurring between a CRS and a user, denoted by
$\mathcal{C}=\\{S_{t}\\}_{t=1}^{n}$. Each utterance is composed of a sequence
of words. Given an $n$-turn conversation, the goal of CRS is to generate
responses to the user, including both the recommendation set
$\mathcal{I}_{n+1}$ and the reply utterance $S_{n+1}$.

Current and Historical Dialogue Sessions. Furthermore, in a CRS, a user
probably interacts with the system multiple times. For example, in an
e-commerce platform, a user might chat with the intelligent assistant on
different days seeking different types of products, reflecting the user’s
long-term and diverse interests. Specifically, the historical conversations
that a user involved in are called historical dialogue sessions, and the
ongoing conversation is called current dialogue session (Li et al., 2022).
Intuitively, it is useful to consider the historical dialogue sessions when
inferring user preference about the items during the current dialogue session.
By leveraging historical data, the system has potential to learn more precise
user preference representations for proper recommendations.

Hypergraph. To better capture the semantics of a dialogue session, a key point
is to effectively model the complex interrelations of entities mentioned in
the session. For a traditional graph structure, an edge connects two vertices,
while hypergraph generalizes the concept of edge to connect more than two
vertices (Benson et al., 2016; Feng et al., 2019). A hypergraph is defined as
$\widetilde{\mathcal{G}}=(\mathcal{V},\mathcal{H})$, which includes a vertex
set $\mathcal{V}$ and a hyperedge set $\mathcal{H}$. The hypergraph can be
represented by an incidence matrix
$\mathbf{H}\in\\{0,1\\}^{|\mathcal{V}|\times|\mathcal{H}|}$, with each entry
$\text{H}_{v,h}$ indicating whether a vertex $v$ is connected by a hyperedge
$h$:

(1) $\text{H}_{v,h}=\left\\{\begin{array}[]{ll}1&\textrm{if}\ v\in h,\\\
0&\textrm{if}\ v\notin h.\end{array}\right.$

For a vertex $v\in\mathcal{V}$, its degree is defined as
$d(v)=\sum_{h\in\mathcal{H}}\text{H}_{v,h}$. For an edge $h\in\mathcal{H}$,
its degree is defined as $\delta(h)=\sum_{v\in\mathcal{V}}\text{H}_{v,h}$.
Further, let $\mathbf{D}\in\mathbb{N}^{|\mathcal{V}|\times|\mathcal{V}|}$ and
$\mathbf{B}\in\mathbb{N}^{|\mathcal{H}|\times|\mathcal{H}|}$ denote the
diagonal matrices of the vertex degrees and the edge degrees, respectively. We
propose to model historical data as session-based and knowledge-based
hypergraphs. The former constructs the items in the same dialogue as a
hyperedge, and the latter constructs a target item and its neighborhood on KG
as a hyperedge. As will be shown in Section 3.2, these hypergraphs can capture
coarse-grained and fine-grained user interest, which benefits item
recommendation.

Figure 1. The overview of our model in a movie recommendation scenario. We
first encode KG and then capture user interest through multi-grained
hypergraph convolution. The learned entity representations are further used
for item recommendation and response generation. Moreover, “SA” and “CA”
denote self-attention and cross-attention layers, respectively.

##  3\. APPROACH

In this section, we present the proposed Multi-grained Hypergraph Interest
Modeling approach for conversational recommendation, named as MHIM. We first
introduce how to encode external KG to represent fine-grained entities. Then,
based on the learned entity representations, we present our method for
modeling historical user interest through multi-grained hypergraph
convolution. We finally describe our solutions for both recommendation and
conversation tasks utilizing the above user interest representations. The
overview illustration of our proposed model is presented in Figure 1.

###  3.1. Knowledge Graph Encoding with Contrastive Pre-training

As KG provides important external information for conversational
recommendation task, we first present our representation learning method for
task-specific KG. Then, we describe the enhanced training method by
introducing contrastive subgraph pre-training.

####  3.1.1. Task-specific KG Encoder

Following (Chen et al., 2019; Zhou et al., 2020a), to capture the semantics of
dialogue contents, we extract the basic semantic units (_i.e.,_ entity) from
utterances and link them to KG entries, and then extend these entities with
two-hop search. The mentioned entities and the extended entities form the
_task-related KG_ for CRS. We adopt the widely-used DBpedia (Lehmann et al.,
2015) and CN-DBpedia (Xu et al., 2017) as external KG. In KG, a semantic fact
is usually denoted as a triplet $\langle e,r,e^{\prime}\rangle$, where
$e,e^{\prime}\in\mathcal{E}$ are entities and $r\in\mathcal{R}$ is an entity
relation. In our setting, an item is also an entity, which means
$\mathcal{I}\subseteq\mathcal{E}$. Considering that the relations are crucial
for learning entity representations, we utilize R-GCN (Schlichtkrull et al.,
2018) to develop the KG encoder $f_{q}(\cdot)$, which improves the basic GCN
architecture by explicitly modeling the relational semantics. Through
information propagation and aggregation upon KG, we obtain the embedding
matrix for entities in $\mathcal{E}$ as
$\mathbf{N}\in\mathbb{R}^{|\mathcal{E}|\times d}$.

####  3.1.2. Improved KG Encoder by Contrastive Subgraph Discrimination

However, limited by the size of CRS datasets, it is usually difficult to
construct a sufficiently large task-related KG for training the KG encoder
$f_{q}(\cdot)$. Thus, we consider utilizing the _large-scale, original_ KG for
improving the KG encoder. To reduce the influence of irrelevant information,
we only keep the relations that occur in the CRS datasets and utilize them to
span the connected component from the original KG, called _extended KG_.
Compared with task-related KG, the extended KG contains more _relevant,
diverse_ entities, since the contained relations are from the CRS dataset. Our
idea is to introduce contrastive pre-training on the extended KG to improve
the KG encoder. Specifically, we propose to leverage subgraphs as contrastive
instances and use subgraph instance discrimination as our pre-training task.
This task treats each subgraph as a distinct class and learns to discriminate
between them.

We adopt random walk with restart (Qiu et al., 2020), in which the walk
returns back to starting vertex $v$ with a positive probability. The subgraph
$\widehat{\mathcal{G}}$ is finally induced by the collected vertices during
the random walk. To generate the subgraph representation, we first employ the
KG encoder (Section 3.1.1) to encode the nodes in a subgraph instance, and
then sum and normalize the node embeddings as subgraph representation. To
construct the contrastive samples, we consider two subgraphs derived from the
same starting vertex as a similar instance pair, and others as dissimilar
instance pairs.

Consider an encoded query $\bm{q}$ (_i.e.,_ the target subgraph) and a
dictionary of $M+1$ encoded keys $\\{\bm{k}_{0},\ldots,\bm{k}_{M}\\}$ (_i.e.,_
the contrastive samples), we assume that there is a single key (_i.e.,_ the
positive sample) that $\bm{q}$ matches in the dictionary, denoted by
$\bm{k}_{+}$. We adopt the InfoNCE contrastive loss in our work:

(2)
$\mathcal{L}=-\log\frac{\exp(\bm{q}^{\top}\bm{k}_{+}/\tau)}{\sum_{i=0}^{M}\exp(\bm{q}^{\top}\bm{k}_{i}/\tau)},$

where $\tau$ is the temperature hyper-parameter, $\bm{q}$ and $\bm{k}$ are
subgraph instance representations encoded by two separate R-GCN encoder
$f_{q}$ and $f_{k}$, denoted by $\bm{q}=f_{q}(\widehat{\mathcal{G}}_{q})$ and
$\bm{k}=f_{k}(\widehat{\mathcal{G}}_{k})$. Here, we follow (He et al., 2020)
to set two different encoders and use a momentum-based update strategy, which
maintains a queue of samples from preceding mini-batches. Formally, the
parameters of $f_{q}$ and $f_{k}$ are denoted as $\mathbf{\Theta}_{q}$ and
$\mathbf{\Theta}_{k}$, respectively. We update $\mathbf{\Theta}_{k}$ by
$\mathbf{\Theta}_{k}\leftarrow m\mathbf{\Theta}_{k}+(1-m)\mathbf{\Theta}_{q}$,
where $m\in[0,1)$ is the momentum hyper-parameter.

Note that our goal is to pre-train a more capable KG encoder $f_{q}$, while
$f_{k}$ is introduced to improve the training of $f_{q}$, which will be
discarded after pre-training.

###  3.2. Multi-grained Hypergraph for Historical User Interest Modeling

In this subsection, we first introduce the general hypergraph convolution for
feature transformation and aggregation. Then, we propose multi-grained
hypergraph convolution for user interest modeling, with the session-based
hypergraph aggregating coarse-grained user preferences beneath historical
data, and the knowledge-based hypergraph capturing fine-grained user interest
on the KG.

####  3.2.1. Hypergraph Convolution

In this paper, we develop a hypergraph convolutional network to capture high-
order relations. Following (Feng et al., 2019; Bai et al., 2021), we define
our hypergraph convolution as:

(3)
$\bm{x}_{i}^{(l+1)}=\sum_{v=1}^{|\mathcal{V}|}\sum_{h=1}^{|\mathcal{H}|}\text{H}_{i,h}\text{H}_{v,h}\bm{x}_{v}^{(l)}\mathbf{W}^{(l)},$

where $\bm{x}_{i}^{(l)}$ is the embedding of the $i$-th vertex in the $l$-th
layer, $\mathbf{W}^{(l)}\in\mathbb{R}^{d\times d}$ is the weight matrix
between the $l$-th and the $(l+1)$-th layer. However, the scale of the vertex
embeddings may change during training, resulting in numerical instabilities.
Therefore, we introduce a proper normalization and rewrite the convolution
operation in the form of the matrix:

(4)
$\mathbf{X}^{(l+1)}=\mathbf{D}^{-1}\mathbf{H}\mathbf{B}^{-1}\mathbf{H}^{\top}\mathbf{X}^{(l)}\mathbf{W}^{(l)},$

where $\mathbf{X}^{(l)}$ and $\mathbf{X}^{(l+1)}$ are the input of the $l$-th
and $(l+1)$-th layers, respectively.

Firstly, the vertex features are transformed by weight matrix $\mathbf{W}$.
Then, the vertex features are aggregated to form the hyperedge features by
transposed incidence matrix $\mathbf{H}^{\top}$, and the related hyperedge
features are aggregated to generate refined vertex features by $\mathbf{H}$.
The degree matrices for vertex and hyperedge (_i.e.,_ $\mathbf{D}$,
$\mathbf{B}$) are introduced for normalization. Therefore, the hypergraph
convolution can be viewed as a two-stage refinement process, performing “
_vertex-hyperedge-vertex_ ” feature transformation upon hypergraph structure.
Note that we do not use nonlinear activation function (_e.g.,_ ReLU) for
hypergraph convolution following (Wu et al., 2019a; Xia et al., 2021).

Figure 2. Illustrative examples of how we build session-based and knowledge-
based hypergraphs based on the items extracted from historical dialogue
sessions.

####  3.2.2. Session-based Hypergraph Convolution

To enrich the limited context of the current dialogue session, we leverage the
historical dialogue sessions of a user for preference modeling. We observe
that each session usually concentrates on a single topic. For example, when a
user is looking for a science fiction, the items that appear in the
conversation utterances would be related to that topic. In addition, different
sessions from one certain user probably share common items, which indicates
intrinsic user preference.

Inspired by this observation, we propose to model each historical dialogue
session as a hyperedge. Specifically, the items that appear in the same
session are extracted as vertices to build a hyperedge, with the chronological
order of the items being ignored. Then, all the hyperedges corresponding to
the user connect with each other via shared items, which constitute a session-
based hypergraph. For example, in Figure 2, both sessions contain two movie
items, which correspond to the two hyperedges in the session-based hypergraph,
respectively. Moreover, the item “Inception (2010)” connects the two
hyperedges which appear in both sessions.

Formally, the items that appear in the historical dialogue sessions constitute
a historical item set $\mathcal{I}_{H}$, and for session-based hypergraph
composed of $\mathcal{I}_{H}$, the incidence matrix is denoted by
$\mathbf{H}_{S}$, the diagonal matrices of the vertex degrees and the edge
degrees are denoted as $\mathbf{D}_{S}$ and $\mathbf{B}_{S}$, respectively. We
extract the representations of $\mathcal{I}_{H}$ from encoded entity
embeddings as $\mathbf{N}_{H}$, which is fed into the session-based hypergraph
convolution:

(5)
$\mathbf{N}_{S}=\textbf{HConv}(\mathbf{H}_{S},\mathbf{D}_{S},\mathbf{B}_{S},\mathbf{N}_{H}),$

where $\textbf{HConv}(\cdot)$ is a hypergraph convolution defined by Equation
(4), and $\mathbf{N}_{S}$ is the embedding matrix enhanced by session-based
hypergraph convolution for historical item set $\mathcal{I}_{H}$.

After the _vertex-hypergraph-vertex_ feature transformation upon the above
session-based hypergraph, where items from the entire session form the
hyperedge, the item representations aggregate session-level semantics from
historical dialogue sessions from such a coarse-grained perspective.

####  3.2.3. Knowledge-based Hypergraph Convolution

In a real-world scenario, the number of historical dialogues related to the
users follows a long-tail distribution (Anderson, 2006), which indicates that
most of the users only interact with the system a few times. In this case, the
contextual information provided by historical dialogue sessions may be
insufficient for discovering intrinsic user preferences. Therefore, we propose
to explore user interest upon external KG.

Specifically, we model each historical item (_i.e.,_ the items from the
historical item set $\mathcal{I}_{H}$) and its $N$-hop neighbors as a
hyperedge. The motivation is that the vertex and the extended neighbors
probably share common semantic meaning. Then, all of the hyperedges derived
from historical items connect with each other via shared entities, which
constitute a knowledge-based hypergraph. The anchor nodes which connect
hyperedges play an important role in aggregating user interest upon KG, since
they may be common attributes shared by different historical items. For
example, in Figure 2, the three items that appear in the historical sessions
and their neighbors on KG constitute a knowledge-based hypergraph. Each
hyperedge contains a specific item and several entities, and the entities are
shared between hyperedges.

Given a knowledge-based hypergraph, it includes a vertex set $\mathcal{E}_{K}$
composed of historical items and their $N$-hop neighbors. The incident matrix
is denoted by $\mathbf{H}_{K}$, and the diagonal matrices of the vertex
degrees and the edge degrees are denoted as $\mathbf{D}_{K}$ and
$\mathbf{B}_{K}$. The representations of $\mathcal{E}_{K}$ constitute an
embedding matrix $\mathbf{N}^{\prime}_{K}$, which is subsequently fed to
knowledge-based hypergraph convolution:

(6)
$\widetilde{\mathbf{N}}_{K}=\textbf{HConv}(\mathbf{H}_{K},\mathbf{D}_{K},\mathbf{B}_{K},\mathbf{N}^{\prime}_{K}),$

where $\widetilde{\mathbf{N}}_{K}$ is the embedding matrix enhanced by the
knowledge-based hypergraph convolution for $\mathcal{E}_{K}$. Then, we
construct representations of historical items from
$\widetilde{\mathbf{N}}_{K}$, denoted by $\mathbf{N}_{K}$. Mean pooling is
performed on each hyperedge to obtain item representations.

After the message passing upon the above knowledge-based hypergraph, where a
single item and its neighbors on the KG form the hyperedge, the item
representations aggregate similar semantics from triplets stored in the KG,
which promotes capturing entity-level user interest from historical dialogue
sessions from such a fine-grained perspective.

####  3.2.4. Hyperedge Extension with Similar Dialogues

To further alleviate the scarcity of user historical dialogues, we propose to
perform hyperedge extension with similar dialogues based on interactions with
common items. The basic idea is that common items in dialogues usually
correspond to similar preferences among users.

Specifically, we extract items from dialogues and build hyperedges, which form
a hyperedge collection. Then, based on the common item ratio between the
current dialogue and the hyperedges in the hyperedge collection, a certain
number of hyperedges are selected for extension. The extended hypergraphs are
further used to construct the hypergraph convolution as described before. We
devise an adaptive method to determine the scale of hyperedge extension. If
the size of a hypergraph is relatively small, we tend to extend it with a
smaller number of hyperedges. Such a strategy prevents incorporating too much
noise, thus leading to more accurate modeling of the current hyperedge.

Above, we model historical data as session-based and knowledge-based
hypergraphs, which provide coarse-grained and fine-grained perspectives for
capturing user preferences. Such a way allows our model to fuse user interest
considering multi-grained semantics, and obtain informative representations
for recommendation.

###  3.3. User Interest-Aware CRS

In this subsection, we fuse the item representations enhanced by the proposed
multi-grained hypergraph convolution (Section 3.2) to obtain user
representation. Based on this, we further build a user interest-aware CRS,
which consists of an item recommender and a response generator.

####  3.3.1. User Representation via Hypergraph-Aware Attention

Since user’s historical interests tend to be diverse, historical dialogue
sessions may not be related to the current user interest. For example, a user
has watched science fictions, comedies and cartoons in the past, but now only
wants to watch a science fiction. Our goal is to leverage the related
historical interest for enhancing the modeling of current user interest. For
this purpose, we propose a hypergraph-aware multi-head attention layer to
integrate historical item representations with current entity representations.

Recall that we have learned session-based and knowledge-based item
representations, _i.e.,_ $\mathbf{N}_{S}$ (Equation (5)) and $\mathbf{N}_{K}$
(Equation (6)), respectively, based on the hypergraph structure. We next take
the entity representations from the current dialogue session denoted by
$\mathbf{N}_{C}$ as the query to attend to both the item representations in
$\mathbf{N}_{S}$ and $\mathbf{N}_{K}$. Formally, we calculate the integrated
representations of historical items as:

(7)
$\mathbf{N}_{SK}=\text{MHA}(\mathbf{N}_{C},[\mathbf{N}_{S};\mathbf{N}_{K}],[\mathbf{N}_{S};\mathbf{N}_{K}]),$

where $\text{MHA}(\mathbf{Q},\mathbf{K},\mathbf{V})$ defines a multi-head
attention function which takes a query matrix $\mathbf{Q}$, a key matrix
$\mathbf{K}$ and a value matrix $\mathbf{V}$ as input, following (Vaswani et
al., 2017).

Such a way can leverage related entity tastes from the constructed hypergraph
structures, considering both multi-grained (_session-_ and _entity-level_)
semantics. Thus, the current user interest can be enhanced by fusing related
historical preferences. Then, we adopt a pooling layer (_e.g.,_ mean pooling)
to fuse historical and current user interests, and obtain the final user
representation $\bm{u}$:

(8) $\bm{u}=\text{Pooling}([\text{Pooling}(\mathbf{N}_{SK});\mathbf{N}_{C}]).$

####  3.3.2. Item Recommendation

Above, we have obtained the user representation enhanced by multi-grained
hypergraph convolution, considering both session-level and entity-level
semantics from historical dialogue sessions. We further calculate the
probabilities that recommend the items to user $u$:

(9) $P_{rec}=\text{Softmax}(\bm{u}\cdot\mathbf{N}_{I}^{\top}),$

where $\mathbf{N}_{I}$ denotes the embeddings of all the candidate items from
item set $\mathcal{I}$, which is encoded with pre-training following Section
3.1.

To learn the parameters of the model, we adopt cross-entropy loss as the
objective function:

(10)
$\mathcal{L}_{rec}=-\sum_{j=1}^{B}\sum_{i=1}^{|\mathcal{I}|}[-(1-y_{ij})\cdot\log(1-P_{rec}^{(j)}(i))+y_{ij}\cdot\log(P_{rec}^{(j)}(i))],$

where $B$ is the size of mini-batch, $y_{ij}\in\\{0,1\\}$ is the target label.

####  3.3.3. Response Generation

Following prior works (Chen et al., 2019; Zhou et al., 2020a), we adopt
Transformer (Vaswani et al., 2017) to develop an encoder-decoder framework for
conversation task. We introduce two separate encoders to encode historical and
current dialogues, respectively. Then, the learned representations are fed
into the decoder as cross-attention signals.

Formally, in each decoder layer, we first obtain text representations after
self-attention and cross-attention layers:

(11) $\displaystyle\mathbf{A}_{0}^{n}$ $\displaystyle=$
$\displaystyle\text{MHA}(\mathbf{R}^{n-1},\mathbf{R}^{n-1},\mathbf{R}^{n-1}),$
(12) $\displaystyle\mathbf{A}_{1}^{n}$ $\displaystyle=$
$\displaystyle\text{MHA}(\mathbf{A}_{0}^{n},\mathbf{N}_{SK},\mathbf{N}_{SK}),$

where $\mathbf{R}^{n-1}$ is the embedding matrix from the decoder at
$(n-1)$-th layer, and $\mathbf{N}_{SK}$ is the item representations of the
current dialogue enhanced by multi-grained hypergraph convolution (Equation
(7)). Then, we fuse the representations of historical and current dialogue
sessions into decoder. To avoid overfitting on historical dialogues, we
introduce a hyper-parameter $\beta$ to achieve the trade-off between these two
types of signals:

(13) $\displaystyle\mathbf{A}_{2}^{n}$ $\displaystyle=$
$\displaystyle\text{MHA}(\mathbf{A}_{1}^{n},\mathbf{X}_{C},\mathbf{X}_{C}),$
(14) $\displaystyle\mathbf{A}_{3}^{n}$ $\displaystyle=$
$\displaystyle\text{MHA}(\mathbf{A}_{1}^{n},\mathbf{X}_{H},\mathbf{X}_{H}),$
(15) $\displaystyle\mathbf{A}_{4}^{n}$ $\displaystyle=$
$\displaystyle\beta\cdot\mathbf{A}_{2}^{n}+(1-\beta)\cdot\mathbf{A}_{3}^{n},$

where $\mathbf{X}_{C}$ is the embedding matrix output by the current dialogue
session encoder, and $\mathbf{X}_{H}$ is the embedding matrix output by the
historical dialogue session encoder. Note that for the conversation module,
the main difference between our method and KGSF (Zhou et al., 2020a) is that
we adopt two separate encoders to encode the current and the historical
dialogues respectively, and fuse the item representations enhanced by multi-
grained hypergraph convolution into the decoder. Finally, we obtain decoder
layer output through a feed-forward network layer following (Vaswani et al.,
2017):

(16) $\mathbf{R}^{n}=\text{FFN}(\mathbf{A}_{4}^{n}).$

Furthermore, the generated responses are expected to reflect user interest and
contain diverse recommended items. Therefore, we adopt a user interest-aware
bias and another item-related bias generated by the copy mechanism. Given the
predicted sequence $y_{1},\ldots,y_{i-1}$, the next token probability is
calculated as:

(17)
$P_{gen}(y_{i}|y_{1},\ldots,y_{i-1})=P_{1}(y_{i}|\mathbf{R}_{i})+P_{2}(y_{i}|\bm{u})+P_{3}(y_{i}|\mathbf{R}_{i},\bm{u}),$

where $P_{1}(\cdot)$ is the vocabulary probability generated by taking the
decoder output $\mathbf{R}_{i}$ as input, $P_{2}(\cdot)$ is the vocabulary
bias generated by user representation $\bm{u}$, and $P_{3}(\cdot)$ denotes the
copy probability where the scores of non-item vocabularies are set to $0$.
Finally, the conversation module is trained with the cross-entropy loss:

(18)
$\mathcal{L}_{gen}=-\sum_{i=1}^{B}\sum_{t=1}^{T}\log(P_{gen}(y_{t}|y_{1},\ldots,y_{t-1})),$

where $B$ is the batch size, and $T$ is the truncated length of utterances.

##  4\. EXPERIMENT

To verify the effectiveness of the proposed method MHIM, we conduct extensive
experiments and provide detailed analysis.

Table 1. Statistics of the datasets in our experiments. Dataset | #Dialogues | #Users | #Items | Sparsity
---|---|---|---|---
ReDial (Li et al., 2018) | 11,348 | 956 | 6,924 | 99.9843%
TG-ReDial (Zhou et al., 2020b) | 10,000 | 1,482 | 33,834 | 99.9973%
Table 2. Experimental results on recommendation task. * indicates statistically significant improvement ($p<0.05$) over all baselines. Source refers to the dataset or external knowledge related to the method, where D refers to datasets ReDial and TG-ReDial, E and W refer to the entity-level KG (_i.e.,_ DBpedia and CN-DBpedia) and word-level KG (_i.e.,_ ConceptNet and HowNet), respectively. We abbreviate Recall@K, MRR@K and NDCG@K as R@K, M@K and N@K, respectively.  Source | Model | ReDial | TG-ReDial
---|---|---|---
R@10 | R@50 | M@10 | M@50 | N@10 | N@50 | R@10 | R@50 | M@10 | M@50 | N@10 | N@50
D | TextCNN (Kim, 2014) | 0.0644 | 0.1821 | 0.0235 | 0.0285 | 0.0328 | 0.0580 | 0.0097 | 0.0208 | 0.0040 | 0.0045 | 0.0053 | 0.0077
D | SASRec (Kang and McAuley, 2018) | 0.1117 | 0.2329 | 0.0540 | 0.0593 | 0.0674 | 0.0936 | 0.0043 | 0.0178 | 0.0011 | 0.0017 | 0.0019 | 0.0047
D | BERT4Rec (Sun et al., 2019) | 0.1285 | 0.3032 | 0.0475 | 0.0555 | 0.0663 | 0.1045 | 0.0043 | 0.0226 | 0.0013 | 0.0020 | 0.0020 | 0.0058
D | ReDial (Li et al., 2018) | 0.1705 | 0.3077 | 0.0677 | 0.0738 | 0.0925 | 0.1222 | 0.0038 | 0.0165 | 0.0012 | 0.0017 | 0.0018 | 0.0045
D | TG-ReDial (Zhou et al., 2020b) | 0.1679 | 0.3327 | 0.0694 | 0.0771 | 0.0924 | 0.1286 | 0.0110 | 0.0174 | 0.0048 | 0.0050 | 0.0062 | 0.0076
D, E | KBRD (Chen et al., 2019) | 0.1796 | 0.3421 | 0.0722 | 0.0800 | 0.0972 | 0.1333 | 0.0201 | 0.0501 | 0.0077 | 0.0090 | 0.0106 | 0.0171
D, E, W | KGSF (Zhou et al., 2020a) | 0.1785 | 0.3690 | 0.0705 | 0.0796 | 0.0956 | 0.1379 | 0.0215 | 0.0643 | 0.0069 | 0.0087 | 0.0103 | 0.0194
D, E | KGConvRec (Sarkar et al., 2020) | 0.1819 | 0.3587 | 0.0711 | 0.0794 | 0.0969 | 0.1358 | 0.0220 | 0.0524 | 0.0088 | 0.0102 | 0.0119 | 0.0185
D, E | KECRS (Zhang et al., 2022) | 0.1746 | 0.3708 | 0.0654 | 0.0748 | 0.0908 | 0.1344 | 0.0234 | 0.0615 | 0.0069 | 0.0086 | 0.0107 | 0.0190
D | BERT (Devlin et al., 2019) | 0.1608 | 0.3525 | 0.0597 | 0.0688 | 0.0831 | 0.1255 | 0.0040 | 0.0194 | 0.0011 | 0.0017 | 0.0018 | 0.0050
D | XLNet (Yang et al., 2019) | 0.1569 | 0.3590 | 0.0583 | 0.0677 | 0.0811 | 0.1255 | 0.0040 | 0.0187 | 0.0011 | 0.0017 | 0.0017 | 0.0048
D | BART (Lewis et al., 2020) | 0.1693 | 0.3783 | 0.0646 | 0.0744 | 0.0888 | 0.1350 | 0.0047 | 0.0187 | 0.0012 | 0.0017 | 0.0020 | 0.0048
D, E | MHIM | 0.1966* | 0.3832* | 0.0742* | 0.0830* | 0.1027* | 0.1440* | 0.0300* | 0.0783* | 0.0108* | 0.0129* | 0.0152* | 0.0256*

###  4.1. Experiment Setup

####  4.1.1. Datasets

We evaluate the proposed model on ReDial (Li et al., 2018) and TG-ReDial (Zhou
et al., 2020b) datasets. ReDial is an English conversational recommendation
dataset constructed through Amazon Mechanical Turk by crowd workers under a
set of comprehensive instructions. TG-ReDial is a Chinese conversational
recommendation dataset created semi-automatically. The statistics of both
datasets are shown in Table 1. To avoid overfitting certain user histories, we
rebuild the dataset into a more strict setting by separating the data based on
user_id and truncating the number of historical dialogues to a certain
limitation. The rebuilt dataset is also split into training, validation, and
test sets in a proportion of 8:1:1. For each conversation, we start from the
first sentence one by one to generate reply utterances or give recommendations
by our model. Moreover, we incorporate open-source knowledge base DBpedia
(Lehmann et al., 2015) and CN-DBpedia (Xu et al., 2017) as the external KG.

####  4.1.2. Baselines

In CRS, we consider two major tasks to evaluate the superiority of our
proposed model, namely the recommendation task and the conversation task.
Therefore, we compare our approach with existing CRS methods, as well as
several representative recommendation and conversation models.

$\bullet$ TextCNN (Kim, 2014) adopts a CNN-based model to extract personalized
features from contextual utterances as user embeddings.

$\bullet$ SASRec (Kang and McAuley, 2018) adopts the self-attention layer to
capture the dynamic patterns in user interaction sequences.

$\bullet$ BERT4Rec (Sun et al., 2019) adapts the original BERT (Devlin et al.,
2019) model with a cloze objective loss for sequential recommendation.

$\bullet$ Transformer (Vaswani et al., 2017) adopts a Transformer-based
encoder-decoder method to generate conversational responses.

$\bullet$ ReDial (Li et al., 2018) consists of a dialogue generation module
based on HRED (Sordoni et al., 2015) and a recommender module based on auto-
encoder (Sedhain et al., 2015).

$\bullet$ TG-ReDial (Zhou et al., 2020b) presents the task of topic-guide
conversational recommendation, and utilizes both historical interaction and
dialogue text for deriving user preference in recommender module.

$\bullet$ KBRD (Chen et al., 2019) is a knowledge-based CRS model that
utilizes R-GCN to construct user representations on DBpedia, and ranks the
items by dot-product for recommendations.

$\bullet$ KGSF (Zhou et al., 2020a) is a knowledge-based CRS model that
utilizes both word-oriented and item-oriented KGs, and aligns the two semantic
spaces using Mutual Information Maximization (MIM).

$\bullet$ KGConvRec (Sarkar et al., 2020) incorporates pre-trained entity
embeddings supplemented with positional embeddings to obtain better entity
representations for recommendation.

$\bullet$ KECRS (Zhang et al., 2022) proposes the Bag-of-Entity loss and the
infusion loss to better integrate KGs and generate more diverse responses for
recommendation.

$\bullet$ BERT (Devlin et al., 2019) is a language model pre-trained with the
masked language model task, and we utilize the representation of the [CLS]
token for recommendation.

$\bullet$ XLNet (Yang et al., 2019) is a language model pre-trained using an
auto-regressive method to learn bidirectional contexts, and we utilize the
representation of the [CLS] token for recommendation.

$\bullet$ BART (Lewis et al., 2020) is a language model pre-trained with the
denoising auto-encoding task, and we also use the representation of the [CLS]
token for recommendation.

Among these baselines, Transformer (Vaswani et al., 2017) is the state-of-the-
art text generation method, BERT (Devlin et al., 2019), XLNet (Yang et al.,
2019) and BART (Lewis et al., 2020) are pre-trained language models (PLMs),
TextCNN (Kim, 2014), SASRec (Kang and McAuley, 2018) and BERT4Rec (Sun et al.,
2019) are recommendation methods, and ReDial (Li et al., 2018), TG-ReDial
(Zhou et al., 2020b), KBRD (Chen et al., 2019), KGSF (Zhou et al., 2020a),
KGConvRec (Sarkar et al., 2020), KECRS (Zhang et al., 2022) are CRS methods.
Besides, we do not compare UCCR (Li et al., 2022) because in their
experiments, different sessions of one user are divided into train, valid, and
test sets, and the user preferences are memorized by model parameters.
However, we strictly separate users based on data partition, and our method
can capture user interest from historical data online. Therefore, it is
unfeasible for these two methods to achieve a fair comparison under the same
data partition.

####  4.1.3. Evaluation Metrics

In our experiments, we adopt different metrics to evaluate the two tasks. For
the recommendation task, we evaluate whether our approach is able to provide
item recommendations accurately. Thus, we adopt Recall@$K$, MRR@$K$, NDCG@$K$
for evaluation ($k$=10, 50). For the conversation task, we use Distinct
$n$-gram ($n$=2, 3, 4) to measure the degree of diversity for text tokens,
which is calculated as the number of distinct $n$-grams scaled by the total
number of sentences in the test set.

####  4.1.4. Implementation Details

We implement our approach with PyTorch111https://pytorch.org/. For text
processing, the lengths of the current and the historical dialogue utterances
are truncated to 256 and 1024, respectively. The dimensions of embeddings are
set to 300 and 128, respectively, for conversation and recommender modules.
The number of layers is set to 1 for R-GCN (Schlichtkrull et al., 2018) and
hypergraph convolution considering effectiveness and efficiency, and the
normalization constant of R-GCN is set to 1. The trade-off hyper-parameter
$\beta$ is set to 0.9. We use Adam (Kingma and Ba, 2015) optimizer with the
default parameter setting, and the learning rate is set to 0.001. For
recommendation, the batch size is set to 256 and 64 on ReDial and TG-ReDial
respectively, and for conversation, the batch size is consistently set to 128.

For R-GCN pre-training, we generate subgraphs using the random walk API
provided by DGL (Wang et al., 2019), the random walk hop is set to 128 with a
restart probability of 0.5. We pre-train our R-GCN for 120 steps and use Adam
optimizer with learning rate of 0.005, $\beta_{1}=0.9$, $\beta_{2}=0.999$,
weight decay of 1e-4, and learning rate warm-up over the first $10\%$ steps.
We use a batch size of 1024, dictionary size of 16384, temperature of 0.07,
and momentum of 0.999.

###  4.2. Evaluation on Recommendation Task

In this subsection, we conduct a series of experiments to verify the
effectiveness of our proposed model MHIM for the recommendation task. The
results are presented in Table LABEL:rec_result.

Table 3.  Experiment results of ablation and variation study of our model on recommendation task. We report the results of Recall@K, which is abbreviated as R@K.  Model | ReDial | TG-ReDial
---|---|---
R@10 | R@50 | R@10 | R@50
MHIM | 0.1966 | 0.3832 | 0.0300 | 0.0783
w/o Contrast | 0.1946 | 0.3777 | 0.0218 | 0.0605
w/o Session | 0.1943 | 0.3816 | 0.0266 | 0.0713
w/o Knowledge | 0.1944 | 0.3791 | 0.0286 | 0.0767
w/o HyperConv | 0.1925 | 0.3823 | 0.0252 | 0.0711
w/o Extension | 0.1975 | 0.3829 | 0.0287 | 0.0774

####  4.2.1. Result Analysis

Table LABEL:rec_result shows the experiment results of different methods on
recommendation task. As we can see, the CRS methods outperform recommendation
methods generally (_e.g.,_ BERT4Rec). The reason might be that recommendation
methods utilize item interaction records to capture user preferences, while
the CRS methods further incorporate textual information from dialogues.
Moreover, the CRS methods integrate the recommendation module and the
conversation module seamlessly, which are mutually beneficial to each other.
For the CRS methods, we can see that the KG-enhanced methods KBRD, KGSF,
KGConvRec and KECRS perform better than ReDial and TG-ReDial. This is because
the KG bridges the gap between unstructured textual information and structural
semantics, which promotes the user preference modeling. For the pre-trained
language models BERT, XLNet and BART, even they do not utilize external KG,
they perform as well as the CRS methods. One possible reason might be that the
pre-training stage on the large-scale text data learns prior knowledge which
is beneficial to the downstream recommendation task. Moreover, we notice that
the KG-enhanced CRS methods outperform the other baselines by a large margin
on the TG-ReDial dataset. One reason is that the item interactions on TG-
ReDial are sparser than ReDial (Table 1), and therefore, the enhancement
effect of the KGs is more obvious.

Our proposed method MHIM outperforms all the baselines. We improve the KG
encoder via contrastive subgraph discrimination, and learn informative user
representations via multi-grained hypergraph convolution for both
recommendation and conversation tasks. Compared to KGSF, even though we do not
utilize external word-level KG, our method outperforms it significantly.
Compared to PLM-based methods, our method is lighter and runs faster.

####  4.2.2. Ablation Study

In order to evaluate the effectiveness of each component, we conduct the
ablation study based on different variants of MHIM, including: (1) MHIM w/o
Contrast removes the contrastive pre-training stage (Section 3.1.2) of the KG
encoder; (2) MHIM w/o Session removes the session-based hypergraphs (Section
3.2.2); (3) MHIM w/o Knowledge removes the knowledge-based hypergraphs
(Section 3.2.3); (4) MHIM w/o HyperConv removes the hypergraph convolution
(Equation (4)) on both session- and knowledge-based hypergraphs; (5) MHIM w/o
Extension removes the hyperedge extension procedure (Section 3.2.4). Note that
compared to (2) and (3), MHIM w/o HyperConv only removes the convolution
operations performed on hypergraphs, and remains the original representations
of items in both session- and knowledge-based hypergraphs. Our motivation for
designing variant (4) is to validate the effectiveness of hypergraph
convolution itself.

The results are shown in Table 3. Firstly, we can observe that removing the
pre-training stage of the KG encoder leads to the largest performance
decrease, which promotes the generalization performance of GNN module. Another
observation is that both session- and knowledge-based hypergraphs and the
hypergraph convolution lead to increased performance, which capture multi-
grained user interest for recommendation. Finally, we notice that for metric
Recall@10, MHIM w/o Extension performs better on ReDial. One possible reason
is that the hypergraph extension introduces noise, which adversely affects
item recommendation.

Figure 3. Performance (_i.e.,_ Recall@10 and Recall@50) comparison of MHIM and
the variant without contrastive pre-training on the ReDial dataset.

####  4.2.3. The Effect of Pre-training Technique

We adopt the contrastive learning technique for pre-training the KG encoder.
As shown in Table 3, it significantly contributes to the final performance,
since it benefits multi-grained representation learning for entities. We would
like to further study whether the improvement is consistent with the increase
of the iteration number. Therefore, we gradually increase the iteration number
on the train set, and report the corresponding evaluation metrics (_i.e.,_
Recall@10 and Recall@50) on the test set. As shown in Figure 3, our model can
achieve an equal performance with fewer iterations compared to the variant
without pre-training, and finally outperforms the latter.

###  4.3. Evaluation on Conversation Task

In this section, we verify the effectiveness of our proposed model MHIM for
the conversation task. We present the results of the evaluation metrics for
different methods in Table 4.

####  4.3.1. Result Analysis

As we can see, among the four baselines, the performance order is consistent
with KBRD ¿ KGSF ¿ Transformer ¿ ReDial. The reason is that KBRD introduces
KG-based vocabulary bias to generate responses that are more consistent with
user interest, and KGSF incorporates cross-attention with embeddings from
entity- and word-level KGs to develop KG-enhanced decoder. However,
Transformer and ReDial only utilize token sequences, ignoring user preferences
hidden under the entities. Compared with these baselines, our model MHIM
consistently performs better. In our approach, the cross-attention layer and
the user interest-aware bias can effectively inject the user preferences
learned from multi-grained hypergraph convolution into the decoder. Therefore,
our model can generate diverse responses consistent with user interest.

Table 4. Experimental results on conversation task. * indicates statistically significant improvement ($p<0.05$) over all baselines.. We abbreviate Distinct-2,3,4 as Dist-2,3,4, and “Trans.” refers to the Transformer model.  Model | ReDial | TG-ReDial
---|---|---
Dist-2 | Dist-3 | Dist-4 | Dist-2 | Dist-3 | Dist-4
ReDial | 0.0214 | 0.0659 | 0.1333 | 0.2178 | 0.5136 | 0.7960
Trans. | 0.0538 | 0.1574 | 0.2696 | 0.2362 | 0.7063 | 1.1800
KBRD | 0.0765 | 0.3344 | 0.6100 | 0.8013 | 1.7840 | 2.5977
KGSF | 0.0572 | 0.2483 | 0.4349 | 0.3891 | 0.8868 | 1.3337
MHIM | 0.3278* | 0.6204* | 0.9629* | 1.1100* | 2.3520* | 3.8200*
Figure 4.  Experiment results of Distinct-2,3,4, abbreviated as Dist-2,3,4, on
the ReDial dataset about the conversation task, and the ablation study shows
that MHIM consistently outperforms its variants w/o Attn and w/o Bias.

####  4.3.2. Ablation Study

We also conduct ablation study to evaluate the effectiveness of each component
for conversation task on the ReDial dataset. We devise two variants of our
model: (1) MHIM w/o Attn removes the cross-attention layer considering item
representations enhanced by multi-grained hypergraph convolution (Equation
(12)); (2) MHIM w/o Bias removes the user interest-aware bias for token
prediction (Equation (17)). We report the results of Distinct-2,3,4 for
evaluation.

The results are shown in Figure 4. We can observe that removing any component
leads to performance decrease, indicating that both components are beneficial
for user preference modeling and contribute to more diverse responses. In
addition, removing the cross-attention layer leads to larger performance
decrease, which evaluates the effectiveness of the proposed multi-grained
hypergraph convolution for item recommendation.

###  4.4. Contrastive Pre-training Settings

As introduced above, the contrastive pre-training procedure for R-GCN encoder
is applied on the large-scale, extended KG. However, if memory is allocated to
each node, the storage and time costs will be unacceptable. Fortunately, some
critical nodes are more likely to appear in random walk sequences, potentially
connecting to more edges in the KG. As a result, we only allocate memory space
to critical nodes that appear frequently, implying that the other nodes are
associated with a specific __UNKNOWN__ embedding.

To determine the number of critical nodes, we conduct a series of experiments
based on different critical node numbers from 0 to 3,000,000 on DBpedia. As we
can see in Figure 5, while the node number is set to 600,000, the pre-training
loss and the Recall@50 in the recommendation task on ReDial both reach the
inflection point, and furthermore, the storage cost is slightly higher than
the group whose critical nodes number is 0. Therefore, we retain 600,000
critical nodes on DBpedia in our experiments. Similarly, we retain 1,540,000
critical nodes on CN-DBpedia for TG-ReDial.

Figure 5. The recommendation performances, pre-training loss and storage costs
with different numbers of critical nodes on the ReDial dataset.

##  5\. RELATED WORK

In the following section, we first introduce the prior work on Conversational
Recommender System (CRS) and session-based recommendation. Then, we briefly
review the existing literature on graph representation learning, including
pre-training on graph neural networks and hypergraph learning.

###  5.1. Recommender System

Conversational Recommendation. Conversational recommender systems model user
interest through multi-turn dialogues and provide high-quality
recommendations. Existing studies about CRS can be roughly divided into two
categories, _i.e.,_ attribute-based CRS and generation-based CRS.

Attribute-based CRS (Sun and Zhang, 2018; Zhang et al., 2018; Lei et al.,
2020a, b; Deng et al., 2021; Ren et al., 2021; Xu et al., 2021) typically
captures user preferences by asking queries about item attributes and
generating responses using pre-defined templates (Sun and Zhang, 2018; Lei et
al., 2020a). Most of these methods gradually narrow down the hypothesis space
to search for the proper items within fewer turns. However, this kind of CRS
does not pay enough attention to generating human-like responses in natural
language, which may hurt user experiences.

Generation-based CRSs (Li et al., 2018; Chen et al., 2019; Zhou et al., 2020a;
Liao et al., 2019; Lu et al., 2021; Liang et al., 2021; Li et al., 2022; Zhou
et al., 2022; Deng et al., 2022; Yang et al., 2022; Wang et al., 2022a; Sarkar
et al., 2020; Zhang et al., 2022) alleviate this problem by adopting the
Seq2Seq architecture (Sordoni et al., 2015; Vaswani et al., 2017) to generate
fluent utterances as responses, which constructs an end-to-end framework for
both conversation task and recommendation task. Researchers release a
benchmark dataset ReDial (Li et al., 2018) which contains human conversations
about movie recommendation. Further studies incorporate external data to
improve user preference modeling and recommendation, including entity-oriented
knowledge graph (Chen et al., 2019; Zhang et al., 2022; Sarkar et al., 2020),
word-oriented knowledge graph (Zhou et al., 2020a) and review information (Lu
et al., 2021). To effectively leverage external data, researchers propose a
coarse-to-fine contrastive learning framework (Zhou et al., 2022) to improve
data semantic fusion. A more recent study (Li et al., 2022) first highlights
that the user’s historical dialogue sessions and look-alike users are
essential for user preference modeling.

However, the user interest that lies beneath complicated historical data has
yet to be comprehensively captured. Our work extends the second category of
research by leveraging historical dialogue sessions and large-scale external
knowledge. The key novelty lies in the user interest modeling through multi-
grained hypergraph convolution, which can effectively model historical user
interest for better recommendation.

Session-based Recommendation. Session-based recommendation focuses on
capturing dynamic user interests for recommendation based on short-term
sessions, where a session refers to multiple user-item interactions that
happen over a short period of time. GRU4Rec (Hidasi et al., 2016) utilizes
gated recurrent units (GRUs) to model user behaviors sequentially, which helps
to utilize complex intra- and inter-session relations for recommendation. NARM
(Li et al., 2017) proposes to incorporate an attention mechanism into
recurrent neural networks (RNNs) to capture user interests accurately.
Recently, graph neural networks (GNNs) have been adopted for session-based
recommendation (Wu et al., 2019b; Xu et al., 2019; Song et al., 2019), because
of their great potential in modeling complex graph-structured context data.
Moreover, researchers propose to extend the session-based scenarios to multi-
session-based scenarios (Wang et al., 2022b), integrating more context
information for accurate recommendation. Compared with session-based
recommendation, our work considers a conversation user takes part in as a
session, and captures user interest by modeling short-term sequences for
conversational recommendation.

###  5.2. Graph Representation Learning

Graph Neural Network. Due to their tremendous capacity to model graph-
structured data, Graph Neural Networks (GNNs) have gained a lot of attention
in recent years. Existing GNN methods can be divided into spectral methods
(Kipf and Welling, 2017) and spatial methods (Veličković et al., 2018;
Hamilton et al., 2017; Schlichtkrull et al., 2018). Most methods follow a
message passing (Gilmer et al., 2017) scheme to aggregate structural
information from nodes’ neighbors. Though GNNs are effective for modeling
graph data, they usually require abundant task-specific data for end-to-end
training. Motivated by the recent advances in pre-training from natural
language processing (Devlin et al., 2019; Brown et al., 2020) and computer
vision (Chen et al., 2020; He et al., 2022), researchers devote efforts to
pre-training on GNNs (Hu et al., 2020b, a; Velickovic et al., 2019; Qiu et
al., 2020; You et al., 2020). The key idea is to pre-train an expressive GNN
encoder on massive unlabeled graph datasets or coarse-grained supervised
datasets. Existing works mainly focus on designing proper pre-training tasks,
_e.g.,_ attribute prediction (Hu et al., 2020b), graph property prediction (Hu
et al., 2020b), graph reconstruction (Hu et al., 2020a) and contrastive
learning (Velickovic et al., 2019; You et al., 2020; Qiu et al., 2020). In the
field of CRS, the training data remains insufficient. Therefore, we propose to
pre-train the graph encoder on large-scale KGs (Lehmann et al., 2015; Xu et
al., 2017) via contrastive learning (Qiu et al., 2020).

Hypergraph Learning. Hypergraph (Bretto, 2013) generalizes the concept of edge
to make it connect more than two nodes, and provides a natural way to capture
high-order relations. It has been explored by combining with promising deep
learning techniques. HGNN (Feng et al., 2019) and HyperGCN (Yadati et al.,
2019) are the first to design hypergraph convolution operations to handle
high-order correlations. The attention mechanism is further introduced to
improve the performance (Bai et al., 2021). There are also several studies
combining hypergraph learning with recommender systems (Wang et al., 2020; Ji
et al., 2020; Xia et al., 2021; Yu et al., 2021; Xia et al., 2022). HyperRec
(Wang et al., 2020) uses hypergraph to model the short-term user preference
for next-item recommendation. DHCF (Ji et al., 2020) captures high-order
correlations among users and items for general collaborative filtering. DHCN
(Xia et al., 2021) further exploits inter-hyperedge information for session-
based recommendation. MHCN (Yu et al., 2021) integrates self-supervised
learning into the training of the hypergraph convolutional network for social
recommendation. Our work is the first to model user interest with hypergraph
learning for conversational recommendation.

##  6\. CONCLUSION AND FUTURE WORK

In this paper, we propose a novel multi-grained hypergraph interest modeling
framework to model user interest for conversational recommendation. By
employing hypergraphs to model historical dialogue sessions and reconstruct
external KGs, we obtain session- and knowledge-based hypergraphs, which help
to comprehensively capture user interest lies beneath complicated historical
conversations from multi-grained perspectives. We then pre-train the KG
encoder with the subgraph instance discrimination task, and learn item
representations by the multi-grained hypergraph convolution. Finally, the
proposed user interest-aware CRS is able to provide proper item recommendation
and give informative responses. Extensive experiments on two datasets show
that our approach yields better performance than several competitive
baselines.

For future work, we consider designing a unified user interest learner
incorporating multi-type historical data. Besides, it is also interesting to
devise an interest transfer module, through which the learned preferences will
benefit each other among different users.

## Acknowledgement

This work was partially supported by National Natural Science Foundation of
China under Grant No. 62222215, Beijing Natural Science Foundation under Grant
No. 4222027, and Beijing Outstanding Young Scientist Program under Grant No.
BJJWZYJH012019100020098.

## References

  * (1)
  * Anderson (2006) Chris Anderson. 2006.  _The long tail: Why the future of business is selling less of more_.  Hachette Books.
  * Bai et al. (2021) Song Bai, Feihu Zhang, and Philip HS Torr. 2021.  Hypergraph convolution and hypergraph attention.  _Pattern Recognition_ 110 (2021), 107637.
  * Benson et al. (2016) Austin R Benson, David F Gleich, and Jure Leskovec. 2016.  Higher-order organization of complex networks.  _Science_ 353, 6295 (2016), 163–166.
  * Bretto (2013) Alain Bretto. 2013.  Hypergraph theory.  _An introduction. Mathematical Engineering. Cham: Springer_ (2013).
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020\.  Language models are few-shot learners.  _Advances in neural information processing systems_ 33 (2020), 1877–1901.
  * Chen et al. (2019) Qibin Chen, Junyang Lin, Yichang Zhang, Ming Ding, Yukuo Cen, Hongxia Yang, and Jie Tang. 2019.  Towards Knowledge-Based Recommender Dialog System. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_. Association for Computational Linguistics, Hong Kong, China, 1803–1813.  https://doi.org/10.18653/v1/D19-1189
  * Chen et al. (2020) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020.  A simple framework for contrastive learning of visual representations. In _International conference on machine learning_. PMLR, 1597–1607.
  * Deng et al. (2021) Yang Deng, Yaliang Li, Fei Sun, Bolin Ding, and Wai Lam. 2021.  Unified conversational recommendation policy learning via graph-based reinforcement learning. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 1431–1441.
  * Deng et al. (2022) Yang Deng, Wenxuan Zhang, Weiwen Xu, Wenqiang Lei, Tat-Seng Chua, and Wai Lam. 2022.  A Unified Multi-task Learning Framework for Multi-goal Conversational Recommender Systems.  _arXiv preprint arXiv:2204.06923_ (2022).
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_. Association for Computational Linguistics, Minneapolis, Minnesota, 4171–4186.  https://doi.org/10.18653/v1/N19-1423
  * Feng et al. (2019) Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. 2019.  Hypergraph neural networks. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 33. 3558–3565.
  * Gilmer et al. (2017) Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017.  Neural message passing for quantum chemistry. In _International conference on machine learning_. PMLR, 1263–1272.
  * Gupta and Sharma (2021) Somil Gupta and Neeraj Sharma. 2021.  Role of Attentive History Selection in Conversational Information Seeking.  _arXiv preprint arXiv:2102.03749_ (2021).
  * Hamilton et al. (2017) William L Hamilton, Rex Ying, and Jure Leskovec. 2017.  Inductive representation learning on large graphs. In _Proceedings of the 31st International Conference on Neural Information Processing Systems_. 1025–1035.
  * He et al. (2022) Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. 2022.  Masked autoencoders are scalable vision learners. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 16000–16009.
  * He et al. (2020) Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020.  Momentum contrast for unsupervised visual representation learning. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 9729–9738.
  * Hidasi et al. (2016) Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016.  Session-based Recommendations with Recurrent Neural Networks. In _4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings_ , Yoshua Bengio and Yann LeCun (Eds.).  http://arxiv.org/abs/1511.06939
  * Hu et al. (2020b) Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. 2020b.  Strategies for Pre-training Graph Neural Networks. In _International Conference on Learning Representations_.  https://openreview.net/forum?id=HJlWWJSFDH
  * Hu et al. (2020a) Ziniu Hu, Yuxiao Dong, Kuansan Wang, Kai-Wei Chang, and Yizhou Sun. 2020a.  Gpt-gnn: Generative pre-training of graph neural networks. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 1857–1867.
  * Ji et al. (2020) Shuyi Ji, Yifan Feng, Rongrong Ji, Xibin Zhao, Wanwan Tang, and Yue Gao. 2020.  Dual channel hypergraph collaborative filtering. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 2020–2029.
  * Kang and McAuley (2018) Wang-Cheng Kang and Julian McAuley. 2018.  Self-attentive sequential recommendation. In _2018 IEEE International Conference on Data Mining (ICDM)_. IEEE, 197–206.
  * Kim (2014) Yoon Kim. 2014.  Convolutional Neural Networks for Sentence Classification. In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Association for Computational Linguistics, Doha, Qatar, 1746–1751.  https://doi.org/10.3115/v1/D14-1181
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015.  Adam: A Method for Stochastic Optimization. In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_ , Yoshua Bengio and Yann LeCun (Eds.).  http://arxiv.org/abs/1412.6980
  * Kipf and Welling (2017) Thomas N. Kipf and Max Welling. 2017.  Semi-Supervised Classification with Graph Convolutional Networks. In _International Conference on Learning Representations (ICLR)_.
  * Lehmann et al. (2015) Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick Van Kleef, Sören Auer, et al. 2015\.  DBpedia–A Large-scale, Multilingual Knowledge Base Extracted from Wikipedia.  _Semantic web_ 6, 2 (2015), 167–195.
  * Lei et al. (2020a) Wenqiang Lei, Xiangnan He, Yisong Miao, Qingyun Wu, Richang Hong, Min-Yen Kan, and Tat-Seng Chua. 2020a.  Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems. In _Proceedings of the 13th International Conference on Web Search and Data Mining_. 304–312.
  * Lei et al. (2020b) Wenqiang Lei, Gangyi Zhang, Xiangnan He, Yisong Miao, Xiang Wang, Liang Chen, and Tat-Seng Chua. 2020b.  Interactive path reasoning on graph for conversational recommendation. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 2073–2083.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Online, 7871–7880.  https://doi.org/10.18653/v1/2020.acl-main.703
  * Li et al. (2017) Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017.  Neural attentive session-based recommendation. In _Proceedings of the 2017 ACM on Conference on Information and Knowledge Management_. 1419–1428.
  * Li et al. (2018) Raymond Li, Samira Ebrahimi Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. 2018.  Towards Deep Conversational Recommendations. In _Advances in Neural Information Processing Systems 31 (NIPS 2018)_.
  * Li et al. (2022) Shuokai Li, Ruobing Xie, Yongchun Zhu, Xiang Ao, Fuzhen Zhuang, and Qing He. 2022.  User-Centric Conversational Recommendation with Multi-Aspect User Modeling. In _Proceedings of the 45nd International ACM SIGIR Conference on Research and Development in Information Retrieval_. Association for Computing Machinery.
  * Liang et al. (2021) Zujie Liang, Huang Hu, Can Xu, Jian Miao, Yingying He, Yining Chen, Xiubo Geng, Fan Liang, and Daxin Jiang. 2021.  Learning Neural Templates for Recommender Dialogue System. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 7821–7833.  https://doi.org/10.18653/v1/2021.emnlp-main.617
  * Liao et al. (2019) Lizi Liao, Ryuichi Takanobu, Yunshan Ma, Xun Yang, Minlie Huang, and Tat-Seng Chua. 2019.  Deep conversational recommender in travel.  _arXiv preprint arXiv:1907.00710_ (2019).
  * Lu et al. (2021) Yu Lu, Junwei Bao, Yan Song, Zichen Ma, Shuguang Cui, Youzheng Wu, and Xiaodong He. 2021.  RevCore: Review-Augmented Conversational Recommendation. In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_. Association for Computational Linguistics, Online, 1161–1173.  https://doi.org/10.18653/v1/2021.findings-acl.99
  * Qiu et al. (2020) Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. 2020.  Gcc: Graph contrastive coding for graph neural network pre-training. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 1150–1160.
  * Qu et al. (2019) C. Qu, L. Yang, M. Qiu, Y. Zhang, C. Chen, W. B. Croft, and M. Iyyer. 2019.  Attentive History Selection for Conversational Question Answering. In _CIKM ’19_.
  * Ren et al. (2021) Xuhui Ren, Hongzhi Yin, Tong Chen, Hao Wang, Zi Huang, and Kai Zheng. 2021.  Learning to ask appropriate questions in conversational recommendation. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 808–817.
  * Sarkar et al. (2020) Rajdeep Sarkar, Koustava Goswami, Mihael Arcan, and John Philip McCrae. 2020.  Suggest me a movie for tonight: Leveraging Knowledge Graphs for Conversational Recommendation. In _Proceedings of the 28th International Conference on Computational Linguistics_. 4179–4189.
  * Schlichtkrull et al. (2018) Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018.  Modeling relational data with graph convolutional networks. In _European semantic web conference_. Springer, 593–607.
  * Sedhain et al. (2015) Suvash Sedhain, Aditya Krishna Menon, Scott Sanner, and Lexing Xie. 2015.  Autorec: Autoencoders meet collaborative filtering. In _Proceedings of the 24th international conference on World Wide Web_. 111–112.
  * Song et al. (2019) Weiping Song, Zhiping Xiao, Yifan Wang, Laurent Charlin, Ming Zhang, and Jian Tang. 2019.  Session-based social recommendation via dynamic graph attention networks. In _Proceedings of the Twelfth ACM international conference on web search and data mining_. 555–563.
  * Sordoni et al. (2015) Alessandro Sordoni, Yoshua Bengio, Hossein Vahabi, Christina Lioma, Jakob Grue Simonsen, and Jian-Yun Nie. 2015.  A hierarchical recurrent encoder-decoder for generative context-aware query suggestion. In _Proceedings of the 24th ACM International on Conference on Information and Knowledge Management_. 553–562.
  * Sun et al. (2019) Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019.  BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer. In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management_ (Beijing, China) _(CIKM ’19)_. ACM, New York, NY, USA, 1441–1450.  https://doi.org/10.1145/3357384.3357895
  * Sun and Zhang (2018) Yueming Sun and Yi Zhang. 2018.  Conversational recommender system. In _The 41st international acm sigir conference on research & development in information retrieval_. 235–244.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017.  Attention is All you Need. In _Advances in Neural Information Processing Systems_ , I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc.  https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
  * Veličković et al. (2018) Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018.  Graph Attention Networks.  _International Conference on Learning Representations_ (2018).
  * Velickovic et al. (2019) Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. 2019.  Deep Graph Infomax.  _ICLR (Poster)_ 2, 3 (2019), 4.
  * Wang et al. (2020) Jianling Wang, Kaize Ding, Liangjie Hong, Huan Liu, and James Caverlee. 2020.  Next-item recommendation with sequential hypergraphs. In _Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval_. 1101–1110.
  * Wang et al. (2019) Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, Tianjun Xiao, Tong He, George Karypis, Jinyang Li, and Zheng Zhang. 2019.  Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks.  _arXiv preprint arXiv:1909.01315_ (2019).
  * Wang et al. (2022a) Ting-Chun Wang, Shang-Yu Su, and Yun-Nung Chen. 2022a.  BARCOR: Towards A Unified Framework for Conversational Recommendation Systems.  _arXiv preprint arXiv:2203.14257_ (2022).
  * Wang et al. (2022b) Zihan Wang, Gang Wu, and Yan Wang. 2022b.  Effectively Using Long and Short Sessions for Multi-Session-based Recommendations.  _arXiv preprint arXiv:2205.04366_ (2022).
  * Wu et al. (2019a) Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. 2019a.  Simplifying graph convolutional networks. In _International conference on machine learning_. PMLR, 6861–6871.
  * Wu et al. (2019b) Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019b.  Session-based recommendation with graph neural networks. In _Proceedings of the AAAI conference on artificial intelligence_ , Vol. 33. 346–353.
  * Xia et al. (2022) Lianghao Xia, Chao Huang, Yong Xu, Jiashu Zhao, Dawei Yin, and Jimmy Huang. 2022.  Hypergraph contrastive collaborative filtering. In _Proceedings of the 45th International ACM SIGIR conference on research and development in information retrieval_. 70–79.
  * Xia et al. (2021) Xin Xia, Hongzhi Yin, Junliang Yu, Qinyong Wang, Lizhen Cui, and Xiangliang Zhang. 2021.  Self-supervised hypergraph convolutional networks for session-based recommendation. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 35. 4503–4511.
  * Xu et al. (2017) Bo Xu, Yong Xu, Jiaqing Liang, Chenhao Xie, Bin Liang, Wanyun Cui, and Yanghua Xiao. 2017.  Cn-dbpedia: A never-ending chinese knowledge extraction system. In _International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems_. Springer, 428–438.
  * Xu et al. (2019) Chengfeng Xu, Pengpeng Zhao, Yanchi Liu, Victor S Sheng, Jiajie Xu, Fuzhen Zhuang, Junhua Fang, and Xiaofang Zhou. 2019.  Graph contextualized self-attention network for session-based recommendation.. In _IJCAI_ , Vol. 19. 3940–3946.
  * Xu et al. (2021) Kerui Xu, Jingxuan Yang, Jun Xu, Sheng Gao, Jun Guo, and Ji-Rong Wen. 2021.  Adapting user preference to online feedback in multi-round conversational recommendation. In _Proceedings of the 14th ACM international conference on web search and data mining_. 364–372.
  * Yadati et al. (2019) Naganand Yadati, Madhav Nimishakavi, Prateek Yadav, Vikram Nitin, Anand Louis, and Partha Talukdar. 2019.  Hypergcn: A new method for training graph convolutional networks on hypergraphs.  _Advances in neural information processing systems_ 32 (2019).
  * Yang et al. (2022) Bowen Yang, Cong Han, Yu Li, Lei Zuo, and Zhou Yu. 2022.  Improving Conversational Recommendation Systems’ Quality with Context-Aware Item Meta-Information. In _Findings of the Association for Computational Linguistics: NAACL 2022_. Association for Computational Linguistics, Seattle, United States, 38–48.  https://doi.org/10.18653/v1/2022.findings-naacl.4
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019.  Xlnet: Generalized autoregressive pretraining for language understanding.  _Advances in neural information processing systems_ 32 (2019).
  * You et al. (2020) Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020.  Graph Contrastive Learning with Augmentations. In _Advances in Neural Information Processing Systems_ , H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 5812–5823.
  * Yu et al. (2021) Junliang Yu, Hongzhi Yin, Jundong Li, Qinyong Wang, Nguyen Quoc Viet Hung, and Xiangliang Zhang. 2021.  Self-supervised multi-channel hypergraph convolutional network for social recommendation. In _Proceedings of the Web Conference 2021_. 413–424.
  * Zhang et al. (2022) Tong Zhang, Yong Liu, Boyang Li, Peixiang Zhong, Chen Zhang, Hao Wang, and Chunyan Miao. 2022.  Toward Knowledge-Enriched Conversational Recommendation Systems. In _Proceedings of the 4th Workshop on NLP for Conversational AI_. Association for Computational Linguistics, Dublin, Ireland, 212–217.  https://doi.org/10.18653/v1/2022.nlp4convai-1.17
  * Zhang et al. (2018) Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, and W Bruce Croft. 2018.  Towards conversational search and recommendation: System ask, user respond. In _Proceedings of the 27th ACM International Conference on Information and Knowledge Management_. ACM, 177–186.
  * Zhou et al. (2020a) Kun Zhou, Wayne Xin Zhao, Shuqing Bian, Yuanhang Zhou, Ji-Rong Wen, and Jingsong Yu. 2020a.  Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusion. In _KDD ’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020_. 1006–1014.
  * Zhou et al. (2020b) Kun Zhou, Yuanhang Zhou, Wayne Xin Zhao, Xiaoke Wang, and Ji-Rong Wen. 2020b.  Towards Topic-Guided Conversational Recommender System. In _Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain, December 8-11, 2020_.
  * Zhou et al. (2022) Yuanhang Zhou, Kun Zhou, Wayne Xin Zhao, Cheng Wang, Peng Jiang, and He Hu. 2022.  C²-CRS: Coarse-to-Fine Contrastive Learning for Conversational Recommender System. In _WSDM_.