# SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog
Understanding and Generation

Wanwei He  Shenzhen Institute of Advanced Technology, Chinese Academy of
SciencesShenzhenChina<EMAIL_ADDRESS>,  Yinpei Dai  Alibaba
GroupBeijingChina<EMAIL_ADDRESS>,  Min Yang  Shenzhen Institute
of Advanced Technology, Chinese Academy of SciencesShenzhenChina
<EMAIL_ADDRESS>,  Jian Sun  Alibaba GroupBeijingChina jian.sun@alibaba-
inc.com ,  Fei Huang  Alibaba GroupBeijingChina<EMAIL_ADDRESS>,  Luo
Si  Alibaba GroupBeijingChina<EMAIL_ADDRESS>and  Yongbin Li  Alibaba
GroupBeijingChina<EMAIL_ADDRESS>
(2022)

###### Abstract.

Recently, pre-training methods have shown remarkable success in task-oriented
dialog (TOD) systems. However, most existing pre-trained models for TOD focus
on either dialog understanding or dialog generation, but not both. In this
paper, we propose SPACE-3, a novel unified semi-supervised pre-trained
conversation model learning from large-scale dialog corpora with limited
annotations, which can be effectively fine-tuned on a wide range of downstream
dialog tasks. Specifically, SPACE-3 consists of four successive components in
a single transformer to maintain a task-flow in TOD systems: (i) a dialog
encoding module to encode dialog history, (ii) a dialog understanding module
to extract semantic vectors from either user queries or system responses,
(iii) a dialog policy module to generate a policy vector that contains high-
level semantics of the response, and (iv) a dialog generation module to
produce appropriate responses. We design a dedicated pre-training objective
for each component. Concretely, we pre-train the dialog encoding module with
span mask language modeling to learn contextualized dialog information. To
capture the structured dialog semantics, we pre-train the dialog understanding
module via a novel tree-induced semi-supervised contrastive learning objective
with the help of extra dialog annotations. In addition, we pre-train the
dialog policy module by minimizing the $\mathcal{L}_{2}$ distance between its
output policy vector and the semantic vector of the response for policy
optimization. Finally, the dialog generation model is pre-trained by language
modeling. Results show that SPACE-3 achieves state-of-the-art performance on
eight downstream dialog benchmarks, including intent prediction, dialog state
tracking, and end-to-end dialog modeling. We also show that SPACE-3 has a
stronger few-shot ability than existing models under the low-resource
setting.111For reproducibility, we release our code, pre-training data and
pre-trained model publicly at https://github.com/AlibabaResearch/DAMO-
ConvAI/tree/main/space-3.

Task-oriented dialog pre-training, dialog understanding, dialog generation,
policy planning

††journalyear: 2022††copyright: rightsretained††conference: Proceedings of the
45th International ACM SIGIR Conference on Research and Development in
Information Retrieval; July 11–15, 2022; Madrid, Spain††booktitle: Proceedings
of the 45th International ACM SIGIR Conference on Research and Development in
Information Retrieval (SIGIR ’22), July 11–15, 2022, Madrid, Spain††doi:
10.1145/3477495.3532069††isbn: 978-1-4503-8732-3/22/07††ccs: Computing
methodologies Discourse, dialogue and pragmatics††ccs: Computing methodologies
Natural language generation††ccs: Computing methodologies Semantic networks
Figure 1.  (a) The pre-trained conversation model (PCM) for dialog
understanding, which directly pre-trains bidirectional language models on
plain dialog corpora. (b) The PCM for dialog generation, which employs a
generative PLM on plain dialog corpora. (c) We propose a unified PCM for both
dialog understanding and dialog generation, which maintains the task-flow and
is trained on both labeled and unlabeled data.

##  1\. Introduction

Task-oriented dialog (TOD) systems, which aim at helping users accomplish
specific tasks through natural language interactions, have attracted
increasing attention recently due to the broad applications such as flight
booking, weather inquiry and restaurant reservation. To fulfill user goals,
TOD systems must be capable of understanding the dialog, planning the policy,
and generating human-like responses, which maintain a task-flow during the
dialog interaction. Concretely, as illustrated in Figure 1 (c), a complete
task-flow consists of three phases: (i) extracting crucial semantics of user
query such as intents and slots, conditioned on dialog context (Dialog
Understanding); (ii) determining dialog acts towards successful task
completion (Policy Planning); (iii) generating natural and appropriate
responses (Dialog Generation).

Recent advances of pre-trained language models (PLMs) (Devlin et al., 2019;
Liu et al., 2019; Yang et al., 2019; Radford et al., 2019) have brought
significant improvement by further fine-tuning on downstream various
downstream tasks (Mehri et al., 2020; Zhang et al., 2019; Heck et al., 2020).
However, previous studies (Zhang et al., 2020b; Wu et al., 2020) reveal that
there are intrinsic differences in linguistic patterns between human
conversations and plain texts, which leads to deficiencies of TOD systems.
Subsequently, many methods attempt to address this issue by building pre-
trained conversation models (PCMs) (Mehri et al., 2020; Zhang et al., 2020b;
Wu et al., 2020), which further pre-train vanilla PLMs on large-scale dialog
corpora to capture conversational knowledge. For dialog understanding tasks
(e.g., intent recognition and dialog state tracking), ConvBERT (Mehri et al.,
2020) continually pre-trained a BERT model on large-scale human-human
conversational logs with the masked language modeling loss. TOD-BERT (Wu et
al., 2020) further integrated a contrastive objective with a better response
selection mechanism. For dialog generation tasks, DialoGPT (Zhang et al.,
2020b) was the first to continually pre-train a GPT-2 on 147M conversations
from Reddit to generate more relevant and consistent responses than strong RNN
baselines. PLATO (Bao et al., 2020) leveraged both Twitter and Reddit data to
pre-train a dialog generation model with discrete latent variables, which
implicitly modeled dialog policy and alleviated the one-to-many mapping
problem in open-domain dialog generation.

Despite the remarkable progress of previous PCMs on dialog understanding or
dialog generation, there are still several technical challenges to
constructing an effective and unified pre-trained conversation model. First,
most current PCMs are tailored for one specific task like dialog understanding
or dialog generation, as illustrated in Figure 1 (a)-(b). Limited exploration
has been attempted to solve the three sub-tasks jointly in a unified
framework. For example, although TOD-BERT significantly improves the
performance of a wide range of dialog understanding tasks (Wu et al., 2020),
it is difficult to apply TOD-BERT for dialog generation due to its
bidirectional nature (Wang and Cho, 2019). Second, a common idea of existing
PCMs is to directly train existing PLMs on dialog corpora with vanilla
language model objectives while neglecting the task-flow characteristics in
TOD. For example, PPTOD (Su et al., 2021) proposes to pre-train a dialog model
by primitively amalgamating different dialog tasks with heterogeneous
annotations into a text-to-text format like T5, which decreases the
performance on individual TOD tasks. Different from open-domain chatting bots,
TOD systems aim to help users accomplish certain tasks with a controllable
step-by-step procedure, therefore it is essential to explicitly incorporate
the task-flow into pre-training for fully exploiting task-oriented dialog
features. Third, in previous PCMs, leveraging semantic structures and manual
annotations (e.g., intents or slots) of dialogs to learn better pre-trained
dialog representations still remains unexplored. Nevertheless, there have been
many works demonstrating that labeled data can not only accelerate the pre-
training procedure but also improve the model performance (Assran et al.,
2020; Khosla et al., 2020; Dai et al., 2021a). Therefore, we argue that it is
also crucial to incorporate dialog annotations and semantic structures to
learn powerful PCMs.

In this paper, all the aforementioned challenges are considered in a holistic
framework. We propose a novel end-to-end semi-supervised pre-trained
conversation model (called SPACE-3), which can be effectively applied to
various downstream dialog tasks. SPACE-3 is pre-trained on both labeled and
unlabeled dialog data, aiming to inject the knowledge of dialog annotations
into task-oriented dialog pre-training on large-scale task-oriented dialog
corpora. To be specific, SPACE-3 consists of four successive core components
to model a complete task-flow in a unified transformer architecture, including
(i) a dialog encoding module to encode dialog history and capture common
context representations, (ii) a dialog understanding module to extract
semantic vectors from either user queries or system responses, (iii) a dialog
policy module to learn a policy vector that contains high-level semantics of
the response, and (iv) a dialog generation module to produce appropriate
responses. Except the encoding module is bidirectional, all the rest three
modules are set as unidirectional to preserve the causal relations within the
task-flow. We also design a dedicated pre-training objective for each
component. Concretely, we pre-train the dialog encoding module with span mask
language modeling to learn the contextualized dialog information. To capture
semantic dialog structure knowledge, we pre-train the dialog understanding
module via a novel tree-induced semi-supervised contrastive learning objective
with the help of extra dialog annotations. We treat structured semantic
annotations stored in labeled dialog corpora as trees, and estimate the
similarity of different dialogs by Tree Editing Distance. According to the
soft similarity scores, supervised contrastive learning is applied to increase
the similarity of semantically similar dialogs and push away the
representations of all other dialogs. We also adopt vanilla self-supervised
contrastive learning to refine the learned representation with the help of
unlabeled dialogs. In addition, the dialog policy module is pre-trained to
determine the semantics of the system response by minimizing the
$\mathcal{L}_{2}$ distance between its output policy vector and the semantic
vector of the response extracted by the dialog understanding module. Finally,
the dialog generation model is pre-trained by language modeling. We aggregate
all pre-training objectives above to learn dialog understanding, policy
planning, and dialog generation abilities from both limited labeled dialog
datasets and large-scale unlabeled dialog corpora. It is noteworthy that
SPACE-3 can avoid error accumulation due to the end-to-end pre-training.

We summarize our main contributions as follows:

  * •

To the best of our knowledge, we are the first to propose a unified PCM for
dialog understanding, policy planning, and dialog generation, which fully
exploits task-oriented dialog characteristics and maintains the task-flow by
connecting the successive dialog-related tasks.

  * •

We explore tree-induced semi-supervised contrastive pre-training for dialog
understanding. The injection of structured semantic annotations via
contrastive learning enhances dialog representation learning on large-scale
dialog corpora.

  * •

For pre-training SPACE-3, we construct a labeled corpus AnPreDial with 3
million utterances and a large-scale unlabeled dialog corpus UnPreDial with 19
million utterances. Experiments show that our pre-trained model achieves new
state-of-the-art performance on eight downstream dialog benchmarks, including
intent prediction, dialog state tracking, and end-to-end dialog modeling. We
believe that the release of the constructed dialog corpora and the unified PCM
would push forward the research in this area.

##  2\. Related Work

###  2.1. Pre-trained Language Models

Pre-trained language models (PLMs) can be roughly divided these PLMs into two
categories (uni-directional and bi-directional) according to the attention
mechanisms applied in the pre-training. For natural language understanding
tasks like classification (Wang et al., 2018) or machine reading comprehesion
(Rajpurkar et al., 2016), BERT (Devlin et al., 2019) and RoBERTa (Liu et al.,
2019) were pre-trained with a bi-directional transformer, aiming to grasp deep
contextualized semantic meanings. For natural language generation tasks, GPT
(Radford et al., 2018, 2019) and T5 (Raffel et al., 2020) employed the uni-
directional transformer encoder to maximize left-to-right generation
likelihood. Recently, the unified language model UniLM (Dong et al., 2019)
enabled both uni-directional and bi-directional attention with flexible self-
attention mask designs. However, previous works (Zhang et al., 2020b; Kulhánek
et al., 2021) reveal that there are intrinsic differences between plain texts
and human conversations. Directly fine-tuning PLMs, which are pre-trained on
plain texts, on downstream dialog tasks prevents the model from sufficiently
capturing conversational patterns and linguistic knowledge, which leads to
sub-optimal results (Mehri et al., 2019a; Zeng and Nie, 2021; Wu and Xiong,
2020).

###  2.2. Pre-trained Conversation Models

To bridge the gap between plain texts and human conversations, many studies
attempted to build pre-trained conversation models (PCMs) by further pre-
training PLMs on dialog corpora (Mehri et al., 2020; Zhang et al., 2020b;
Henderson et al., 2019), which achieved impressive performance on both dialog
understanding (Wu et al., 2020) and generation tasks (Peng et al., 2020). A
majority of PCMs are trained on open-domain conversational data from Reddit or
Twitter for dialog response generation. For example, DialoGPT (Zhang et al.,
2020b) extended and pre-trained the GPT-2 on 147M conversations extracted from
Reddit. Blender (Roller et al., 2020) was pre-trained on 1.5B open-domain
dialogs, demonstrating powerful dialog generation performance. Furthermore,
PLATO (Bao et al., 2020) used discrete latent variables to tackle the one-to-
many mapping problem in open-domain dialog generation.

Different from open-domain dialog systems, task-oriented dialog (TOD) systems
have explicit goals (e.g. flight booking or restaurant reservation), making
dialog understanding and policy planning components important (Wu et al.,
2020; He et al., 2020a, 2021). The PCMs for TOD systems can be roughly divided
into two types. The first type is designing PCMs that are tailored for dialog
understanding tasks. For example, ConvBERT (Mehri et al., 2020) utilized a
masked language modeling objective to fine-tune BERT on a large dialog corpus
so that semantically rich representations can be learned. TOD-BERT (Wu et al.,
2020) employed a contrastive objective for response selection and incorporated
user/system tokens into the masked language modeling. DialogueBERT (Zhang et
al., 2021b) explored the masked utterance modeling and response contrastive
loss, showing good performance on intent, emotion, and entity recognition. The
second line is to design PCMs that are tailored for dialog generation tasks.
SC-GPT (Peng et al., 2020) focused on the natural language generation (NLG)
task, assuming that dialog acts and slot-tagging results are given to generate
a natural language response. SOLOIST (Peng et al., 2022) parameterized a task
bot using a Transformer-based auto-regressive language model, which subsumed
different dialog modules into a single neural model and was pre-trained on two
TOD datasets. SPACE (He et al., 2022) proposed to use consistency
regularization loss to learn dialog policy from labeled and unlabeled dialog
corpora via a semi-supervised manner. To exploit more heterogeneous TOD
corpora, PPTOD (Su et al., 2021) converted different TOD tasks into the text-
to-text generation task with task-specific prompts on T5 model (Raffel et al.,
2020). Different from these methods, we propose SPACE-3 that explicitly models
task-flow during pre-training while avoiding the error accumulation in the
pipeline TOD architecture via multi-task training.

###  2.3. Contrastive Learning

Contrastive learning (CL) is a popular pre-training method, which takes
advantage of self-supervised objectives on large-scale plain texts to learn
semantic representations. The key idea of contrastive learning is to narrow
the distance between two semantically similar sentence representations and
push away the representations of dissimilar sentences (Gao et al., 2021; Wu et
al., 2021; Yan et al., 2021). Recently, contrastive learning has also
attracted increasing attention for dialog understanding tasks. For example,
(Zhang et al., 2021a) performed self-supervised pre-training by employing
contrastive learning objectives and fine-tuned the pre-trained models on few-
shot intent recognition tasks. (He et al., 2020b) used adversarial samples as
difficult negative instances in contrastive learning for zero-shot cross-
domain slot-filling. The most related works to ours are (Mehri and Eric, 2021;
Vulić et al., 2021), which utilized supervised contrastive learning at the
fine-tuning stages for intent recognition tasks, where the samples from the
same class are all regarded as positive ones. Different from (Mehri and Eric,
2021; Vulić et al., 2021), we explore semantic structures in TOD, and propose
a new contrastive learning approach that calculates semantic tree-structure
similarity among all possible labeled data, which is essential for the
semantic modeling in the task-flow.

Subsequently, semi-supervised contrastive learning has been proven as a
promising pre-training approach in various research fields, such as image
recognition (Yuan et al., 2021; Park et al., 2021; Li et al., 2021; Kim et
al., 2021a), image segmentation (Alonso et al., 2021; Zhou et al., 2021) and
speech recognition (Xiao et al., 2021; Inoue and Goto, 2020). It adopts
supervised and self-supervised contrastive objectives together on the
combination of labeled and unlabeled data. However, semi-supervised
contrastive learning remains unexplored in task-oriented dialog understanding.
In this paper, we utilize semantic annotations in TOD and propose a new tree-
induced semi-supervised contrastive learning paradigm to improve dialog
understanding.

Table 1. Statistics of our pre-trained dialog dataset.

Statistics | AnPreDial | UnPreDial
---|---|---
# Datasets | 32 | 21
# Dialogs | 459,465 | 3,217,058
# Turns | 3,366,479 | 19,578,028
Avg. tokens per turn | 13.9 | 14.5
Avg. tokens per dialog | 101.8 | 88.2
Total unique tokens | 46.8M | 283.7M

Table 2. Data Composition of AnPreDial and UnPreDial.

AnPreDial |  | BANKING77 (Casanueva et al., 2020), CLINC150 (Larson et al., 2019), HWU64 (Liu et al., 2021), REST8K (Coope et al., 2020), TOP (Gupta et al., 2018), ATIS (Hemphill et al., 1990), SNIPS (Coucke et al., 2018), CrossNER (Liu et al., 2020), FB_TOD_SF (Schuster et al., 2019),
---
MIT-restaurant (Liu et al., 2013), MIT-movies-eng (Liu et al., 2013), MIT-
movies-trival10k13 (Liu et al., 2013), MultiWOZ_coco (Li et al., 2020),
MultiWOZ (Eric et al., 2020), STAR (Mosig et al., 2020), DailyDialog (Li et
al., 2017),
SGD (Rastogi et al., 2020), Frames (El Asri et al., 2017), MSRe2e (Li et al.,
2018), DSTC2 (Williams et al., 2016), DSTC3 (Williams et al., 2016), SimJoint
(Shah et al., 2018), MulDoGo (Peskov et al., 2019), WOZ (Mrkšić et al., 2017),
TaskMaster1 (Byrne et al., 2019), TaskMaster2 (Byrne et al., 2019),
TaskMaster3 (Byrne et al., 2019), InCar (Eric et al., 2017),
MultiWOZ_synthesis (Campagna et al., 2020), SwDA (Stolcke et al., 2000), BiTOD
(Lin et al., 2021), PersuaGOOD (Wang et al., 2019)
UnPreDial |  | MulDoGo_un (Peskov et al., 2019), ABCD (Chen et al., 2021), AirDialog (Wei et al., 2018), CCPE (Radlinski et al., 2019), MetalWOZ (Shalyminov et al., 2020), CMU_DoG (Zhou et al., 2018), ConvQuestions (Kacupaj et al., 2021), CoQA (Reddy et al., 2019), CoSQL (Yu et al., 2019),
---
doc2dial (Feng et al., 2020), DSTC10-track2 (Kim et al., 2021b), DSTC10-track3
(Kottur et al., 2021), MedicalDialog (Zeng et al., 2020), Self-Dialog
(Fainberg et al., 2018), WOW (Dinan et al., 2018), TopicChat (Gopalakrishnan
et al., 2019), Persona-Chat (Zhang et al., 2018),
MMD (Saha et al., 2017), CSQA (Saha et al., 2018), AmazonQA (Gupta et al.,
2019), ChitChat (Will et al., 2020)

##  3\. Pre-training Data Construction

Previous TOD datasets are usually small and scattered since obtaining and
labeling such data is time-consuming and labor-intensive. In this paper, we
collect new dialog datasets used for pre-training our SPACE-3 model, including
a labeled dialog corpus (AnPreDial) and a large-scale unlabeled dialog corpus
(UnPreDial). The total statistics of our collected dialog corpora are shown in
Table 1. More details about data composition can be found in Table 2.

###  3.1. Labeled Dataset: AnPreDial

As revealed by previous works, labeled data can not only accelerate the pre-
training procedure but also improve the model performance (Assran et al.,
2020; Khosla et al., 2020; Dai et al., 2021a). Motivated by this observation,
we exploit extra dialog annotations to learn better pre-trained dialog
representations. Specifically, task-oriented dialogs often involve several
user goals, and the TOD systems need to help fulfill these goals through
interaction with users. To this end, the user queries and system responses are
usually composed of rich semantic annotations, including DOMAIN, INTENT, SLOT
and VALUE labels. For instance, given a user utterance “I want an indian
restaurant in park, could you offer me the name?”, the semantic annotations
are “restaurant-inform(food=indian, area=park); restaurant-request(name=?)”.
As shown in Figure 2, such semantic annotations are naturally structured,
which could be expanded into tree structures called semantic trees. A semantic
tree typically contains four layers: DOMAIN layer, INTENT layer, SLOT layer,
and VALUE layer. Each layer regards respective annotation elements as nodes.
If no matched annotations are provided, the nodes of the corresponding layer
are set as empty (denoted as NULL). The first layer consists of DOMAIN nodes
as successors of the ROOT node, preceded by INTENT nodes. As children of
INTENT nodes, SLOT nodes occupy the third layer of the semantic tree. As leaf
nodes, VALUE nodes take SLOT nodes as parents.

To provide sufficient high-quality dialog corpora with structured semantic
annotations, we carefully examine currently available sources and aggregate
them together into a large dataset called AnPreDial, which contains 32
existing labeled TOD datasets with 3 million turns, ranging from single-turn
QA to multi-turn dialogs.

Figure 2. An example of user query with its semantic annotations in the
labeled dataset AnPreDial.

###  3.2. Unlabeled Dataset: UnPreDial

We aggregate 21 online dialog corpora and build a large-scale unlabeled dialog
corpus (UnPreDial) with 19 million utterances, ranging from online forums to
conversational machine reading comprehension. Due to the massive noises in
UnPreDial, we perform careful data cleaning, including: (1) removing the
dialogs when there is a URL in the utterances; (2) removing the dialogs that
contain at least three repetitive words; (3) removing non-English sentences;
(4) removing the sentences that contain special markers such as “[” or “]”,
since these sentences could be markup; (5) removing the dialogs that contain
offensive language; (6) replacing the non-unicode characters such as emojis.

Figure 3.  The architecture of proposed SPACE-3 model, which successively
includes a dialog encoder, a understanding decoder, a policy decoder and a
response decoder. In each module, the lines with different colors denote the
self-attention positions where current token can attend. The encoder is bi-
directional, while three decoders are uni-directional. As extra inputs of the
understanding and policy decoders, two kinds of prompts are leveraged to
extract semantics and help pass the task-flow in a TOD system. Finally, five
pre-training objectives are carried out to optimize SPACE-3 jointly in a
multi-task paradigm.  Figure 4.  The procedure of policy semantic modeling,
minimizing the $\mathcal{L}_{2}$ distance between the predicted policy vector
$h^{o}$ and the extracted response semantic vector $h^{r}$.

##  4\. Method

In this section, we first introduce the model architecture. Then we elaborate
on each objective used in our pre-training.

###  4.1. Model Architecture

We choose UniLM (Dong et al., 2019) as our backbone model for task-oriented
dialog modeling. UniLM contains a bi-directional Transformer encoder and a
uni-directional Transformer decoder with shared parameters, which is naturally
suitable for both dialog understanding and dialog generation. As illustrated
in Figure 3, our proposed SPACE-3 architecture consists of four successive
components to maintain the task-flow in TOD systems, including a dialog
context encoder, a dialog understanding decoder, a dialog policy decoder and a
dialog generation decoder. Similar to UniLM, we share parameters of all
modules and adopt different self-attention mechanisms to distinguish the
encoder and decoders.

Let $D=\\{d_{i}=<u_{i},s_{i}>\\}^{K}_{i=1}$ denotes a conversation with $K$
dialog turns. $u_{i}$ and $s_{i}$ denote the user utterances and system
utterances, respectively. At current turn $k$, we define the dialog context
$c$ as the concatenation of utterances $\\{d_{j}\\}^{k-1}_{j=1}$ and user
query $u_{k}$. To simplify notations, we use $q=\\{q_{1},q_{2},...,q_{M}\\}$
and $r=\\{r_{1},r_{2},...,r_{N}\\}$ to represent the user query $u_{k}$ and
system response $s_{k}$, where $q_{m}$ denotes the $m$-th token in $u_{k}$ and
$r_{n}$ denotes the $n$-th token in $s_{k}$. $M$ and $N$ denote corresponding
utterance lengths. Next, we describe the four modules and the input
representations in detail.

####  4.1.1. Input Representation

Following previous work (Bao et al., 2020), for each input token, its vector
representation is computed by summing the corresponding token embedding, the
role embedding, the turn embedding and the position embedding. The role
embeddings are used to differentiate the characters evolved in the current
utterance (i.e., $E_{U}$ for user role while $E_{S}$ for system role). The
turn embeddings are assigned to each token according to its relative turn
order in multi-turn utterances. The position embeddings are assigned to each
token according to its relative position within the corresponding utterance.
In addition, we add special tokens [BOU], [EOU] to bound the user utterance,
and [BOS], [EOS] to bound the system utterance. It is noteworthy that the
role, turn and position embeddings are set to empty for the prompt tokens.

####  4.1.2. Dialog Context Encoder

To overcome the challenge of modeling long dialog history in multi-turn
conversations, the dialog context encoder employs a bi-directional Transformer
to encode the context utterances $c$, which can capture the common and
essential information for subsequent modules.

####  4.1.3. Dialog Understanding Decoder

We leverage a dialog understanding decoder (a uni-directional Transformer) to
extract semantic representations of current user query $q$ conditioned on the
dialog context $c$, which are used for classification tasks such as intent
recognition. Therefore, it is vital to map multiple utterances to a
semantically meaningful representation. Prior work (Clark et al., 2019)
revealed that transformer-based models tend to attribute a significant amount
of attention to the [CLS] token, resulting in diluted representations. Instead
of using the latent representation of the [CLS] token, (Mehri and Eric, 2021)
proposed to have observer tokens that attend to the input words but are not
being attended to by the model. Inspired by the observers (Mehri and Eric,
2021) and prompt tuning (Lester et al., 2021), we take the concatenation of a
sequence of soft prompt tokens and the dialog context $c$ as input of the
dialog understanding decoder to extract the rich semantic representations of
dialog context. To be specific, we use
$p^{u}=\\{p^{u}_{1},p^{u}_{2},...,p^{u}_{A}\\}$ to represent the dialog
understanding prompt sequence, where $A$ is the length of the prompt.
Different from (Mehri and Eric, 2021) that directly average observers’ final
representations from the last transformer layer, we select the hidden state of
the last prompt token $p^{u}_{A}$ as the sentence representation $h^{q}$ of
the user query $q$ due to the uni-directional sequential property. As shown in
Figure 4, conditioned on the concatenation of $c$, $r$ and $p^{u}$, we can
also obtain the hidden state of the prompt token $p^{u}_{A}$ as the semantic
representation $h^{r}$ of the response $r$ for better understanding the system
behaviors.

####  4.1.4. Dialog Policy Decoder

The dialog policy decoder (a uni-directional Transformer) is devised to
determine high-level behaviors of the system, which leads to successful task
completion in TOD systems. Similar to the dialog understanding prompts, we
also adopt a policy prompt sequence, which is denoted as
$p^{o}=\\{p^{o}_{1},p^{o}_{2},...,p^{o}_{B}\\}$ where $B$ denotes the length
of the policy prompt sequence. Taking $(c,p^{u},p^{o})$ as input, the dialog
policy decoder obtains the hidden state of the prompt token $p^{o}_{B}$ as the
output policy vector $h^{o}$.

####  4.1.5. Dialog Generation Decoder

Given the dialog context $c$, the understanding prompt sequence $p^{u}$ and
policy prompt sequence $p^{o}$, we apply the dialog generation decoder to
generate an appropriate response $r$ in an auto-regressive manner, returning
the reply to the user at the current dialog turn.

###  4.2. Pre-training Objectives

We employ five objectives to pre-train SPACE-3 in a multi-task learning
paradigm, including span masked language modeling, semi-supervised
understanding semantic modeling, semantic region modeling, policy semantic
modeling and response generation modeling.

####  4.2.1. Span Masked Language Modeling

Previous work (Devlin et al., 2019) show that masked language modeling (MLM)
is an effective pre-training strategy for distributed representation learning,
in which a random sample of tokens in the input sequence is replaced with the
special token [MASK] and the model needs to predict the masked tokens. Span
Masking (Joshi et al., 2020) improves the original MLM by masking random
contiguous spans rather than random individual tokens. In this paper, we
employ the span masking language modeling (SLM) to pre-train the dialog
context encoder on both the labeled dataset AnPreDial and the unlabeled
dataset UnPreDial. Specifically, for labeled data annotated with slot-value
pairs, we mask all value spans in $c$ to construct the perturbed input
sequence; for unlabeled data, we randomly mask contiguous text spans instead.
Then the cross-entropy loss is employed to minimize the negative log-
likelihood of the masked tokens, which is defined as:

(1) $\mathcal{L}_{\mathrm{slm}}=-\sum_{\hat{c}\in m\left(c\right)}\log
p\left(\hat{c}\mid c_{\backslash m\left(c\right)}\right)$

where $m(c)$ denotes the masked tokens from $c$ and ${c}_{\backslash
m(\mathbf{c})}$ indicates the rest tokens except $m(c)$.

####  4.2.2. Semi-supervised Understanding Semantic Modeling

To enhance dialog representation learning with structured semantic
annotations, we perform semi-supervised pre-training on both the labeled data
and unlabeled data. Specifically, we adopt a novel tree-induced supervised
contrastive objective on the labeled dataset AnPreDial, while a vanilla self-
supervised contrastive objective is applied on the unlabeled dataset
UnPreDial. For the labeled data, we treat all data in the same batch as
positive samples, which are assigned with different similarity scores. As
depicted in Section 3.1, we reorganize the structured semantic annotations
stored in labeled dialog corpora as semantic trees. According to the semantic
trees’ similarity, we estimate the similarity of corresponding dialog samples.
For each pair of semantic trees, we leverage the Tree Editing Distance to
calculate their similarity score, which is considered to be the weighted
number of edit operations (insert, delete, and modify) to transform one tree
to another. Different from (Zhang and Shasha, 1989) that computes the distance
between two ordered trees, we do not differentiate the order of the children
of a node. For example in Figure 2, as children of the inform node, the left-
to-right arrangement of ¡food, area¿ is equivalent to the arrangement of
¡area, food¿. Therefore, we reorder all children nodes recursively according
to the alphabetical order of their annotation strings. Given a pair of
reordered semantic trees (denoted as $T_{i},T_{j}$), we calculate the
similarity coefficient $f_{i,j}$ by:

(2) $\displaystyle
f_{i,j}=\frac{\max\left\\{|T_{i}|,|T_{j}|\right\\}-d_{i,j}}{\max\left\\{|T_{i}|,|T_{j}|\right\\}}$
(3) $\displaystyle
d_{i,j}=\mathrm{TreeEditingDistance}\left(T_{i},T_{j}\right)$

where $\mathrm{TreeEditingDistance}$ is the function to calculate the tree
editing distance and $|\cdot|$ denotes the number of nodes in the semantic
tree. The similarity coefficient $f_{i,j}\in\left[0,1\right]$ as the distance
$d_{i,j}\in\left[0,\max\left\\{|T_{i}|,|T_{j}|\right\\}\right]$.

As mentioned in Section 4.1.3, we use the hidden states of the last tokens in
the respective prompt sequences to represent the sentence representations
$h^{q},h^{r}\in\mathbb{R}^{768}$ of user query and system response,
respectively. Thus for each pair of dialog samples $(i,j)$, their output
sentence representations are $h^{q}_{i},h^{r}_{i}$ for the sample $i$ and
$h^{q}_{j},h^{r}_{j}$ for the sample $j$. Here, we use the superscript q and r
to represent the variables related to the user query and the system response,
respectively. Hence, we use $f^{q}_{i,j}$ and $f^{r}_{i,j}$ to denote the
similarity scores between semantic trees of user queries and system responses,
respectively. Note that we use the same way to compute the semi-supervised
pre-training objective for user queries or system responses. For simplicity,
we only introduce the pre-training objective on the user side as an example.
Suppose the size of the current batch is $L$. We follow the common practice in
(Gao et al., 2021) to duplicate the training data via dropout-based data
augmentation. In this way, we acquire a new $2L$-length batch. Let
$I=\\{1\dots 2L\\}$ be the index set of the new batch, and we adopt the tree-
induced supervised contrastive objective for labeled data as follows:

(4) $\small\mathcal{L}^{q}_{\mathrm{sup}}=-\sum_{i\in I}\sum_{j\in
I}\frac{f^{q}_{i,j}}{\sum_{v\in
I}f^{q}_{i,v}}\log\frac{\exp\left(\sigma\left({h^{q}_{i}}\right)\cdot\sigma\left({h^{q}_{j}}\right)/\tau\right)}{\sum_{l\in
I,l\neq
i}\exp\left(\sigma\left({h^{q}_{i}}\right)\cdot\sigma\left({h^{q}_{l}}\right)/\tau\right)}$

where $\sigma$ is a transformation function which is defined by:

(5) $\sigma\left(h^{q}\right)=\mathrm{Norm}\left(W_{1}h^{q}+b_{1}\right)$

where $W_{1}\in\mathbb{R}^{768\times 768}$ and $b_{1}\in\mathbb{R}^{768\times
1}$ are learnable parameters. $\mathrm{Norm(\cdot)}$ is the normalization
operation to constrain $||\mathrm{Norm(\cdot)||}=1$ and $\tau\in[0,1]$ is a
temperature hyper-parameter. Note that our method is quite different from the
common supervised contrastive learning (Vulić et al., 2021) in which only
augmented data or data from the exact same label are considered as positive
samples, and all other data are considered as negative samples. Instead, we
regarded all in-batch data as positive samples, assigned with different
similarity scores ranging from $0$ to $1$.

For unlabeled data, we do not use semantic trees due to the lack of available
semantic annotations. Instead, we adopt a self-supervised contrastive
objective, where only the augmented data by dropout is treated as a positive
sample (Gao et al., 2021). The loss function is defined as:

(6) $\small\mathcal{L}^{q}_{\mathrm{self}}=-\sum_{i\in I}\sum_{j\in
I}\textbf{1}\left(j=i^{+}\right)\log\frac{\exp\left(\sigma\left({h^{q}_{i}}\right)\cdot\sigma\left({h^{q}_{j}}\right)/\tau\right)}{\sum_{l\in
I,l\neq
i}\exp\left(\sigma\left({h^{q}_{i}}\right)\cdot\sigma\left({h^{q}_{l}}\right)/\tau\right)}$

where $\textbf{1}(j=i^{+})$ indicates that it takes 1 if and only if the data
sample $j$ is the augmentation ($i^{+}$) of the data sample $i$; otherwise 0
is obtained.

Overall, the tree-induced semi-supervised contrastive learning objective for
the understanding of the user query is:

(7)
$\mathcal{L}^{q}_{\mathrm{scl}}=\mathcal{L}^{q}_{\mathrm{sup}}+\mathcal{L}^{q}_{\mathrm{self}}$

Combined with the objective $\mathcal{L}^{r}_{\mathrm{scl}}$ on the system
side which is calculated in the same way as $\mathcal{L}^{q}_{\mathrm{scl}}$,
our total loss for the dialog understanding modeling is computed as:

(8)
$\mathcal{L}_{\mathrm{scl}}=\mathcal{L}^{q}_{\mathrm{scl}}+\mathcal{L}^{r}_{\mathrm{scl}}$

####  4.2.3. Semantic Region Modeling

We additionally design a bag-of-words loss to eliminate the influence of
uninformative utterances in the long dialog context $c$ using the prompts
$p^{u}$. This objective is to optimize the log-likelihood of $q_{\rm bow}$
which is the variant of $q$ without considering word order:

(9) $\displaystyle\mathcal{L}^{q}_{\mathrm{bow}}=-\log p\left(q_{\rm bow}\mid
c,p^{u}\right)=-\log\prod_{m=1}^{M}H^{{}^{\prime}}_{\textit{index}\left(q_{m}\right)}$
(10) $\displaystyle
H^{{}^{\prime}}=\operatorname{softmax}\left(W_{2}h^{q}+b_{2}\right)\in\mathbb{R}^{|V|}$

where $|V|$ refers to the vocabulary size and
$\textit{index}\left(q_{m}\right)$ is the word index in the vocabulary of the
$m$-th word $q_{m}$ in the utterance $q$.
$H^{{}^{\prime}}_{\textit{index}\left(q_{m}\right)}$ denotes the estimated
probability of $q_{m}$. $W_{2}$ and $b_{2}$ are learnable parameters. The bag-
of-words loss discards the order of words and predicts the words in $q$ in a
non-autoregressive way, which could force the semantic representation to
capture the global information of the given utterance.

In addition, we also compute the bag-of-words loss on the top of the
representation $h^{r}$ for the system response $r$, which is to predict the
unordered words $r_{\rm bow}$ as follows:

(11) $\displaystyle\mathcal{L}^{r}_{\mathrm{bow}}=-\log p\left(r_{\rm bow}\mid
c,r,p^{u}\right)=-\log\prod_{n=1}^{N}H^{{}^{\prime\prime}}_{\textit{index}\left(r_{n}\right)}$
(12) $\displaystyle
H^{{}^{\prime\prime}}=\operatorname{softmax}\left(W_{2}h^{r}+b_{2}\right)\in\mathbb{R}^{|V|}$

Totally, the loss for semantic region modeling is defined as:

(13)
$\mathcal{L}_{\mathrm{bow}}=\mathcal{L}^{q}_{\mathrm{bow}}+\mathcal{L}^{r}_{\mathrm{bow}}$

####  4.2.4. Policy Semantic Modeling

The dialog policy module determines high-level dialog acts of TOD systems
towards successful task completion. SPACE-3 predicts the policy vector $h^{o}$
to determine the high-level semantics of the response $r$ conditioned on the
context $c$, the dialog understanding prompts $p^{u}$, and the policy prompts
$p^{o}$. It is intuitive that the meanings of the learned policy vector should
be close to the meanings of semantic vectors inferred from the response as a
posterior. To this end, we first extract the semantic response vector $h^{r}$
based on the sequence ($c$, $r$, $p^{u}$) and the policy vector $h^{o}$ based
on the the sequence ($c$, $p^{u}$, $p^{o}$). Then we decrease their
discrepancy by minimizing the $L_{2}$ distance. As illustrated in Figure 4, we
define the policy semantic modeling objective $\mathcal{L}_{\rm psm}$ as
follows:

(14) $\mathcal{L}_{\mathrm{psm}}=||h^{o}-h^{r}||^{2}_{2}$

####  4.2.5. Response Generation Modeling

Based on the dialog context $c$, understanding prompts $p^{u}$ and policy
prompts $p^{o}$, the response generation task is carried out to produce the
appropriate system response $r$ in an auto-regressive manner. We adopt the
standard negative log-likelihood loss $\mathcal{L}_{\mathrm{rgm}}$ for the
generation task:

(15) $\mathcal{L}_{\mathrm{rgm}}=-\sum_{n=1}^{N}\log p\left(r_{n}\mid
c,p^{u},p^{o},r_{<n}\right)$

where $r_{n}$ is the $n$-th word in $r$ and $r_{<n}=\\{r_{1},...,r_{n-1}\\}$
represents the words of previous steps.

####  4.2.6. Joint Training

Our unified pre-training strategy allows the TOD systems to learn the dialog
understanding, policy planning, and dialog generation abilities from large-
scale dialog corpora via multi-task learning. To sum up, the overall training
objective $\mathcal{L}_{\mathrm{joint}}$ of SPACE-3 can be defined as follows:

(16) $\mathcal{L}_{\rm
joint}=\mathcal{L}_{\mathrm{slm}}+\mathcal{L}_{\mathrm{scl}}+\mathcal{L}_{\mathrm{bow}}+\mathcal{L}_{\mathrm{psm}}+\mathcal{L}_{\mathrm{rgm}}$

###  4.3. Fine-tuning

In the fine-tuning stage, we concentrate on task-oriented dialog tasks,
including dialog understanding, policy optimization, and dialog generation.
For dialog understanding tasks, we only leverage the dialog encoding module
and the dialog understanding module to fine-tune our pre-trained model,
discarding the dialog policy and generation modules. Thus only the
understanding prompt sequence $p^{u}$ is employed to extract sentence
representations, which are used for classification tasks such as intent
recognition or dialog state tracking. For policy optimization and dialog
generation tasks, we retain the integrated model architecture to produce
appropriate responses, where two kinds of prompts $p^{u}$ and $p^{o}$ are
adopted to characterize the dialog understanding and dialog policy for better
dialog generation.

##  5\. Experimental Setup

In this section, we will introduce experimental data, compared baselines and
the implementation details.

###  5.1. Experimental Data

To comprehensively evaluate our SPACE-3, we carefully choose eight downstream
tasks in terms of dialog understanding, policy optimization and response
generation. For the understanding modeling, we have three intent prediction
datasets and one dialog state tracking dataset. For policy and generation
modeling, we follow the setup in previous work (Lubis et al., 2020) and
evaluate the proposed model on four well-studied end-to-end dialog modeling
datasets.

##### Intent Prediction (IP)

The goal of IP is to predict which intention the user expresses in the current
utterance, which is a single-turn classification problem. Recently, three
large benchmarks have been used for evaluating IP models, including (i)
BANKING77 (Casanueva et al., 2020), a single turn dataset in the banking
domain with 77 intent labels and 13,083 utterances; (ii) CLINIC150 (Larson et
al., 2019), a multi-domain dataset that contains 23,700 utterances, 150
different intent classes and 10 domains; (iii) HWU64 (Liu et al., 2021)
includes 25,716 utterances for 64 intents spanning 21 domains. To conduct the
downstream task, a linear layer neural network is added to the pooled
representation output $z$ for intent prediction. Following previous work
(Mehri et al., 2020), we set the evaluation metric as turn accuracy (ACC).

##### Dialog State Tracking (DST)

DST is a multi-turn slot-filling task, where the model needs to maintain a
dialog state in terms of slot-value pairs throughout the duration of a dialog.
MultiWOZ (Budzianowski et al., 2018) is one of the most challenging datasets
for DST, which contains 10k dialogs spanning 7 distinct domains. In particular
we choose the MultiWOZ2.2 version (Zang et al., 2020) as our benchmark since
it has fixed a large proportion of dialog state annotation errors. Following
(Heck et al., 2020; Mehri et al., 2020; Dai et al., 2021b), we adopt the joint
goal accuracy (JGA) as our evaluation metric, counting the ratio of turns that
model gives correct predictions for all slots. For a fair comparison, all
compared pre-trained conversation models are used to replace the original BERT
part in the TripPy model (Heck et al., 2020) for downstream fine-tuning in our
experiments.

##### End-to-End Dialog Modeling (E2E)

This is a standard task to evaluate the capability of both policy optimization
and dialog generation. To be specific, we evaluate the proposed model on four
popular benchmark task-oriented dialog datasets: the Cambridge Restaurant
(CamRest676) (Wen et al., 2017), Stanford In-Car Assistant (In-Car) (Eric et
al., 2017), MultiWOZ2.0 and MultiWOZ2.1. CamRest676 contains single-domain
dialogs belonging to the restaurant domain. In-Car consists of dialogs between
a user and a in-car assistant system covering several tasks, including
calendar scheduling, weather information retrieval and point-of-interest
navigation. MultiWOZ2.1 is a newer version of MultiWOZ2.0 where the wrong
dialog actions have been rectified. Both datasets are widely used for end-to-
end dialog evaluation. We use BLEU (Papineni et al., 2002) to measure the
response generation quality. For MultiWOZ, we report the metrics of whether
the system provides an correct entity (Inform) and answers all the requested
information (Success). A combined score (Comb) (Mehri et al., 2019b) is
computed by (Inform+Success)$\times$0.5+BLEU as an overall quality measure.
Similarly, for In-Car and CamRest676, we use Match and SuccF1 following (Lei
et al., 2018), and calculate a combined score (Comb) via
(Match+SuccF1)$\times$0.5+BLEU.

Table 3. Comparison of different pre-trained conversation models on eight
dialog benchmarks under full-data and few-shot settings. $*$ denotes
continually pre-traning on our pre-trained dialog corpora. $\dagger$ denotes
results from the original paper. We calculate ACC, JGA and Comb metrics for
IP, DST and E2E tasks respectively and average them for the overall scores.

Model | IP | DST | E2E | Average
---|---|---|---|---
BANKING77 | CLINIC150 | HWU64 | MultiWOZ2.2 | CamRest | In-Car | MultiWOZ2.0 | MultiWOZ2.1
TOD-BERT | 92.76 | 96.64 | 91.91 | 54.26 | 113.28 | 103.64 | 100.88 | 101.23 | 94.33
PLATO | 92.56 | 96.00 | 89.96 | 53.89 | 114.25 | 104.07 | 102.16 | 102.98 | 94.48
PPTOD | 93.86$\dagger$ | 97.13 | 92.84 | 55.26 | 114.92 | 104.68 | 102.92$\dagger$ | 102.26$\dagger$ | 95.48
UniLM∗ | 93.12 | 96.11 | 90.15 | 54.25 | 115.03 | 104.91 | 105.35 | 105.94 | 95.61
PPTOD∗ | 93.64 | 97.29 | 92.94 | 56.01 | 115.73 | 106.01 | 107.14 | 107.22 | 97.00
SPACE-3 | 94.94 | 97.89 | 94.14 | 57.50 | 116.67 | 107.13 | 110.95 | 110.76 | 98.75
Few-shot setting (10%)
TOD-BERT | 85.19 | 94.18 | 87.36 | 47.67 | 101.89 | 92.18 | 85.17 | 86.03 | 84.96
PLATO | 84.03 | 93.33 | 86.71 | 46.98 | 102.98 | 93.14 | 90.72 | 90.31 | 86.03
PPTOD | 82.81$\dagger$ | 94.27 | 87.36 | 46.98 | 103.88 | 93.89 | 91.96$\dagger$ | 92.05 | 86.65
UniLM∗ | 85.32 | 94.00 | 87.08 | 46.88 | 103.42 | 92.97 | 90.96 | 90.20 | 86.35
PPTOD∗ | 87.63 | 94.56 | 88.57 | 48.24 | 105.01 | 94.01 | 93.80 | 94.22 | 88.26
SPACE-3 | 88.34 | 95.00 | 89.22 | 50.66 | 106.79 | 98.30 | 98.88 | 98.74 | 90.74

###  5.2. Baselines

All compared baselines can be divided into two categories: (i) the pre-trained
conversation models that are adapted for the eight downstream dialog tasks;
and (ii) the specific models that are tailored to certain tasks such as intent
prediction.

##### Baselines for Overall Comparison

We compare SPACE-3 with three strong pre-trained conversation models: 1) TOD-
BERT (Wu et al., 2020), a BERT${}_{\text{base}}$ model that has been
continually pre-trained on large scale TOD datasets with MLM and response
selection losses; 2) PLATO (Bao et al., 2020), a pre-trained dialog generation
model with discrete latent variable based on UniLM (Dong et al., 2019); 3)
PPTOD (Su et al., 2021), a T5 (Raffel et al., 2020) model that has been
continually pre-trained on eleven heterogeneous annotated TOD corpora under
the same data format.

##### Baselines for Specific Tasks

For intent prediction, we compare SPACE-3 with four state-of-the-art
baselines: ConvBERT (Mehri et al., 2020), Example+Observer (Mehri and Eric,
2021), USE+ConveRT (Casanueva et al., 2020), and ConvFit (Vulić et al., 2021).
For the dialog state tracking task, we compare with DS-DST (Zhang et al.,
2019), Seq2Seq-DU (Feng et al., 2021), PLATO-XL (Bao et al., 2021) and AG-DST
(Tian et al., 2021). For the end-to-end dialog modeling task, we have
different baselines on different datasets for comparison. For MultiWOZ2.0 and
MultiWOZ2.1, we compare with SimpleTOD (Hosseini-Asl et al., 2020), DoTS (Jeon
and Lee, 2021), UBAR (Yang et al., 2020), and MTTOD (Lee, 2021). For
CamRest676 and In-Car, we compare with SEDST (Jin et al., 2018), TSCP (Lei et
al., 2018), FSDM (Shu et al., 2019) and LABES (Zhang et al., 2020a).

###  5.3. Implementation Details

In our experiments, the number of transformer blocks in SPACE-3 is 12 and the
hidden embedding dimension is 768. In the pre-training stage, SPACE-3 is
initialized with UniLM. The maximum sequence length of dialog context and
response is set to 256 and 50. The sequence length of understanding prompts
and policy prompts is set to 5. The batch size is set to 128 and an AdamW
optimizer is employed for optimization with an initial learning rate of 1e-5.
The dropout rate is set to 0.2. In the fine-tuning stage, the grid search
algorithm is applied on the validation set to tune hyper-parameters
automatically. For all downstream tasks, we do experiments with 3 random seeds
and take the average as final results.

##  6\. Experimental Results

###  6.1. Overall Comparison with PCMs

We first compare our SPACE-3 with three pre-trained conversation models
(PCMs): TOD-BERT, PLATO and PPTOD. Since TOD-BERT is not designed for dialog
generation, we adapt it into the same UniLM architecture with its own
parameter initialization. We also compare with a UniLM to see the basic
performance of the unified encoder-decoder structure. For fair comparison, we
continually pre-trained the PPTOD and the UniLM on our pre-trained dialog
corpora (AnPreDial and UnPreDial) to mitigate the data discrepancy, which are
denoted with ∗ in the Table 3. To extensively evaluate our model, we aggregate
the results on eight different benchmarks, including intent prediction (IP),
dialog state tracking (DST) and end-to-end dialog modeling (E2E) tasks. We
calculate the turn accuracy (ACC), joint goal accuracy (JGA) and combined
scores (Comb) for these tasks, and compute the average score as the overall
metric to measure the model performance. Due to few-shot learning gains
increasing attention in various dialog tasks to assess model capability (Geng
et al., 2019; Dai et al., 2020; Tseng et al., 2021), we conduct experiments
under both a full-data setting that uses all training data to fine-tune the
models and a few-shot setting that uses only 10% of training data to fine-tune
the models. To avoid the data leakage in the few-shot setting, we only use the
10% of the training data of the target evaluation dataset for pre-training if
it exists in the AnPreDial. As shown in Table 3, SPACE-3 outperforms all
baselines on all datasets. On the full-data setting and the few-shot setting,
SPACE-3 surpasses the PPTOD∗ with 1.75 and 2.48 absolute average score
improvement respectively, indicating the SPACE-3 have better adaptability on
all types of dialog tasks. The good results of UniLM∗ show that its basic
model architecture is well suitable for both dialog understanding and
generation, but SPACE-3 achieves much better performance due to our pre-
trained objectives can learn better characteristics of task-flow in TOD.

Table 4. Comparison results on dialog understanding tasks.

Model | BANKING77 | CLINIC150 | HWU64
---|---|---|---
ConvBERT | 92.95 | 97.07 | 90.43
USE+ConvRT | 93.36 | 97.16 | 92.62
Example+Observer | 93.83 | 97.31 | 93.03
ConvFiT | 94.16 | 97.34 | 92.42
SPACE-3 | 94.94 | 97.89 | 94.14
Model | MultiWOZ2.2
DS-DST | 51.7
Seq2Seq-DU | 54.40
PLATO-XL | 57.16
AG-DST | 57.26
SPACE-3 | 57.50

###  6.2. Model Comparison on Individual Tasks

We also compare SPACE-3 with state-of-the-art methods on individual tasks.
Generally, these strong baselines are tailored to a specific task and cannot
be adapted to the other tasks.

####  6.2.1. Intent Prediction

The experimental results of three intent prediction datasets under the full-
data setting are shown in the upper part of Table 4. We can see that SPACE-3
achieves the best results on all the datasets. Specifically, SPACE-3
outperforms the previous SOTA model ConvFiT by 0.78%, 0.55% and 1.72% on
BANKING77, CLINC150 and HWU64, respectively. The performance improvements
suggest that SPACE-3 has a better ability to discriminate intents via tree-
structure based contrastive learning than the general pre-training method in
ConvFiT.

####  6.2.2. Dialog State Tracking

To demonstrate whether SPACE-3 can handle multi-turn dialog understanding, we
compare it with previous SOTA models on MultiWOZ2.2. As shown in the bottom
part of Table 4, using SPACE-3 as the base of the TripPy model is able to
obtain the best joint goal accuracy in comparison with other baselines. In
particular, without any special post-processing or gigantic parameters,
SPACE-3 outperforms AG-DST and PLATO-XL (11B parameters) by 0.24% and 0.34%,
respectively.

####  6.2.3. End-to-End Dialog Modeling

Following previous work (Lubis et al., 2020; Zhang et al., 2020a), the Success
and SuccF1 can be used to assess the performance of policy optimization since
they indicate whether the system has solved all user’s requests and provided
correct entities for successful task completion. From Table 5 we can see that
SPACE-3 achieves best Success or SuccF1 on all four benchmarks, and also has
promising BLEU on par with other strong baselines. SPACE-3 acquires the best
combined scores, outperforming the best baselines by 1.02, 2.18, 3.91 and 4.54
points for CamRest676, In-Car, MultiWOZ2.0 and MultiWOZ2.1, respectively.

Table 5. Comparison results on four E2E dialog tasks.

Model | CamRest676 | In-Car
---|---|---
Match | SuccF1 | BLEU | Comb | Match | SuccF1 | BLEU | Comb
SEDST | 92.7 | 75.4 | 23.6 | 107.65 | 84.5 | 82.9 | 19.3 | 103.00
TSCP | 92.7 | 85.4 | 25.3 | 114.35 | 84.5 | 81.1 | 21.9 | 104.70
LABES | 96.4 | 83.0 | 25.5 | 115.2 | 85.8 | 77.0 | 22.8 | 104.20
FSDM | 93.5 | 86.2 | 25.8 | 115.65 | 84.8 | 82.1 | 21.5 | 104.95
SPACE-3 | 97.74 | 88.24 | 23.68 | 116.67 | 85.26 | 83.16 | 22.92 | 107.13
Model | MultiWOZ2.0 | MultiWOZ2.1
Inform | Success | BLEU | Comb | Inform | Success | BLEU | Comb
SimpleTOD | 84.40 | 70.10 | 15.01 | 92.26 | 85.00 | 70.50 | 15.23 | 92.98
DoTS | 86.59 | 74.14 | 15.06 | 95.43 | 86.65 | 74.18 | 15.90 | 96.31
UBAR | 95.40 | 80.70 | 17.00 | 105.05 | 95.70 | 81.80 | 16.50 | 105.25
MTTOD | 90.99 | 82.58 | 20.25 | 107.04 | 90.99 | 82.08 | 19.68 | 106.22
SPACE-3 | 95.30 | 88.00 | 19.30 | 110.95 | 95.60 | 86.10 | 19.91 | 110.76

###  6.3. Ablation Study

In this section, we attempt to answer three questions regarding SPACE-3: (i)
Whether the prompt tokens are useful? (ii) How do the understanding semantic
modeling and semantic region modeling contribute to different tasks? (iii) Is
the policy semantic modeling crucial for the end-to-end dialog modeling?

Table 6 provides detailed results of our ablation study. We first replace our
understanding prompt tokens $p^{u}$ with the original input representations,
calculating all semantic modeling loss based on the representation of [CLS].
As shown in the upper part of Table 6, the performance of the new model
(denoted as w/o $p^{u}$) degrades largely on all understanding tasks,
demonstrating the effectiveness of using those prompt tokens. Then, we discard
the $L_{slm},L_{scl},L_{bow}$ losses individually, and investigate SPACE-3 on
the three IP tasks and the DST task. Results show that the span prediction
$L_{slm}$ is more important for the intent prediction, while $L_{scl}$
contributes more to dialog state tracking. This may be because multi-turn
dialog understanding requires more complex semantics learned by $L_{scl}$. As
an auxiliary loss, $L_{bow}$ can improve the model performance slightly. For
the E2E task, from the bottom part of Table 6 we can find that the results of
Success and BLUE significantly deteriorate if we discard $L_{psm}$, suggesting
that dialog policy is the most crucial component for the final dialog success
and overall generation quality since it is the basis for the later dialog
generation.

Table 6. Ablation study on the IP, DST and E2E tasks. Bold numbers indicate
the worst performance. The differences are shown in the bracket.

Model | IP | DST
---|---|---
BANKING77 | CLINIC150 | HWU64 | MultiWOZ2.2
SPACE-3 | 94.94 | 97.89 | 94.14 | 57.50
w/o $p^{u}$ | 94.29(-0.65) | 97.56(-0.33) | 93.49(-0.65) | 57.15(-0.35)
w/o $L_{bow}$ | 94.87(-0.07) | 97.76(-0.13) | 93.77(-0.37) | 57.08(-0.42)
w/o $L_{slm}$ | 94.12(-0.82) | 97.31(-0.58) | 92.94(-1.20) | 56.42(-1.08)
w/o $L_{scl}$ | 94.25(-0.69) | 97.31(-0.58) | 93.59(-0.55) | 55.96(-1.54)
Model | E2E: MultiWOZ2.0
Inform | Success | BLEU | Comb
SPACE-3 | 95.30 | 88.00 | 19.30 | 110.95
w/o $L_{bow}$ | 94.50(-0.8) | 86.40(-1.6) | 19.01(-0.29) | 109.46(-1.49)
w/o $L_{slm}$ | 93.10(-2.2) | 86.10(-1.9) | 18.95(-0.35) | 108.55(-2.40)
w/o $L_{scl}$ | 92.60(-2.7) | 85.80(-2.2) | 18.70(-0.60) | 107.90(-3.05)
w/o $L_{psm}$ | 93.20(-2.1) | 83.10(-4.9) | 18.59(-0.71) | 106.74(-4.21)

Figure 5. Case Study: Belief states, system acts and delexicalized responses
generated by SPACE-3 and MTTOD on MultiWOZ2.0 test data, which are marked in
different colors.

###  6.4. Case Study

As illustrated in Figure 5, both SPACE-3 and MTTOD can predict the correct
belief states during the conversation, showing the great performance of
understanding. However, MTTOD takes the wrong dialog acts offerbook and inform
and at the beginning turn, and a redundant dialog act offerbook at the second
turn, which leads to inappropriate responses and a failure for the
interaction. On the contrary, SPACE-3 chooses correct dialog acts for the
first two turns so that the whole conversation can steer towards successful
task completion, which also verifies our stronger policy planning and
generation abilities.

##  7\. Conclusion

In this paper, we proposed SPACE-3, a unified pre-trained conversation model,
which could be effectively applied to various downstream dialog tasks. SPACE-3
consisted of four successive components to maintain the task-flow in a task-
oriented dialog system via multi-task learning, including a context encoder to
encode dialog history, an understanding module to extract structured dialog
semantics, a policy module to determine dialog acts, and a generation module
to produce responses. Five dedicated pre-training objectives were employed to
optimize the four components of SPACE-3 jointly in a multi-task paradigm.
Experiments demonstrated that SPACE-3 created new SOTA results on several TOD
benchmarks, on both full-data or the few-shot training settings. We hope that
SPACE-3 and the newly collected dialog dataset (AnPreDial and UnPreDial) can
push forward the research in this area.

###### Acknowledgements.

This work was supported by National Natural Science Foundation of China (No.
61906185), Youth Innovation Promotion Association of CAS China (No. 2020357),
Shenzhen Science and Technology Innovation Program (Grant No.
KQTD20190929172835662), Shenzhen Basic Research Foundation (No.
JCYJ20210324115614039 and No. JCYJ20200109113441941). This work was supported
by Alibaba Group through Alibaba Research Intern Program.

## References

  * (1)
  * Alonso et al. (2021) Inigo Alonso, Alberto Sabater, David Ferstl, Luis Montesano, and Ana C Murillo. 2021.  Semi-Supervised Semantic Segmentation with Pixel-Level Contrastive Learning from a Class-wise Memory Bank.  _arXiv preprint arXiv:2104.13415_ (2021).
  * Assran et al. (2020) Mahmoud Assran, Nicolas Ballas, Lluis Castrejon, and Michael Rabbat. 2020.  Supervision Accelerates Pre-training in Contrastive Semi-Supervised Learning of Visual Representations.  _arXiv preprint arXiv:2006.10803_ (2020).
  * Bao et al. (2020) Siqi Bao, Huang He, Fan Wang, Hua Wu, and Haifeng Wang. 2020.  PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Online, 85–96.  https://doi.org/10.18653/v1/2020.acl-main.9
  * Bao et al. (2021) Siqi Bao, Huang He, Fan Wang, Hua Wu, Haifeng Wang, Wenquan Wu, Zhihua Wu, Zhen Guo, Hua Lu, Xinxian Huang, et al. 2021\.  Plato-xl: Exploring the large-scale pre-training of dialogue generation.  _arXiv preprint arXiv:2109.09519_ (2021).
  * Budzianowski et al. (2018) Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gašić. 2018.  MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics, Brussels, Belgium, 5016–5026.  https://doi.org/10.18653/v1/D18-1547
  * Byrne et al. (2019) Bill Byrne, Karthik Krishnamoorthi, Chinnadhurai Sankar, Arvind Neelakantan, Ben Goodrich, Daniel Duckworth, Semih Yavuz, Amit Dubey, Kyu-Young Kim, and Andy Cedilnik. 2019.  Taskmaster-1: Toward a Realistic and Diverse Dialog Dataset. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_. Association for Computational Linguistics, Hong Kong, China, 4516–4525.  https://doi.org/10.18653/v1/D19-1459
  * Campagna et al. (2020) Giovanni Campagna, Agata Foryciarz, Mehrad Moradshahi, and Monica Lam. 2020.  Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State Tracking. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Online, 122–132.  https://doi.org/10.18653/v1/2020.acl-main.12
  * Casanueva et al. (2020) Iñigo Casanueva, Tadas Temčinas, Daniela Gerz, Matthew Henderson, and Ivan Vulić. 2020\.  Efficient Intent Detection with Dual Sentence Encoders. In _Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI_. Association for Computational Linguistics, Online, 38–45.  https://doi.org/10.18653/v1/2020.nlp4convai-1.5
  * Chen et al. (2021) Derek Chen, Howard Chen, Yi Yang, Alex Lin, and Zhou Yu. 2021.  Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems.  _arXiv preprint arXiv:2104.00783_ (2021).
  * Clark et al. (2019) Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019.  What Does BERT Look at? An Analysis of BERT’s Attention. In _Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_. Association for Computational Linguistics, Florence, Italy, 276–286.  https://doi.org/10.18653/v1/W19-4828
  * Coope et al. (2020) Samuel Coope, Tyler Farghly, Daniela Gerz, Ivan Vulić, and Matthew Henderson. 2020.  Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational Representations. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Online, 107–121.  https://doi.org/10.18653/v1/2020.acl-main.11
  * Coucke et al. (2018) Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, et al. 2018\.  Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces.  _arXiv preprint arXiv:1805.10190_ (2018).
  * Dai et al. (2021b) Yinpei Dai, Hangyu Li, Yongbin Li, Jian Sun, Fei Huang, Luo Si, and Xiaodan Zhu. 2021b.  Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialogue State Tracking. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_. Association for Computational Linguistics, Online, 879–885.  https://doi.org/10.18653/v1/2021.acl-short.111
  * Dai et al. (2020) Yinpei Dai, Hangyu Li, Chengguang Tang, Yongbin Li, Jian Sun, and Xiaodan Zhu. 2020\.  Learning Low-Resource End-To-End Goal-Oriented Dialog for Fast and Reliable System Deployment. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Online, 609–618.  https://doi.org/10.18653/v1/2020.acl-main.57
  * Dai et al. (2021a) Zhigang Dai, Bolun Cai, Yugeng Lin, and Junying Chen. 2021a.  UniMoCo: Unsupervised, Semi-Supervised and Full-Supervised Visual Representation Learning.  _arXiv preprint arXiv:2103.10773_ (2021).
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_. Association for Computational Linguistics, Minneapolis, Minnesota, 4171–4186.  https://doi.org/10.18653/v1/N19-1423
  * Dinan et al. (2018) Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2018\.  Wizard of wikipedia: Knowledge-powered conversational agents.  _arXiv preprint arXiv:1811.01241_ (2018).
  * Dong et al. (2019) Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019.  Unified Language Model Pre-training for Natural Language Understanding and Generation. In _33rd Conference on Neural Information Processing Systems (NeurIPS 2019)_.
  * El Asri et al. (2017) Layla El Asri, Hannes Schulz, Shikhar Sharma, Jeremie Zumer, Justin Harris, Emery Fine, Rahul Mehrotra, and Kaheer Suleman. 2017\.  Frames: a corpus for adding memory to goal-oriented dialogue systems. In _Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue_. Association for Computational Linguistics, Saarbrücken, Germany, 207–219.  https://doi.org/10.18653/v1/W17-5526
  * Eric et al. (2020) Mihail Eric, Rahul Goel, Shachi Paul, Abhishek Sethi, Sanchit Agarwal, Shuyang Gao, Adarsh Kumar, Anuj Goyal, Peter Ku, and Dilek Hakkani-Tur. 2020\.  MultiWOZ 2.1: A Consolidated Multi-Domain Dialogue Dataset with State Corrections and State Tracking Baselines. In _Proceedings of the 12th Language Resources and Evaluation Conference_. European Language Resources Association, Marseille, France, 422–428.  https://aclanthology.org/2020.lrec-1.53
  * Eric et al. (2017) Mihail Eric, Lakshmi Krishnan, Francois Charette, and Christopher D. Manning. 2017.  Key-Value Retrieval Networks for Task-Oriented Dialogue. In _Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue_. Association for Computational Linguistics, Saarbrücken, Germany, 37–49.  https://doi.org/10.18653/v1/W17-5506
  * Fainberg et al. (2018) Joachim Fainberg, Ben Krause, Mihai Dobre, Marco Damonte, Emmanuel Kahembwe, Daniel Duma, Bonnie Webber, and Federico Fancellu. 2018\.  Talking to myself: self-dialogues as data for conversational agents.  _arXiv preprint arXiv:1809.06641_ (2018).
  * Feng et al. (2020) Song Feng, Hui Wan, Chulaka Gunasekara, Siva Patel, Sachindra Joshi, and Luis Lastras. 2020\.  doc2dial: A Goal-Oriented Document-Grounded Dialogue Dataset. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Association for Computational Linguistics, Online, 8118–8128.  https://doi.org/10.18653/v1/2020.emnlp-main.652
  * Feng et al. (2021) Yue Feng, Yang Wang, and Hang Li. 2021.  A Sequence-to-Sequence Approach to Dialogue State Tracking. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_. Association for Computational Linguistics, Online, 1714–1725.  https://doi.org/10.18653/v1/2021.acl-long.135
  * Gao et al. (2021) Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021.  SimCSE: Simple Contrastive Learning of Sentence Embeddings.  _arXiv preprint arXiv:2104.08821_ (2021).
  * Geng et al. (2019) Ruiying Geng, Binhua Li, Yongbin Li, Xiaodan Zhu, Ping Jian, and Jian Sun. 2019\.  Induction Networks for Few-Shot Text Classification. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_. Association for Computational Linguistics, Hong Kong, China, 3904–3913.  https://doi.org/10.18653/v1/D19-1403
  * Gopalakrishnan et al. (2019) Karthik Gopalakrishnan, Behnam Hedayatnia, Qinglang Chen, Anna Gottardi, Sanjeev Kwatra, Anu Venkatesh, Raefer Gabriel, Dilek Hakkani-Tür, and Amazon Alexa AI. 2019.  Topical-Chat: Towards Knowledge-Grounded Open-Domain Conversations.. In _INTERSPEECH_. 1891–1895.
  * Gupta et al. (2019) Mansi Gupta, Nitish Kulkarni, Raghuveer Chanda, Anirudha Rayasam, and Zachary C Lipton. 2019\.  AmazonQA: A review-based question answering task.  _arXiv preprint arXiv:1908.04364_ (2019).
  * Gupta et al. (2018) Sonal Gupta, Rushin Shah, Mrinal Mohit, Anuj Kumar, and Mike Lewis. 2018.  Semantic Parsing for Task Oriented Dialog using Hierarchical Representations. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics, Brussels, Belgium, 2787–2792.  https://doi.org/10.18653/v1/D18-1300
  * He et al. (2020b) Keqing He, Jinchao Zhang, Yuanmeng Yan, Weiran Xu, Cheng Niu, and Jie Zhou. 2020b.  Contrastive Zero-Shot Learning for Cross-Domain Slot Filling with Adversarial Attack. In _Proceedings of the 28th International Conference on Computational Linguistics_. 1461–1467.
  * He et al. (2022) Wanwei He, Yinpei Dai, Yinhe Zheng, Yuchuan Wu, Zheng Cao, Dermot Liu, Peng Jiang, Min Yang, Fei Huang, Luo Si, et al. 2022\.  SPACE: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection.  _Proceedings of the AAAI Conference on Artificial Intelligence_ (2022).
  * He et al. (2021) Wanwei He, Yang Sun, Min Yang, Feng Ji, Chengming Li, and Ruifeng Xu. 2021\.  Multi-goal multi-agent learning for task-oriented dialogue with bidirectional teacher–student learning.  _Knowledge-Based Systems_ 213 (2021), 106667.  https://doi.org/10.1016/j.knosys.2020.106667
  * He et al. (2020a) Wanwei He, Min Yang, Rui Yan, Chengming Li, Ying Shen, and Ruifeng Xu. 2020a.  Amalgamating Knowledge from Two Teachers for Task-oriented Dialogue System with Adversarial Training. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Association for Computational Linguistics, Online, 3498–3507.  https://doi.org/10.18653/v1/2020.emnlp-main.281
  * Heck et al. (2020) Michael Heck, Carel van Niekerk, Nurul Lubis, Christian Geishauser, Hsien-Chin Lin, Marco Moresi, and Milica Gasic. 2020\.  TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Tracking. In _Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue_. Association for Computational Linguistics, 1st virtual meeting, 35–44.  https://aclanthology.org/2020.sigdial-1.4
  * Hemphill et al. (1990) Charles T. Hemphill, John J. Godfrey, and George R. Doddington. 1990.  The ATIS Spoken Language Systems Pilot Corpus. In _Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27,1990_.  https://aclanthology.org/H90-1021
  * Henderson et al. (2019) Matthew Henderson, Inigo Casanueva, Nikola Mrkšić, Pei-Hao Su, Tsung-Hsien Wen, and Ivan Vulić. 2019.  Convert: Efficient and accurate conversational representations from transformers.  _arXiv preprint arXiv:1911.03688_ (2019).
  * Hosseini-Asl et al. (2020) Ehsan Hosseini-Asl, Bryan McCann, Chien-Sheng Wu, Semih Yavuz, and Richard Socher. 2020.  A simple language model for task-oriented dialogue.  _arXiv preprint arXiv:2005.00796_ (2020).
  * Inoue and Goto (2020) Nakamasa Inoue and Keita Goto. 2020.  Semi-supervised contrastive learning with generalized contrastive loss and its application to speaker recognition. In _2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)_. IEEE, 1641–1646.
  * Jeon and Lee (2021) Hyunmin Jeon and Gary Geunbae Lee. 2021.  Domain State Tracking for a Simplified Dialogue System.  _arXiv preprint arXiv:2103.06648_ (2021).
  * Jin et al. (2018) Xisen Jin, Wenqiang Lei, Zhaochun Ren, Hongshen Chen, Shangsong Liang, Yihong Zhao, and Dawei Yin. 2018.  Explicit state tracking with semi-supervisionfor neural dialogue generation. In _Proceedings of the 27th ACM International Conference on Information and Knowledge Management_. 1403–1412.
  * Joshi et al. (2020) Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020\.  SpanBERT: Improving Pre-training by Representing and Predicting Spans.  _Transactions of the Association for Computational Linguistics_ 8 (2020), 64–77.  https://doi.org/10.1162/tacl_a_00300
  * Kacupaj et al. (2021) Endri Kacupaj, Joan Plepi, Kuldeep Singh, Harsh Thakkar, Jens Lehmann, and Maria Maleshkova. 2021\.  Conversational Question Answering over Knowledge Graphs with Transformer and Graph Attention Networks. In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_. Association for Computational Linguistics, Online, 850–862.  https://doi.org/10.18653/v1/2021.eacl-main.72
  * Khosla et al. (2020) Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020.  Supervised contrastive learning.  _arXiv preprint arXiv:2004.11362_ (2020).
  * Kim et al. (2021a) Byoungjip Kim, Jinho Choo, Yeong-Dae Kwon, Seongho Joe, Seungjai Min, and Youngjune Gwon. 2021a.  SelfMatch: Combining Contrastive Self-Supervision and Consistency for Semi-Supervised Learning.  _arXiv preprint arXiv:2101.06480_ (2021).
  * Kim et al. (2021b) Seokhwan Kim, Yang Liu, Di Jin, Alexandros Papangelis, Karthik Gopalakrishnan, Behnam Hedayatnia, and Dilek Hakkani-Tur. 2021b.  ”How robust r u?”: Evaluating Task-Oriented Dialogue Systems on Spoken Conversations.  arXiv:2109.13489 [cs.CL]
  * Kottur et al. (2021) Satwik Kottur, Seungwhan Moon, Alborz Geramifard, and Babak Damavandi. 2021.  SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations.  _arXiv preprint arXiv:2104.08667_ (2021).
  * Kulhánek et al. (2021) Jonáš Kulhánek, Vojtěch Hudeček, Tomáš Nekvinda, and Ondřej Dušek. 2021.  Augpt: Dialogue with pre-trained language models and data augmentation.  _arXiv preprint arXiv:2102.05126_ (2021).
  * Larson et al. (2019) Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, and Jason Mars. 2019.  An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_. Association for Computational Linguistics, Hong Kong, China, 1311–1316.  https://doi.org/10.18653/v1/D19-1131
  * Lee (2021) Yohan Lee. 2021\.  Improving End-to-End Task-Oriented Dialog System with A Simple Auxiliary Task. In _Findings of the Association for Computational Linguistics: EMNLP 2021_. Association for Computational Linguistics, Punta Cana, Dominican Republic, 1296–1303.  https://doi.org/10.18653/v1/2021.findings-emnlp.112
  * Lei et al. (2018) Wenqiang Lei, Xisen Jin, Min-Yen Kan, Zhaochun Ren, Xiangnan He, and Dawei Yin. 2018\.  Sequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. Association for Computational Linguistics, Melbourne, Australia, 1437–1447.  https://doi.org/10.18653/v1/P18-1133
  * Lester et al. (2021) Brian Lester, Rami Al-Rfou, and Noah Constant. 2021\.  The Power of Scale for Parameter-Efficient Prompt Tuning. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 3045–3059.  https://doi.org/10.18653/v1/2021.emnlp-main.243
  * Li et al. (2021) Junnan Li, Caiming Xiong, and Steven CH Hoi. 2021.  Comatch: Semi-supervised learning with contrastive graph regularization. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 9475–9484.
  * Li et al. (2020) Shiyang Li, Semih Yavuz, Kazuma Hashimoto, Jia Li, Tong Niu, Nazneen Rajani, Xifeng Yan, Yingbo Zhou, and Caiming Xiong. 2020.  Coco: Controllable counterfactuals for evaluating dialogue state trackers.  _arXiv preprint arXiv:2010.12850_ (2020).
  * Li et al. (2018) Xiujun Li, Yu Wang, Siqi Sun, Sarah Panda, Jingjing Liu, and Jianfeng Gao. 2018\.  Microsoft dialogue challenge: Building end-to-end task-completion dialogue systems.  _arXiv preprint arXiv:1807.11125_ (2018).
  * Li et al. (2017) Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017\.  Dailydialog: A manually labelled multi-turn dialogue dataset.  _arXiv preprint arXiv:1710.03957_ (2017).
  * Lin et al. (2021) Zhaojiang Lin, Andrea Madotto, Genta Indra Winata, Peng Xu, Feijun Jiang, Yuxiang Hu, Chen Shi, and Pascale Fung. 2021\.  BiToD: A Bilingual Multi-Domain Dataset For Task-Oriented Dialogue Modeling.  _arXiv preprint arXiv:2106.02787_ (2021).
  * Liu et al. (2013) Jingjing Liu, Panupong Pasupat, Scott Cyphers, and Jim Glass. 2013.  Asgard: A portable architecture for multilingual dialogue systems. In _2013 IEEE International Conference on Acoustics, Speech and Signal Processing_. IEEE, 8386–8390.
  * Liu et al. (2021) Xingkun Liu, Arash Eshghi, Pawel Swietojanski, and Verena Rieser. 2021.  Benchmarking natural language understanding services for building conversational agents.  In _Increasing Naturalness and Flexibility in Spoken Dialogue Interaction_. Springer, 165–183.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_ (2019).
  * Liu et al. (2020) Zihan Liu, Yan Xu, Tiezheng Yu, Wenliang Dai, Ziwei Ji, Samuel Cahyawijaya, Andrea Madotto, and Pascale Fung. 2020\.  CrossNER: Evaluating Cross-Domain Named Entity Recognition.  _arXiv preprint arXiv:2012.04373_ (2020).
  * Lubis et al. (2020) Nurul Lubis, Christian Geishauser, Michael Heck, Hsien-chin Lin, Marco Moresi, Carel van Niekerk, and Milica Gasic. 2020.  LAVA: Latent Action Spaces via Variational Auto-encoding for Dialogue Policy Optimization. In _Proceedings of the 28th International Conference on Computational Linguistics_. International Committee on Computational Linguistics, Barcelona, Spain (Online), 465–479.  https://doi.org/10.18653/v1/2020.coling-main.41
  * Mehri and Eric (2021) Shikib Mehri and Mihail Eric. 2021.  Example-Driven Intent Prediction with Observers. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. Association for Computational Linguistics, Online, 2979–2992.  https://doi.org/10.18653/v1/2021.naacl-main.237
  * Mehri et al. (2020) Shikib Mehri, Mihail Eric, and Dilek Hakkani-Tur. 2020\.  Dialoglue: A natural language understanding benchmark for task-oriented dialogue.  _arXiv preprint arXiv:2009.13570_ (2020).
  * Mehri et al. (2019a) Shikib Mehri, Evgeniia Razumovskaia, Tiancheng Zhao, and Maxine Eskenazi. 2019a.  Pretraining methods for dialog context representation learning.  _arXiv preprint arXiv:1906.00414_ (2019).
  * Mehri et al. (2019b) Shikib Mehri, Tejas Srinivasan, and Maxine Eskenazi. 2019b.  Structured Fusion Networks for Dialog. In _Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue_. Association for Computational Linguistics, Stockholm, Sweden, 165–177.  https://doi.org/10.18653/v1/W19-5921
  * Mosig et al. (2020) Johannes EM Mosig, Shikib Mehri, and Thomas Kober. 2020\.  Star: A schema-guided dialog dataset for transfer learning.  _arXiv preprint arXiv:2010.11853_ (2020).
  * Mrkšić et al. (2017) Nikola Mrkšić, Diarmuid Ó Séaghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve Young. 2017\.  Neural Belief Tracker: Data-Driven Dialogue State Tracking. In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. Association for Computational Linguistics, Vancouver, Canada, 1777–1788.  https://doi.org/10.18653/v1/P17-1163
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002\.  Bleu: a Method for Automatic Evaluation of Machine Translation. In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, 311–318.  https://doi.org/10.3115/1073083.1073135
  * Park et al. (2021) Jongjin Park, Sukmin Yun, Jongheon Jeong, and Jinwoo Shin. 2021\.  OpenCoS: Contrastive Semi-supervised Learning for Handling Open-set Unlabeled Data.  _arXiv preprint arXiv:2107.08943_ (2021).
  * Peng et al. (2022) Baolin Peng, Chunyuan Li, Jinchao Li, Shahin Shayandeh, Lars Liden, and Jianfeng Gao. 2022\.  SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching.  _Transactions of the Association for Computational Linguistics_ 9, 0 (2022), 907–824.  https://transacl.org/ojs/index.php/tacl/article/view/2877
  * Peng et al. (2020) Baolin Peng, Chenguang Zhu, Chunyuan Li, Xiujun Li, Jinchao Li, Michael Zeng, and Jianfeng Gao. 2020.  Few-shot natural language generation for task-oriented dialog.  _arXiv preprint arXiv:2002.12328_ (2020).
  * Peskov et al. (2019) Denis Peskov, Nancy Clarke, Jason Krone, Brigi Fodor, Yi Zhang, Adel Youssef, and Mona Diab. 2019.  Multi-Domain Goal-Oriented Dialogues (MultiDoGO): Strategies toward Curating and Annotating Large Scale Dialogue Data. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_. Association for Computational Linguistics, Hong Kong, China, 4526–4536.  https://doi.org/10.18653/v1/D19-1460
  * Radford et al. (2018) Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018.  Improving language understanding by generative pre-training.  (2018).
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019\.  Language models are unsupervised multitask learners.  _OpenAI blog_ 1, 8 (2019), 9.
  * Radlinski et al. (2019) Filip Radlinski, Krisztian Balog, Bill Byrne, and Karthik Krishnamoorthi. 2019.  Coached Conversational Preference Elicitation: A Case Study in Understanding Movie Preferences. In _Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue_. Association for Computational Linguistics, Stockholm, Sweden, 353–360.  https://doi.org/10.18653/v1/W19-5941
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.  _Journal of Machine Learning Research_ 21, 140 (2020), 1–67.  http://jmlr.org/papers/v21/20-074.html
  * Rajpurkar et al. (2016) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016.  SQuAD: 100,000+ Questions for Machine Comprehension of Text. In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics, Austin, Texas, 2383–2392.  https://doi.org/10.18653/v1/D16-1264
  * Rastogi et al. (2020) Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, and Pranav Khaitan. 2020.  Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 34. 8689–8696.
  * Reddy et al. (2019) Siva Reddy, Danqi Chen, and Christopher D Manning. 2019\.  Coqa: A conversational question answering challenge.  _Transactions of the Association for Computational Linguistics_ 7 (2019), 249–266.
  * Roller et al. (2020) Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M Smith, et al. 2020\.  Recipes for building an open-domain chatbot.  _arXiv preprint arXiv:2004.13637_ (2020).
  * Saha et al. (2017) Amrita Saha, Mitesh M. Khapra, and Karthik Sankaranarayanan. 2017\.  Towards Building Large Scale Multimodal Domain-Aware Conversation Systems.  (2017).  arXiv:arXiv:1704.00200
  * Saha et al. (2018) Amrita Saha, Vardaan Pahuja, Mitesh M. Khapra, Karthik Sankaranarayanan, and Sarath Chandar. 2018\.  Complex Sequential Question Answering: Towards Learning to Converse Over Linked Question Answer Pairs with a Knowledge Graph.  (2018).  arXiv:arXiv:1801.10314
  * Schuster et al. (2019) Sebastian Schuster, Sonal Gupta, Rushin Shah, and Mike Lewis. 2019\.  Cross-lingual Transfer Learning for Multilingual Task Oriented Dialog. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_. Association for Computational Linguistics, Minneapolis, Minnesota, 3795–3805.  https://doi.org/10.18653/v1/N19-1380
  * Shah et al. (2018) Pararth Shah, Dilek Hakkani-Tür, Bing Liu, and Gokhan Tür. 2018.  Bootstrapping a Neural Conversational Agent with Dialogue Self-Play, Crowdsourcing and On-Line Reinforcement Learning. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers)_. Association for Computational Linguistics, New Orleans - Louisiana, 41–51.  https://doi.org/10.18653/v1/N18-3006
  * Shalyminov et al. (2020) Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, and Hannes Schulz. 2020.  Fast domain adaptation for goal-oriented dialogue using a hybrid generative-retrieval transformer. In _ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 8039–8043.
  * Shu et al. (2019) Lei Shu, Piero Molino, Mahdi Namazifar, Hu Xu, Bing Liu, Huaixiu Zheng, and Gokhan Tur. 2019.  Flexibly-structured model for task-oriented dialogues.  _arXiv preprint arXiv:1908.02402_ (2019).
  * Stolcke et al. (2000) Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000.  Dialogue act modeling for automatic tagging and recognition of conversational speech.  _Computational Linguistics_ 26, 3 (2000), 339–374.  https://aclanthology.org/J00-3003
  * Su et al. (2021) Yixuan Su, Lei Shu, Elman Mansimov, Arshit Gupta, Deng Cai, Yi-An Lai, and Yi Zhang. 2021.  Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System.  _CoRR_ abs/2109.14739 (2021).  arXiv:2109.14739 https://arxiv.org/abs/2109.14739
  * Tian et al. (2021) Xin Tian, Liankai Huang, Yingzhan Lin, Siqi Bao, Huang He, Yunyi Yang, Hua Wu, Fan Wang, and Shuqi Sun. 2021.  Amendable Generation for Dialogue State Tracking. In _Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI_. Association for Computational Linguistics, Online, 80–92.  https://doi.org/10.18653/v1/2021.nlp4convai-1.8
  * Tseng et al. (2021) Bo-Hsiang Tseng, Yinpei Dai, Florian Kreyssig, and Bill Byrne. 2021.  Transferable Dialogue Systems and User Simulators. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_. Association for Computational Linguistics, Online, 152–166.  https://doi.org/10.18653/v1/2021.acl-long.13
  * Vulić et al. (2021) Ivan Vulić, Pei-Hao Su, Sam Coope, Daniela Gerz, Paweł Budzianowski, Iñigo Casanueva, Nikola Mrkšić, and Tsung-Hsien Wen. 2021.  ConvFiT: Conversational Fine-Tuning of Pretrained Language Models.  _arXiv preprint arXiv:2109.10126_ (2021).
  * Wang and Cho (2019) Alex Wang and Kyunghyun Cho. 2019.  BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model. In _Proceedings of the Workshop on Methods for Optimizing and Evaluating Neural Language Generation_. Association for Computational Linguistics, Minneapolis, Minnesota, 30–36.  https://doi.org/10.18653/v1/W19-2304
  * Wang et al. (2018) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018\.  GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In _Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_. Association for Computational Linguistics, Brussels, Belgium, 353–355.  https://doi.org/10.18653/v1/W18-5446
  * Wang et al. (2019) Xuewei Wang, Weiyan Shi, Richard Kim, Yoojung Oh, Sijia Yang, Jingwen Zhang, and Zhou Yu. 2019.  Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Florence, Italy, 5635–5649.  https://doi.org/10.18653/v1/P19-1566
  * Wei et al. (2018) Wei Wei, Quoc Le, Andrew Dai, and Jia Li. 2018\.  AirDialogue: An Environment for Goal-Oriented Dialogue Research. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics, Brussels, Belgium, 3844–3854.  https://doi.org/10.18653/v1/D18-1419
  * Wen et al. (2017) Tsung-Hsien Wen, David Vandyke, Nikola Mrkšić, Milica Gašić, Lina M. Rojas-Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2017.  A Network-based End-to-End Trainable Task-oriented Dialogue System. In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers_. Association for Computational Linguistics, Valencia, Spain, 438–449.  https://aclanthology.org/E17-1042
  * Will et al. (2020) Myers Will, Etchart Tyler, and Fulda Nancy. 2020\.  Conversational scaffolding: An analogybased approach to response prioritization in open-domain dialogs. In _Proceedings of the 12th International Conference on Agents and Artificial Intelligence (ICAART)_.
  * Williams et al. (2016) Jason D Williams, Antoine Raux, and Matthew Henderson. 2016\.  The dialog state tracking challenge series: A review.  _Dialogue & Discourse_ 7, 3 (2016), 4–33.
  * Wu et al. (2020) Chien-Sheng Wu, Steven C.H. Hoi, Richard Socher, and Caiming Xiong. 2020.  TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Association for Computational Linguistics, Online, 917–929.  https://doi.org/10.18653/v1/2020.emnlp-main.66
  * Wu and Xiong (2020) Chien-Sheng Wu and Caiming Xiong. 2020.  Probing task-oriented dialogue representation from language models.  _arXiv preprint arXiv:2010.13912_ (2020).
  * Wu et al. (2021) Xing Wu, Chaochen Gao, Liangjun Zang, Jizhong Han, Zhongyuan Wang, and Songlin Hu. 2021\.  ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embedding.  _arXiv preprint arXiv:2109.04380_ (2021).
  * Xiao et al. (2021) Alex Xiao, Christian Fuegen, and Abdelrahman Mohamed. 2021\.  Contrastive semi-supervised learning for asr. In _ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 3870–3874.
  * Yan et al. (2021) Yuanmeng Yan, Rumei Li, Sirui Wang, Fuzheng Zhang, Wei Wu, and Weiran Xu. 2021\.  ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer.  _arXiv preprint arXiv:2105.11741_ (2021).
  * Yang et al. (2020) Yunyi Yang, Yunhao Li, and Xiaojun Quan. 2020.  UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2.  _arXiv preprint arXiv:2012.03539_ (2020).
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019\.  Xlnet: Generalized autoregressive pretraining for language understanding.  _Advances in neural information processing systems_ 32 (2019).
  * Yu et al. (2019) Tao Yu, Rui Zhang, Heyang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vincent Zhang, Caiming Xiong, Richard Socher, Walter Lasecki, and Dragomir Radev. 2019.  CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_. Association for Computational Linguistics, Hong Kong, China, 1962–1979.  https://doi.org/10.18653/v1/D19-1204
  * Yuan et al. (2021) Xinkai Yuan, Zilinghan Li, and Gaoang Wang. 2021.  ActiveMatch: End-to-end Semi-supervised Active Representation Learning.  _arXiv preprint arXiv:2110.02521_ (2021).
  * Zang et al. (2020) Xiaoxue Zang, Abhinav Rastogi, Srinivas Sunkara, Raghav Gupta, Jianguo Zhang, and Jindong Chen. 2020\.  MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines. In _Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI_. Association for Computational Linguistics, Online, 109–117.  https://doi.org/10.18653/v1/2020.nlp4convai-1.13
  * Zeng et al. (2020) Guangtao Zeng, Wenmian Yang, Zeqian Ju, Yue Yang, Sicheng Wang, Ruisi Zhang, Meng Zhou, Jiaqi Zeng, Xiangyu Dong, Ruoyu Zhang, Hongchao Fang, Penghui Zhu, Shu Chen, and Pengtao Xie. 2020\.  MedDialog: Large-scale Medical Dialogue Datasets. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Association for Computational Linguistics, Online, 9241–9250.  https://doi.org/10.18653/v1/2020.emnlp-main.743
  * Zeng and Nie (2021) Yan Zeng and Jian-Yun Nie. 2021.  An Investigation of Suitability of Pre-Trained Language Models for Dialogue Generation–Avoiding Discrepancies.  (2021).
  * Zhang et al. (2021a) Jianguo Zhang, Trung Bui, Seunghyun Yoon, Xiang Chen, Zhiwei Liu, Congying Xia, Quan Hung Tran, Walter Chang, and Philip Yu. 2021a.  Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning.  _arXiv preprint arXiv:2109.06349_ (2021).
  * Zhang et al. (2019) Jian-Guo Zhang, Kazuma Hashimoto, Chien-Sheng Wu, Yao Wan, Philip S Yu, Richard Socher, and Caiming Xiong. 2019.  Find or classify? dual strategy for slot-value predictions on multi-domain dialog state tracking.  _arXiv preprint arXiv:1910.03544_ (2019).
  * Zhang and Shasha (1989) Kaizhong Zhang and Dennis Shasha. 1989.  Simple fast algorithms for the editing distance between trees and related problems.  _SIAM journal on computing_ 18, 6 (1989), 1245–1262.
  * Zhang et al. (2018) Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018\.  Personalizing Dialogue Agents: I have a dog, do you have pets too?. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. Association for Computational Linguistics, Melbourne, Australia, 2204–2213.  https://doi.org/10.18653/v1/P18-1205
  * Zhang et al. (2020a) Yichi Zhang, Zhijian Ou, Min Hu, and Junlan Feng. 2020a.  A Probabilistic End-To-End Task-Oriented Dialog Model with Latent Belief States towards Semi-Supervised Learning. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Association for Computational Linguistics, Online, 9207–9219.  https://doi.org/10.18653/v1/2020.emnlp-main.740
  * Zhang et al. (2020b) Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020b.  Dialogpt: Large-scale generative pre-training for conversational response generation.  _ACL 2020_ (2020).
  * Zhang et al. (2021b) Zhenyu Zhang, Tao Guo, and Meng Chen. 2021b.  DialogueBERT: A Self-Supervised Learning based Dialogue Pre-training Encoder.  _arXiv preprint arXiv:2109.10480_ (2021).
  * Zhou et al. (2018) Kangyan Zhou, Shrimai Prabhumoye, and Alan W Black. 2018\.  A Dataset for Document Grounded Conversations. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics, Brussels, Belgium, 708–713.  https://doi.org/10.18653/v1/D18-1076
  * Zhou et al. (2021) Yanning Zhou, Hang Xu, Wei Zhang, Bin Gao, and Pheng-Ann Heng. 2021.  C3-SemiSeg: Contrastive Semi-supervised Segmentation via Cross-set Learning and Dynamic Class-balancing. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 7036–7045.