# A Better and Faster End-to-End Model for Streaming ASR

###### Abstract

End-to-end (E2E) models have shown to outperform state-of-the-art conventional
models for streaming speech recognition[1] across many dimensions, including
quality (as measured by word error rate (WER)) and endpointer latency[2].
However, the model still tends to delay the predictions towards the end and
thus has much higher partial latency compared to a conventional ASR model. To
address this issue, we look at encouraging the E2E model to emit words early,
through an algorithm called _FastEmit_[3]. Naturally, improving on latency
results in a quality degradation. To address this, we explore replacing the
LSTM layers in the encoder of our E2E model with _Conformer_ layers[4], which
has shown good improvements for ASR. Secondly, we also explore running a 2nd-
pass beam search to improve quality. In order to ensure the 2nd-pass completes
quickly, we explore non-causal Conformer layers that feed into the same 1st-
pass RNN-T decoder, an algorithm called _Cascaded Encoders_[5]. Overall, the
Conformer RNN-T with Cascaded Encoders offers a better quality and latency
tradeoff for streaming ASR.

Index Terms—  RNN-T, Conformer, cascaded encoders, latency

##  1 Introduction

Having an end-to-end (E2E) speech recognition model [6, 7, 8, 9, 10, 11], that
matches/surpasses the quality of a conventional model (which has separate
acoustic, pronunciation and language model components) across all dimensions
of quality and latency, has become a challenging research direction over the
past few years.

On the latency side, one such requirement of E2E models is that they have to
be streaming. They must emit words to the screen with minimal delay as the
user speaks. Models such as Recurrent Neural Network Transducer (RNN-T) [9],
Neural Transducer [12, 13], Monotonic Attention [14, 15], Triggered attention
[16] and Scout Network [17] have been explored to address the streaming
nature. Another requirement is that the model must stop decoding quickly once
the user stops speaking, a problem known as endpointing. Folding the
endpointer into the E2E model helped address this issue [2].

On the quality side, we require a model which achieves better word error rate
(WER) compared to a conventional model. Using a 1st-pass streaming RNN-T model
coupled with a 2nd-pass non-streaming attention-based model helped address
quality concerns [18]. In addition, improving the performance of the model to
recognize proper nouns was achieved with contextual biasing [19].

Putting many of these above components together, we recently released an on-
device model that surpassed a conventional model in terms of both WER and
endpointer latency [1]. However, one of the issues with that model is that
partial hypotheses were still delayed to the end compared to a conventional
model trained with an alignment.

In this work, we explore ideas to improve the model’s partial latency. Speech-
to-text alignment can be used to improve the token prediction timing [20, 21],
which normally causes large quality degradation. We adopted the recently
proposed FastEmit[3]. It regularizes the RNN-T lattice to encourage non-blank
tokens and surpass the blank token across the entire sequence based on the
forward-backward probabilities. It is found to be effective in reducing
models’ partial latency with much less quality degradation.

One can expect that any research attempt to improve partial latency results in
a slight quality degradation. To address this, we explore improving our core
E2E model by replacing the recurrent LSTM layers in the encoder with Conformer
layers, which have shown promise in Librispeech [4]. In addition, we also look
at improving our 2nd-pass strategy so that we can run a beam search rather
than a rescorer [5]. We explored the improved latency and quality ideas on a
Voice Search task, which yielded 10% relative WER reduction with 250ms and
170ms reduction in median partial and prefetch latency respectively than [1].

##  2 Model Improvements

###  2.1 Conformer

We use Conformer [4] to improve the model quality. The model used in [4] is
designed for tasks with non-streaming, single-domain and short utterances. In
this work, we adapt it for our streaming multi-domain task [1]. We first limit
the self-attention, the convolution and the normalization layers to take in
only the previous context for streaming applications. We also replaced the
full context self-attention with local self-attention which helps generalize
to long utterances [22]. The same as [1], we train our systems on a
combination of data from different domains. For efficiency, we aggregate
utterances with similar lengths before creating batches (i.e. bucketing). Due
to the different length distributions across domains, for a particular batch
the data may bias towards certain domain. This could lead to biased statistics
for the batch normalization used in the original Conformer layer, and then
limit the model’s generalization capability across domains. We addressed this
issue by replacing the batch normalization layer with the group normalization
layer [23]. This is found critical for the quality of our multi-domain models.

Additionally, we speedup the training and inference by removing the original
relative positional encoding [24]. Instead, we reuse the existing convolution
module that aggregates information from neighboring context to implicitly
provide relative positional information [24, 25]. This is done by simply
swapping the order of the convolution module and the multi-head self-attention
module, leading to the improved Conformer block depicted in Figure 1.

Fig. 1: Improved Conformer block architecture.

###  2.2 Two-pass with Cascaded Encoders

A big goal of two-pass models is to make up for the quality degradation in the
1st-pass. To improve our current two-pass approach, which is to run rescoring,
we look to run a beam search instead. One of the challenges with this beam
search is that we need the model to be robust to decoding short search
utterances as well as long caption utterances. Typically attention-based
models do very poorly on long-form [22].

To address this, we look at running a 2nd-pass beam search with RNN-T itself,
which has shown better robustness to long-form[26]. The model explored, which
we call _Cascaded Encoders_[5], is shown in Figure 2. To add the non-causal
aspect of LAS (which improves quality)[18] into a streaming, causal RNN-T
system, we add additional non-causal encoder layers on top of the causal
encoder layers. The 1st-pass uses only the causal encoder and the RNN-T
decoder. In the second pass, the additional non-causal layers take in both
left and right context of the 1st-pass encoder outputs, and again feeds to the
same decoder. A single RNN-T decoder is shared between the 1st and 2nd-passes
for smaller model size and on-device benefits.

Fig. 2: A block diagram of the Cascaded Encoders model.

##  3 Latency Improvements

In this section, we present latency metrics for streaming ASR and techniques
to reduce them. Different from the average token latency [21], we adopt
metrics that are directly related to streaming speech applications such as
Voice Search and Assistant.

###  3.1 Endpointer Latency

In streaming ASR based applications, when the user finishes speaking is a
critical signal that triggers the next step fulfillment. For Voice
Search/Assistant, it signals the ASR system to finalize the recognition result
and initiate either a server request for more information or an action on the
client side. _Endpointer latency_ measures the time difference between when
the user finishes speaking and when the system predicts the end of the query
(EOQ). This is the latency metric we normally referred to in previous work [2,
1]. A lower Endpointer latency leads to a lower application latency and faster
system responses. We adopt the same RNN-T Endpointer model together with early
and late penalties on EOQ during training to reduce Endpointer latency [2]. We
report both the median (i.e. $50^{th}$ percentile, EP50) and the $90^{th}$
percentile (EP90) latency for comparisons.

###  3.2 Prefetch Latency

Prefetching is a common technique used in web-based applications to reduce
network latency involved in the application stack [27, 28]. Instead of waiting
for the EOQ signal, streaming applications normally send early server requests
(either retrieving relevant information or necessary resources for next step
actions) based on partial/incomplete recognition hypotheses. If a prefetch is
based on a partial hypothesis that does not match the finalized hypothesis at
the time EOQ is generated, it is discarded as a premature prefetching,
increasing the overall computation cost. However, if the partial hypothesis
used in a prefetching matches with the final result, the system can
immediately execute the following action without waiting for additional server
responses. This saves the time used to compute and fetch responses from the
server and reduces the network processing latency. _Prefetch latency_ is
defined as the time difference between when the first correct prefetch is
triggered and when the user finishes speaking [29]. A prefetch is correct only
if the partial recognition result it used matches the final recognition
result. If no correct prefetch is found, the latency falls back to Endpointer
latency. Besides latency, the prefetch rate, which is the number of prefetches
for each query, is also an important metric. For comparison, we report both
the $50^{th}$ and $90^{th}$ percentile latency (PF50 and PF90) together with
the prefetching rate (PFR).

####  3.2.1 E2E Prefetching

There are multiple ways to trigger prefetching [29]. It normally relies on
observing a fixed interval of silence either from the decoding or an external
voice activity detector (VAD). An E2E Prefetching has recently been proposed
and yielded a much better prefetch latency and prefetch rate tradeoff [29]. It
leverages on the EOQ prediction of the RNN-T Endpointer model for prefetching
decisions. Different from endpointing, we want to generate EOQ for every
partial result but avoid changing decoding. To achieve that, for each frame
before EOQ shows on the top beam, we artificially add it to the end of the
current hypothesis to compute its probability given the observations and the
best path so far. This allows us to predict how likely the query is completed
at each time frame and the corresponding partial result while decoding
following frames. If the probability is greater than a predefined threshold,
the system declares a prefetch decision.

###  3.3 Partial Latency

Prefetching relies on partial results generated by the ASR model. Systems that
output partial hypothesis with less latency are preferred. We hence define
_partial latency_ as the time difference between when the first correct
partial hypothesis is generated by the model and when the user finishes
speaking. Similarly, the correctness of a partial hypothesis refers to whether
it matches the finalized recognition hypothesis. Unlike prefetching latency,
partial latency is inherent to the ASR model itself and does not rely on
additional logic implemented in the application pipeline such as the decision
to trigger prefecthing. Partial latency is the lower bound for prefetch
latency. Developing techniques that reduce models’ partial latency would give
more room for reducing the overall application latency and is independent of
the actual application pipeline.

####  3.3.1 Constrained Alignment

One way to improve partial latency is to constrain the model’s prediction
based on alignments generated from other models [20]. Similar to the RNN-T EP
model that constrains the EOU token, constrained alignment penalizes the
prediction of each word’s beginning and ending tokens if they fall out of a
buffer centered around the corresponding ground truth obtained from the
alignment. It has been shown to yield more accurate prediction timing
information and can be used to improve partial latency.

####  3.3.2 FastEmit

In conventional RNN-T optimization, emitting a non-blank token and the blank
token are treated equally, as long as the log-probability of the target
sequence is maximized. However, in streaming ASR systems, the blank token
(i.e. output nothing) should be discouraged as it leads to higher latency. A
simple and effective sequence-level emission regularization technique,
_FastEmit_[3], is hence developed. It encourages predicting non-blank tokens
and suppresses blank tokens across the entire sequence based on transducer
forward-backward probabilities. It is implemented by adding a regularization
hyper parameter $\lambda_{\text{FastEmit}}$ to the original Equation 20 in
[30] (please refer to [30] for detailed notion definitions):

$\frac{\partial\mathcal{L}}{\partial\text{Pr}(k|t,u)}=-\frac{\alpha(t,u)}{\text{Pr}(\mathbf{y}^{*}|\mathbf{x})}\begin{cases}{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}(1+\lambda_{\text{FastEmit}})}\beta(t,u+1)\text{~{}if~{}}k=y_{u+1}\\\
\beta(t+1,u)\text{~{}if~{}}k=\varnothing\\\ 0\text{~{}otherwise.}\end{cases}$

Intuitively, FastEmit applies a “higher learning rate”
($\lambda_{\text{FastEmit}}\geqslant 0$) to the prediction of non-blank token
when back-propagating into the streaming ASR network, while the prediction of
blank token remains the same. It has been shown to give significant lower
latency while retaining recognition accuracy on different RNN-T models. More
importantly, FastEmit does not require any prior alignment information [20,
21] and has no additional training or serving cost.

##  4 Experimental Setup

Model Architecture: All models are trained using a 128D log-Mel feature
frontend [26]. The features are computed using 32ms windows with a 10ms hop.
Features from 4 contiguous frames are stacked to form a 512D input
representation, which is further sub-sampled by a factor of 3. A 16D one-hot
domain-id vector is then appended to form a 528D input vector as the inputs to
the model. Following [4], we use 512D Conformer layers in the encoder. Causal
convolution and left-context attention layers are used for the Conformer layer
to strictly restrict the model to use no future inputs. 8-head attention is
used in the self-attention layer and the Convolution kernel size used is 15.
The encoder consists of 17 Conformer layers. The RNN-T decoder consists of a
prediction network with 2 LSTM layers with 2048 units projected down to 640
output units, and a joint network with a single feed-forward layer with 640
units. Two non-causal Conformer layers that take in additional 5.04s right
context are used in the Cascaded Encoders on top of the streaming encoder
before the decoder. All the E2E models are trained to predict 4,096 word
pieces [31]. The decoding is rescored with a 2nd-pass MaxEnt language
model[32].

The Conformer RNN-T model has 137M parameters. The two non-causal Conformer
layers in the Cascaded Encoders model bring in additional 12M parameters. All
models are trained in Tensorflow [33] using the Lingvo [34] toolkit on
$8\times 8$ Tensor Processing Units (TPU) slices with a global batch size of
4,096 utterances. Models are optimized using synchronized stochastic gradient
descent with the Adam optimizer [35].

Datasets: All the E2E models are trained on audio-text pairs the same as [1],
which is a small fraction of data compared to the trillion-word text-only data
a conventional LM is trained with. To increase vocabulary and diversity of
training data, we explore using more data by incorporating multi-domain
utterances as described in [26]. These multi-domain utterances span domains of
search, farfield, telephony and YouTube. All datasets are anonymized and hand-
transcribed; the transcription for YouTube utterances is done in a semi-
supervised fashion [36]. In addition to the diverse training sets, multi-
condition training (MTR) [37] and random data down-sampling to 8kHz [38] are
also used to further increase data diversity. Noisy data is generated at
signal-noise-ratio (SNR) from 0 to 30 dB, with an average SNR of 12 dB, and
with T60 times ranging from 0 to 900ms, averaging 500ms. Noise segments are
sampled from YouTube and daily life noisy environmental recordings. Both 8 kHz
and 16 kHz versions of the data are generated, each with equal probability, to
make the model robust to varying sample rates.

The test set includes around 12K Voice Search utterances with duration less
than 5.5s long. They are anonymized and hand-transcribed, and are
representative of Google’s Voice Search traffic.

##  5 Results

Table 1: Performance of RNN-T models with different encoders. Exp. | Model | Training Speed | WER
---|---|---|---
Size (M) | (examples/sec.) | (%)
B1 LSTM w MWER [1] | 122 | 3100 | 6.0
C0 w/o MWER [4] | 141 | 3970 | 6.8
C0 No bucketing, w/o MWER | 141 | 2450 | 5.8
C1 w/o MWER | 141 | 3550 | 5.9
C2 w/o MWER | 137 | 4200 | 5.8
w MWER | 5.6
Table 2: Quality and Latency comparisons of different RNN-T models on the Voice Search test set. Exp. | WER | Endpointer Latency | Partial Latency | Prefetch Latency
---|---|---|---|---
(%) | EP50 (ms) | EP90 (ms) | PR50 (ms) | PR90 (ms) | PF50 (ms) | PF90 (ms) | PFR
B0 Conventional [1] | 6.6 | 460 | 870 | -150 | 60 | 90 | 190 | 1.48
B1 LSTM RNN-T[1] | 6.0 | 310 | 710 | 170 | 310 | 170 | 320 | 1.80
B2 B1 + Constrained Alignment | 6.9 | 230 | 560 | -40 | 80 | 100 | 200 | 1.29
B3 B1 + FastEmit | 6.2 | 330 | 650 | -10 | 180 | 80 | 210 | 1.47
C2 Conformer RNN-T | 5.6 | 260 | 590 | 150 | 290 | 220 | 350 | 1.65
C3 C2 + FastEmit | 5.8 | 290 | 660 | -110 | 90 | 70 | 210 | 1.29
C4 C3 + E2E Prefetch | 6.0 | 290 | 660 | -110 | 90 | -50 | 110 | 1.86
T1 Two-pass LAS Rescoring | 5.3 | 290 | 660 | -100 | 140 | 80 | 210 | 1.30
T2 Single-pass Causal | 6.0 | 290 | 660 | -90 | 120 | -20 | 130 | 1.90
T2 Two-pass | 5.4 | 290 | 660 | -80 | 140 | 0 | 140 | 1.84
T2 Single-pass Non-causal | 4.8 | - | - | - | - | - | - | -

###  5.1 Quality Improvement with Conformer

First we experimented with Conformer encoders for quality improvement. Table 1
presents the comparison on the Voice Search testset for RNN-T models with
different encoders. The RNN-T model with an LSTM encoder serves as our
baseline (B1), which has been shown to give a better quality and latency
performance than conventional hybrid ASR systems [1]. Limiting the Conformer
encoder presented in [4] to use only left-context for streaming applications,
system C0 shows significant WER degradation due to the biased batch
normalization statistics in our multi-domain task. This can be resolved by
randomly sampling utterances (i.e. no bucketing) instead of aggregating
utterances with similar lengths into each batch (C0 No bucketing in Table 1)
with the cost of large training speed reduction due to the lower computation
utilization per batch. By replacing the batch normalization with the group
normalization, system C1 addressed the issue with similar WER gains but less
training speed regression. Further swapping the order of the convolution
module and the self-attention module, system C2 achieves both WER gains and
training speedups. Comparing to B1, C2 yields a total of 7% relative WER
reduction and 35% speedup. Note for B1 we did not see quality gains with more
LSTM encoder layers.

###  5.2 Latency Improvement

In previous work [1, 2], we focused on improving RNN-T models’ endpointer
latency and have shown that E2E models have a better WER and endpointer
latency tradeoff than conventional hybrid ASR systems. However, even with that
we still observe large total latency in E2E based streaming applications. This
is then tracked down to the fact that the E2E systems tend to delay the
partial hypotheses towards the end of the utterance where the endpointer
fires, leaving less space for prefetching to reduce network latency (B0 vs. B1
in Table 2).

####  5.2.1 Reducing Partial Latency with Constrained Alignment

In this experiment, we explored constraining the RNN-T prediction with
alignment information following [20]. When applied to B1, the use of
constrained alignment achieves 210ms and 230ms latency reduction for PR50 and
PR90 respectively with the cost of a quality degradation from 6.0% to 6.9% (B2
vs. B1 in Table 2). Negative latency suggests the model is capable of
hypothesizing the whole query even before seeing the end of the speech.

####  5.2.2 Reducing Partial Latency with FastEmit

We then applied FastEmit to the same baseline system B1. From Table 2, B1 with
FastEmit (B3) gives 180ms and 130ms reduction for PR50 and PR90 with much less
WER regression (6.0% to 6.2%). Additionally, no alignment is needed during B3
training. To address the quality degradation, we used FastEmit on a better E2E
model - C2 from Table 1. Although FastEmit still brings in 0.2% absolute WER
regression, the system ‘C2+FastEmit’ (i.e. C3) has a WER of 5.8% with a -110ms
PR50 and 90ms PR90, which is better both in quality and latency than B1.

####  5.2.3 Reducing Prefetch Latency with E2E Prefetching

Next, we explored the use of E2E prefetch to minimize the gap between partial
and prefetch as prefetch latency is the one that is reflected in the total
latency. E2E prefetch can only be used in E2E systems and is not applicable to
conventional models, where silence-based prefetching is used. From Table 2,
adding E2E prefetch to C3 (i.e. C4) brings down the prefetch latency to -50ms
for PF50 and 110ms for PF90 with a PFR of 1.86. The WER increased to 6.0%
which is still comparable to B1.

###  5.3 Two-Pass with Cascaded Encoders

To further address the quality degradation due to latency related optimization
in the 1st-pass, a LAS-based two-pass rescoring model [1, 18] was firstly
applied, which is referred to as T1 in Table 2. It reduces the WER to 5.3%
with comparable endpointer and partial latency metrics; however, the E2E
Prefetch is less effective as it only applies to RNN-T EP model. We then train
a Cascaded Encoders model (T2 in Table 2). In the causal only mode, ‘T2
Single-pass Casual’ performs similar to C4; while with the additional non-
causal 2nd-pass at EOQ, ‘T2 Two-pass’ achieves a similar WER as T1 but with
much better prefetch latency benefiting from the E2E prefetching. In non-
streaming application, we can use ‘T2 Single-pass Non-causal’ to get an even
better WER of 4.8%. The computation cost of T2 is much higher than T1 due to
the use of beam-search instead of rescoring. We will further investigate
techniques in reducing it in future work. With the two-pass Cascaded Encoders
model T2, we can achieve a much better WER and latency (including endpointer,
partial and prefetch) comparing to the previous published best E2E system B1.

###  5.4 Comparison to Conventional Model

We also included the performance of a conventional hybrid ASR model (B0 in
Table 2) that consists of a low-frame-rate acoustic model emitting context-
dependent phonemes [39] (0.1GB), a 764k-word pronunciation model (2.2GB), a
1st-pass 5-gram language-model (4.9GB). RNN-T models (except for B2) have a
better quality and endpointer latency comparing to the conventional system B0,
reconfirming the finding in [1]. With FastEmit, we can achieve a better
quality with very close partial latency (C3). The -150ms median partial
latency (PR50) for B0 mainly comes from the stronger LM trained on large
amount of text only data that can potentially hypothesis the full query even
before the speech finishes, which E2E models currently lack of. However, even
with slightly worse partial latency, E2E systems can still achieve much better
prefetch latency with the help of E2E prefetching (C4), which directly affects
the total latency of streaming ASRs. With an additional non-casual 2nd-pass
(T2 Two-pass), we can further bring down the WER to 5.4% while still having a
faster prefetch than B0.

##  6 Conclusions

In this work, we further improve the quality and latency of the RNN-T based
E2E models for streaming applications. We replaced the encoder LSTM layers
with the recently developed _Conformer_ layers [4] for quality and adopted a
simple yet effective latency regularization technique, _FastEmit_ [3], which
is generic to any transducer based models. Furthermore, we employed a
_Cascaded Encoders_ [5] RNN-T model that shares the same RNN-T decoder for the
two passes. It gives us a system that is better and faster than the previous
best E2E system [1] and surpassing the conventional model in quality and all
latency metrics.

## References

  * [1] T. N. Sainath, Y. He, B. Li, et al.,  “A Streaming On-Device End-To-End Model Surpassing Server-Side Conventional Model Quality and Latency,”  in Proc. ICASSP, 2020.
  * [2] B. Li, S.-Y. Chang, T. N. Sainath, et al.,  “Towards fast and accurate streaming end-to-end ASR,”  in Proc. ICASSP, 2020.
  * [3] J. Yu, C.-C. Chiu, B. Li, et al.,  “FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularization,”  in Proc. ICASSP, 2021.
  * [4] A. Gulati, J. Qin, C.-C. Chiu, et al.,  “Conformer: Convolution-augmented Transformer for Speech Recognition,”  in Proc. Interspeech, 2020.
  * [5] A. Narayanan, T. N. Sainath, R. Pang, et al.,  “Cascaded encoders for unifying streaming and non-streaming ASR,”  in Proc. ICASSP, 2021.
  * [6] J. Li, Y. Wu, Y. Gaur, et al.,  “On the Comparison of Popular End-to-End Models for Large Scale Speech Recognition,”  arXiv:2005.14327, 2020.
  * [7] Y. He, T. N. Sainath, R. Prabhavalkar, et al.,  “Streaming End-to-end Speech Recognition For Mobile Devices,”  in Proc. ICASSP, 2019.
  * [8] C.-C. Chiu, T. N. Sainath, Y. Wu, et al.,  “State-of-the-art Speech Recognition With Sequence-to-Sequence Models,”  in Proc. ICASSP, 2018.
  * [9] A. Graves,  “Sequence Transduction with Recurrent Neural Networks,”  CoRR, vol. abs/1211.3711, 2012.
  * [10] W. Chan, N. Jaitly, Q. V. Le, and O. Vinyals,  “Listen, attend and spell,”  CoRR, vol. abs/1508.01211, 2015.
  * [11] S. Kim, T. Hori, and S. Watanabe,  “Joint CTC-attention based end-to-end speech recognition using multi-task learning,”  in Proc. ICASSP, 2017.
  * [12] N. Jaitly, D. Sussillo, Q. V. Le, et al.,  “An Online Sequence-to-sequence Model Using Partial Conditioning,”  in Proc. NIPS, 2016.
  * [13] E. Tsunoo, Y. Kashiwagi, and S. Watanabe,  “Streaming Transformer ASR with Blockwise Synchronous Inference,”  arXiv:2006.14941, 2020.
  * [14] C. Raffel, M. Luong, P. J. Liu, et al.,  “Online and Linear-Time Attention by Enforcing Monotonic Alignments,”  in Proc. ICML, 2017.
  * [15] R. Fan, P. Zhou, W. Chen, et al.,  “An online attention-based model for speech recognition,”  arXiv:1811.05247, 2018.
  * [16] N. Moritz, T. Hori, and J. Le Roux,  “Triggered attention for end-to-end speech recognition,”  in Proc. ICASSP, 2019.
  * [17] C. Wang, Y. Wu, S. Liu, et al.,  “Low Latency End-to-End Streaming Speech Recognition with a Scout Network,”  arXiv:2003.10369, 2020.
  * [18] T. N. Sainath, R. Pang, D. Rybach, et al.,  “Two-Pass End-to-End Speech Recognition,”  in Proc. Interspeech, 2019.
  * [19] D. Zhao, T. N. Sainath, D. Rybach, et al.,  “Shallow-Fusion End-to-End Contextual Biasing,”  in Proc. Interspeech, 2019.
  * [20] T. N. Sainath, R. Pang, D. Rybach, et al.,  “Emitting word timings with end-to-end models,”  in Proc. Interspeech, 2020.
  * [21] H. Inaguma, Y. Gaur, L. Lu, et al.,  “Minimum latency training strategies for streaming sequence-to-sequence ASR,”  in Proc. ICASSP, 2020.
  * [22] C.-C. Chiu, W. Han, Y. Zhang, et al.,  “A comparison of end-to-end models for long-form speech recognition,”  in Proc. ASRU, 2019.
  * [23] Y. Wu and K. He,  “Group normalization,”  in Proc. ECCV, 2018.
  * [24] Z. Dai, Z. Yang, Y. Yang, et al.,  “Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context,”  in ACL, 2019.
  * [25] Y. Wang, A. Mohamed, D. Le, et al.,  “Transformer-based acoustic modeling for hybrid speech recognition,”  arXiv:1910.09799, 2019.
  * [26] A. Narayanan, R. Prabhavalkar, C.-C. Chiu, et al.,  “Recognizing Long-Form Speech Using Streaming End-to-End Models,”  in Proc. ASRU, 2019.
  * [27] L.A. Barroso, J. Dean, and U. Hölzle,  “Web Search for a Planet: The Google Cluster Architecture,”  IEEE Micro, vol. 23, pp. 22–28, 2003.
  * [28] S. Jonassen, B. B. Cambazoglu, and F. Silvestri,  “Prefetching query results and its impact on search engines,”  in Proc. SIGIR, 2012.
  * [29] S.-Y. Chang, B. Li, D. Rybach, et al.,  “Low latency speech recognition using end-to-end prefetching,”  in Proc. Interspeech, 2020.
  * [30] A. Graves, S. Fernandez, F. Gomez, and J. Schmidhuber,  “Connectionist Temporal Classification: Labeling Unsegmented Sequenece Data with Recurrent Neural Networks,”  in Proc. ICML, 2006.
  * [31] M. Schuster and K. Nakajima,  “Japanese and Korean voice search,”  in Proc. ICASSP, 2012.
  * [32] F. Biadsy, M. Ghodsi, and D. Caseiro,  “Effectively Building Tera Scale MaxEnt Language Models Incorporating Non-Linguistic Signals,”  in Proc. Interspeech, 2017.
  * [33] M. Abadi et al.,  “Tensorflow: Large-scale machine learning on heterogeneous distributed systems,” Available online: http://download.tensorflow.org/paper/whitepaper2015.pdf, 2015.
  * [34] J. Shen, P. Nguyen, Y. Wu, et al.,  “Lingvo: a modular and scalable framework for sequence-to-sequence modeling,”  arXiv:2005.08100, 2019.
  * [35] D. P. Kingma and J. Ba,  “Adam: A method for stochastic optimization,”  in Proc. ICLR, 2015.
  * [36] H. Liao, E. McDermott, and A. Senior,  “Large Scale Deep Neural Network Acoustic Modeling with Semi-supervised Training Data for YouTube Video Transcription,”  in Proc. ASRU, 2013.
  * [37] C. Kim, A. Misra, K. Chin, et al.,  “Generation of Large-Scale Simulated Utterances in Virtual Rooms to Train Deep-Neural Networks for Far-Field Speech Recognition in Google Home,”  in Proc. Interspeech, 2017.
  * [38] J. Li, D. Yu, J. Huang, and Y. Gong,  “Improving Wideband Speech Rcognition using Mixed-bandwidth Training Data in CD-DNN-HMM,”  in Proc. SLT, 2012.
  * [39] G. Pundak and T. N. Sainath,  “Lower frame rate neural network acoustic models,”  in Proc. Interspeech, 2016.


# DLGNet-Task: An End-to-end Neural Network Framework for Modeling Multi-turn
Multi-domain Task-Oriented Dialogue

Oluwatobi O. Olabiyi, 1 Prarthana Bhattarai, 2 C. Bayan Bruss, 2 Zachary Kulis
1
1 Capital One - Conversation Research, McLean, VA
2 Capital One - Center for Machine Learning, McLean, VA
{oluwatobi.olabiyi, prarthana.bhattari, bayan.bruss,
<EMAIL_ADDRESS>
###### Abstract

Task oriented dialogue (TOD) requires the complex interleaving of a number of
individually controllable components with strong guarantees for explainability
and verifiability. This has made it difficult to adopt the multi-turn multi-
domain dialogue generation capabilities of streamlined end-to-end open-domain
dialogue systems. In this paper, we present a new framework, DLGNet-Task, a
unified task-oriented dialogue system which employs autoregressive transformer
networks such as DLGNet and GPT-2/3 to complete user tasks in multi-turn
multi-domain conversations. Our framework enjoys the controllable, verifiable,
and explainable outputs of modular approaches, and the low development,
deployment and maintenance cost of end-to-end systems. Treating open-domain
system components as additional TOD system modules allows DLGNet-Task to learn
the joint distribution of the inputs and outputs of all the functional blocks
of existing modular approaches such as, natural language understanding (NLU),
state tracking, action policy, as well as natural language generation (NLG).
Rather than training the modules individually, as is common in real-world
systems, we trained them jointly with appropriate module separations. When
evaluated on the MultiWOZ2.1 dataset, DLGNet-Task shows comparable performance
to the existing state-of-the-art approaches. Furthermore, using DLGNet-Task in
conversational AI systems reduces the level of effort required for developing,
deploying, and maintaining intelligent assistants at scale.

Figure 1: DLGNet-Task framework - The framework captures the end-to-end
relationships of all the components of a multi-turn multi-domain task-oriented
dialogue system.

Figure 2: The graph of the dialogue flow modeled by DLGNet-Task - DLGNet-Task
models the joint distribution of multiple random variables that exist in both
open-domain and task-oriented dialogue.

## Introduction

The desire for human-like interfaces to technical systems, as evidenced by
growing use of intelligent assistants, belies the need for conversational AI
systems that can accomplish a wide range of tasks, such as booking
restaurants, trains, and flights, IT help desk and accessing financial
accounts and transaction records. The wide range of tasks have necessitated
the need for a flexible and scalable dialogue system that can support a
variety of use cases with minimal development and maintenance effort. Existing
dialogue systems are broken into two major categories, open-domain dialogue
systems, which focus on non-task related conversations, and task-oriented
dialogue systems, which focus on user task completion. A typical open-domain
system uses an end-to-end neural architecture often trained with input and
output utterances from human-to-human conversations (Sutskever, Vinyals, and
Le 2014; Serban et al. 2016, 2017; Olabiyi et al. 2018, 2019; Olabiyi and
Mueller 2019; Zhang et al. 2019). While open-domain systems are optimized for
engaging in human-like conversation, they lack any inherent ability to
interface with any other systems on behalf of their conversation partner.
Whereas, a typical task-oriented dialogue system seeks to understand human
intents and execute them. This is done by adopting a modularized pipeline
architecture with three modules that are sequentially connected as shown in
Fig. 1. A natural language understanding (NLU) module that recognizes user
intents and extract useful entity information (Kim, Lee, and Stratos 2017; Lee
et al. 2019). The dialogue management (DM) module contains two submodules, the
dialogue state tracker (DST) and the dialogue action policy (POL) modules. The
DST module tracks the mapping of entities to slots that are relevant or
required for completing user tasks (Williams et al. 2013). The POL module
decides which actions to execute via the API. Finally, the natural language
generation (NLG) module generates the user response based on the user aspects
of the system actions (Wen et al. 2015). In some cases, multiple modules are
combined together, e.g. systems with a composite NLU and DST module (Ramadan,
Budzianowski, and Gasic 2018; Lee, Lee, and Kim 2019), and systems with a
composite POL and NLG module that maps previous utterances and dialogue states
to the system response (Budzianowski et al. 2018; Pei, Ren, and de Rijke 2019;
Chen et al. 2019; Mehri, Srinivasan, and Eskenazi 2019; Zhao, Xie, and
Eskenazi 2019).

Despite research advances in modular neural approaches, they are hardly used
in practice. Industrial dialogue systems, though modularized, still use
expensive expert driven rule-based heuristics implemented with several lines
of codes and hand-crafted templates, and therefore difficult to scale as the
number of use cases grows. More recently, there has been a renewed effort to
apply a single end-to-end neural architecture (Budzianowski and Vulic 2019;
Ham et al. 2020; Peng et al. 2020; Hosseini-Asl et al. 2020) to model task-
oriented dialogue with the use of autoregressive transformer architecture
(Radford et al. 2019). This has led to the reformulation of dialogue system
design as a text generation or sequence modeling task. While some of these
efforts have obtained state-of-the-art performance on publicly available task-
oriented dialogue datasets, there is still room for improvement, especially in
the areas of generality and practicality. First, their problem formulation
fails to reconcile open-domain and task-oriented dialogue in the same model
architecture. Also, in many cases, they do not address the complexity of the
action policy especially towards the back-end API system. Finally, they don’t
fully incorporate the control, verification and explanation capabilities that
make modularized approaches attractive.

To resolve these shortcomings, we propose DLGNet-Task, an end-to-end neural
network that simultaneously handles both open-domain and task-oriented
dialogue, in such a way that the model outputs are controllable, verifiable,
and explainable at the module level. This system is compatible with both data
driven and expert driven rule-based approaches. That is, our approach is
simultaneously modular and end-to-end, and can be a drop-in replacement for
traditional modular task-oriented dialogue (TOD) systems. To the best of our
knowledge, this is the most expressive approach to date in achieving this
objective. In summary, we are able to model the individual behavior of NLU, DM
and NLG components with a single neural network model trained end-to-end.
Still, the model is flexible enough to allow individual modules to be
separately trained and validated in line with the traditional TOD system.

## DLGNet-Task System Description

The goal of DLGNet-Task framework is to support the modeling of both open-
domain and task-oriented dialogue. To achieve this, we follow a different
modularized pipeline architecture from the ones in recent existing work
(Budzianowski and Vulic 2019; Ham et al. 2020; Peng et al. 2020; Hosseini-Asl
et al. 2020). DLGNet-Task system consists of (i) the GPT-2 model finetuned
using DLGNet objective on a modified MultiWOZ2.1 dataset, (ii) a fine-grained
representation of the action policy for calling the backend API and responding
to the user, and (iii) intermediate knowledge that allows for controlling,
verifying and explaining system behavior. Fig. 1 shows the functional blocks
of the DLGNet-Task framework, which can be applied to any TOD dataset. Fig. 2
shows the graph of the information flow between the functional blocks in Fig.
1. Because it is trained in a single network the DLGNet-Task model learns the
joint distribution of the functional blocks based on the information flow
graph in Fig. 2. This gives it additional flexibility across tasks and
objectives. For example, the direct connection between the utterance node
$U_{t}$ and the response node $R_{t}$ allows the model to operate in an open-
domain context, while other connections allow it to be task oriented.

### Framework Components

DLGNet-Task is broken into three modules as depicted in Fig. 1, the Natural
Language Understanding (NLU) module, the dialogue management module and the
Natural Language Generation (NLG) module. Below we describe the components of
each of these modules in more detail.

#### NLU Module

The NLU module contains components for identifying intents and entities in the
incoming dialogue. It is worth noting that although our NLU model has access
to multi-turn context, it only makes predictions relevant to the current turn
of the conversation.

Table 1: Example dialogue including ground truth responses and DLGNet-Task generated responses User Prompt |  Ground Truth Response |  DLGNet-Task Response
---|---|---
I am looking for places to go in the centre of town. |  I have 44 matches for central area, I can narrow it down by type or entrance prices. |  Okay. My computer shows forty-four attractions in the centre of town. Can you give me some more details so we can narrow the search?
How about a park? |  that should be Cambridge university botanic gardens located in bateman street. can i give you the phone number? |  We have cambridge university botanic gardens. Would you like more information about them?
Yes, I’ll need the number along with the address and postal code. |  The phone number is 01223336265. |  The entrance fee is 4 pounds, phone number is 01223336265 and postcode is cb21jf, and the address is bateman street.
I also need the address please. |  The address is Bateman Street in the centre area of town. |  Postcode is cb21jf, address is bateman street, and you can reach them on 01223336265.
Can I get the postcode too please? |  The postcode is cb21jf. Do you need anything else? |  I apologize, the postcode is cb21jf, can I assist with anything else?
I’m also looking for a place to stay in the same area as Cambridge university botanic gardens |  Would you prefer a guesthouse or a hotel? |  We have 5 places to stay all in the city center. Would you like to hear about some of our other options?
A guesthouse please and with a star rating of zero. |  The only place available is the El Shaddai. Would you like for me to book a room? |  Cityroomz is a moderately priced guesthouse. Would you like me to book a room for you?
Actually could you get me the postcode, their price range, and whether they have free parking? |  The postcode is cb11eg they offer free parking and is cheaply priced. |  El Shaddai is in the cheap price range, and their postcode is cb11eg. Is there anything else I can assist you with today?
Thank you very much. that should be it. |  I can also recommend restaurants and arrange transportation. Do you need help with these? |  Thank you for contacting us and have a nice day.
No, thanks. I have everything I need. |  Ok! Have a great day! |  Great. Glad to help.
Table 2: Example ground truth component values and DLGNet-Task predictions |  Ground Truth |  Model Output
---|---|---
User |  Yes, Ill need the number along with the address and postal code. |  Yes, Ill need the number along with the address and postal code.
Domains |  attraction |  attraction
Slots Search |  domain:attraction, area:centre, name:?, type:park |  domain:attraction, area:centre, name:?, type:park
API Actions |  domain:attraction, search-execute |  domain:attraction, search-execute
Results |  domain:attraction, [’Choice’, 1], [’Sample’, [{’id’: ’15’, ’address’: ’bateman street’, ’area’: ’centre’, ’entrance’: ’4 pounds’, ’name’: ’cambridge university botanic gardens’, ’phone’: ’01223336265’, ’postcode’: ’cb21jf’, ’pricerange’: ’moderate’, ’openhours’: ’it opens from 10:00 a.m. to 6:00 p.m. from april to september, from 10:00 a.m. to 5:00 p.m. in february march and october, and from 10:00 a.m. to 4:00 p.m. from november to january’, ’type’: ’park’}]] |  domain:attraction, [’Choice’, 1], [’Sample’, [{’id’: ’15’, ’address’: ’bateman street’, ’area’: ’centre’, ’entrance’: ’4 pounds’, ’name’: ’cambridge university botanic gardens’, ’phone’: ’01223336265’, ’postcode’: ’cb21jf’, ’pricerange’: ’moderate’, ’openhours’: ’it opens from 10:00 a.m. to 6:00 p.m. from april to september, from 10:00 a.m. to 5:00 p.m. in february march and october, and from 10:00 a.m. to 4:00 p.m. from november to january’, ’type’: ’park’}]]
DLG Actions |  domain:attraction, search-noerror-inform:[[’Phone’, ’01223336265’]] |  domain:attraction, search-noerror-inform:[[’Addr’, ’bateman street’], [’Phone’, ’01223336265’], [’Post’, ’cb21jf’], [’Fee’, ’4 pounds’]]
Delexicalized Output |  the phone number is [attraction_phone] . |  the entrance fee is [value_count] pounds , phone number is [attraction_phone] and postcode is [attraction_postcode] , and the address is [attraction_address] .
System Output |  The phone number is 01223336265. |  The entrance fee is 4 pounds, phone number is 01223336265 and postcode is cb21jf, and the address is bateman street.

_Intent Recognition:_ The intent recognition maps the customer utterance to a
label. For example, in Table A1, “ I would like to book a reservation at Caffe
Uno” would be mapped to the label request_booking. One of the limitations of
the existing NLU module for natural multi-turn dialogue is the requirement to
label every utterance within a conversation even when it is not appropriate.
For example, mapping “I am looking for information in Cambridge.” to a label
might be unnecessary. In this case, it is more beneficial to focus on giving
an appropriate response rather than the intent recognition.

_Entity Recognition:_ In our framework, we separate named entity recognition
(NER) from slot filling in order to be compatible with existing industrial
modularized pipeline architecture. Our NER component assigns value to an
entity without identifying the domain. Not predicting the domain at this stage
makes the NER compatible with both conversational on non-conversational
datasets.

#### Dialogue Management Module

The dialogue management module has two broad functions, state tracking and
action policy. Each of these functions are comprised of four components. State
tracking maintains All Entities, All Domains, Domains, and Domain-Plan Slots,
while the Action Policy maintains Plans, API Actions, API Results, and
Dialogue Actions.

_State Tracking_

The state tracking handles the multi-turn understanding by mapping the context
to a well defined representation of the dialogue system ontology e.i., DOMAIN,
PLAN, SLOT, ENTITY, VALUES, CONSTRAINTS. The state tracking identifies all the
entities and domains in the dialogue history, predicts the active domain, and
fill the slots of the active domain using the entity information. This
arrangement allows for the verification of slot values with the options and
constraints provided in the system ontology.

_All Entities:_ We introduce an all entities information node that adds the
currently recognized entities (if any) to the previous all entity state i.e.,
$AE_{t}=AE_{t-1}+E_{t}$. With this, we can easily verify or replace the
generated all entities at any conversation turn.

_All Domains:_ We also introduce an all domains information node that
maintains a running list of the domains. This node adds the domain $D_{t}$ in
the current turn (if any) to the previously recognized all domains state
$AD_{t-1}$, i.e., $AD_{t}=AD_{t-1}+D_{t}$. With this, we can easily verify or
replace the generated all domains values at any conversation turn.

_Domains:_ Similar to Ham et al. (2020), we predict active domains at the
current turn, which is a subset of the all domains node above. The MultiWOZ
dataset consists of five domains, hotel, train, restaurant, hotel, and
attraction. We support both single and multi-domain interactions over multiple
turns of conversation.

_Domain-Plan Slots:_ The slots are divided into three, informable, requestable
and book slots. Informable slots represent user constraints and requestable
slots hold additional information that the user wants to obtain. Book slots
are used to reserve a place recommended by the system or selected by the user
(Ham et al. 2020). In order to generalize the slot types to new use cases, we
map these categories to their functions (plans). That is, we map informable
and book slots to search and booking slots respectively, indicating what the
slots are being used for. The requestable slots remain the slots to hold
additional information that the user wants to obtain. We predict the plan
slots for each predicted domains.

In order to avoid an open ended generation of the dialogue state as in
existing work (Budzianowski and Vulic 2019; Ham et al. 2020; Peng et al. 2020;
Hosseini-Asl et al. 2020) and improve generalization to new domains, we
provide the model with a list of slots for each plan type, and indicate if
each slot is filled or not. During inference, with a new domain or plan type,
we only need to provide an appropriate slot list, and the model can easily
fill it based on the utterance and entity information.

_Action Policy_

The action policy predicts the sub-domain plans relevant to the current turn
of the conversation. It also predicts the API actions, and conditioned on the
results of the API action, predicts the dialogue act.

_Plans:_ For each domain, we predict the relevant plans for the current
conversation turn. For the MultiWOZ dataset, we have two main plans with slots
(search and booking), and others without slots (welcome, greet, bye, and
reqmore).

_API Actions:_ For each plan, we determine the appropriate API actions. In the
MultiWOZ dataset, we observe two actions, execute (for both search and booking
plans), and retrieve (for booking plan). Therefore, our API action uses the
format [PLAN-ACTION] for each domain as shown in the example in Table A1.

_API Results:_ For each API action, we call the plan’s API with filled plan
type slots. For the MultiWOZ dataset, in the case of search, we return the
number for matches, as well as top k results. In our ablation studies, we
considered $k=\\{1,3,5\\}$. For booking, we use the results from the dialog-
act and/or the requestable slots in the metadata.

_Dialogue Actions:_ For each plan, we select appropriate action(s) among
inform, request, recommend, select, book, offerbook, offerbooked. We also
identify three status codes, nobook (booking error) and nooffer (search error)
and noerror (otherwise). The errors were originally identified as actions in
the MultiWOZ dataset, but we assign the corresponding actions as inform in our
modification. Therefore, our dialogue action uses the format [PLAN-STATUSCODE-
ACTION] for each domain with appropriate slot information as shown in the
example in Table A1.

Also, we noticed that booking a train requires an offer confirmation before
the system can make the booking. This behavior may be required in commercial
applications where customer confirmation may be legally required before
creating an order or executing a particular plan. To provide a trigger signal
to the model, we added a ‘confirm’ slot to the booking slots of train domain
(see Table A1). Filling of this slot consequently triggers the ‘offerbook’
action. Since this data modification is an after thought, the entity
recognition and slot filling are based on heuristics, which is imperfect and
introduces noise into the training data. However, this modification is
necessary to fully describe the DLGNet-Task framework.

#### NLG Module

DLGNet-Task handles both template and system response generation. In order to
do this, we delexicalized all the values of requestable slots (reference
number, name, postcode, phone number, address) as [DOMAIN_SLOTNAME] (e.g.
[train_reference] for train’s booking reference that appear in the dataset)
similar to Ham et al. (2020). Unlike in the existing work, where post
processing routine is used to string-replace the delexicalized token later by
the real information from the API results, the DLGNet-Task model directly
generates the final system response. This makes DLGNet-Task a truly end-to-end
neural TOD system.

Since DLGNet-Task model is a word-token sequence generation model, the
traditional decoding approach is to explore sequence decoding strategies such
as greedy decoding, beam-search decoding, top_k sampling and top_p sampling
strategies (Holtzman et al. 2019; Olabiyi and Mueller 2019). However, TOD
systems contains both natural language and several ontology-driven key-value
pairs, such as graph node-value, intent-value, entity-value, slot-entity,
domain-value, plan-value, plan-API action, plan-dialogue action pairs. The
ontology-driven key-value pairs provide opportunities for discrimination,
since some of the key and possible values may be known apriori from the system
ontology. Note that the ontology itself is not used during training, it is
only used here to ground value generation or selection during inference. For
example, given the triples $(C,K_{i},\\{V_{i}\\}_{j}^{J})$ of context $C$, key
$K$ and possible values $V$, we can estimate the likelihood of each possible
value $V_{i}^{j}$, i.e.,

$\displaystyle
P_{\theta}\big{(}V_{i}^{j}|K_{i},C\big{)}=DLGNet([C,DL_{key},K_{i},DL_{value},V_{i}^{j}])$
(1)

where $DL$’s are delimiter tokens.

The likelihood scores can be used to rank possible values during inference,
which would also improve generalization to new key-value pairs. Using the
likelihood score in Eq. (1), a normalized conditional distribution over the
value options can be estimated, i.e.,

$\displaystyle
P\big{(}V_{i}^{j}|K_{i},C\big{)}=\frac{\exp\big{(}\frac{1}{T_{i}}\log
P_{\theta}(V_{i}^{j}|K_{i},C)\big{)}}{\sum_{j}^{J}\exp\big{(}\frac{1}{T_{i}}\log
P_{\theta}(V_{i}^{j}|K_{i},C)\big{)}}$ (2)

where the hyperparamter $T_{i}\in(0,1]$ is the decoding temperature.

### Input Representation

DLGNet-Task system is implemented with the use of autoregressive transformer
networks, such as GPT-2 and DLGNet. To do this, we need to convert the
dialogue data to word tokens using the information flow graph. In the case of
the MultiWOZ dataset, ‘metadata’ and ‘dialog-act’ correspond to the current
dialogue state, and the current system actions. The dataset also contains the
user utterance and system response. We introduce a delimiter token for each
functional block used in each conversation turn. We also introduce two special
delimiter tokens, $<$turn_sep$>$ and $<$conversation_sep$>$ for turn and
conversation separation respectively based on the DLGNet framework (Olabiyi
and Mueller 2019).

### Dialogue Information Flow

In order to obtain the processed data as shown in Table A1, we adopted a
dialogue flow process based on the information flow graph in Fig. 2. The model
inference follows the same process after training. The procedure for
processing the training data and model inference using the dialogue flow is
described below:

Given the current turn utterance $U_{t}$, and the information from the
previous dialogue turn $M_{t-1}=\\{U_{t-1},I_{t-1},E_{t-1},\\\
AE_{t-1},AD_{t-1},D_{t-1},S_{t-1},P_{t-1},AA_{t-1},AR_{t-1},DA_{t-1},\\\
T_{t-1},R_{t-1}\\}$:

  * •

NLU Functional Blocks

    1. 1.

Predict intent $I_{t}$ if applicable else skip.

    2. 2.

Predict entities $E_{t}$ if applicable else skip.

  * •

DST Functional Blocks
Given the NLU predictions:

    1. 1.

Predict all entities $AE_{t}$ if applicable else skip.

    2. 2.

Predict all domains $AD_{t}$ if applicable else skip.

    3. 3.

Predict active domains $D_{t}$ if applicable else skip.

    4. 4.

Given the domain, predict plan slots $S_{t}$ if applicable else skip.

  * •

POL Functional Blocks
Given the NLU and DST predictions:

    1. 1.

Predict plans $P_{t}$ if applicable else skip.

    2. 2.

Given the plan, predict API actions $AA_{t}$ if applicable else skip.

    3. 3.

Given the API actions, obtain API results $AR_{t}$ if applicable else skip.

    4. 4.

Given the plan, predict dialogue actions $DA_{t}$ if applicable else skip.

  * •

NLG Functional Blocks
Given NLU, DST and POL predictions:

    1. 1.

Predict delexicalized template $T_{t}$ if applicable else skip.

    2. 2.

Predict system response $R_{t}$.

It is worth mentioning that the introduction of NLU functional blocks, and all
entities, domain, plan, and template functional blocks helps to control,
verify and explain the system response.

### Training Objective

For training DLGNet-Task model, we finetune GPT-2 pretrained model using
modified MultiWOZ dataset based on the input representation described above.
In our experiments, due to the use of turn and conversation separations, we
only use the objective of autoregressive language modeling for joint
distribution modeling with random informative padding (Radford et al. 2019;
Olabiyi and Mueller 2019), i.e.,

$\displaystyle L_{DLGNet}(x_{1},x_{2},\cdots,x_{n}\big{)}=\sum_{i}\log
P_{\theta}\big{(}x_{i}|x_{1},\cdots,x_{i-1}\big{)}$ (3)

### DLGNet-Task Training

using the Adaptive Moment Estimation (Adam) stochastic gradient descent
algorithm with a learning rate of 0.0001 with a maximum sequence length is
1024. Due to GPU memory limitations, we use a batch size of 2 and accumulate
gradients over 5 iterations, making the effective batch size 10. The models
are trained until the training perplexity on the dialogue datasets reaches a
steady state.

Table 3: Context-result-to-response evaluation on MultiWOZ Model |  $\uparrow$ Inform |  $\uparrow$ Success |  $\uparrow$ BLEU |  $\uparrow$ Combined
---|---|---|---|---
Baseline (Budzianowski et al. 2018) | 71.29 | 60.94 | 18.80 | 84.93
GPT-2 (Budzianowski and Vulic 2019) | 70.96 | 61.36 | 19.05 | 85.21
Structured Fusion (Mehri, Srinivasan, and Eskenazi 2019) | 82.70 | 72.10 | 16.34 | 93.74
SOLOIST (Peng et al. 2020) | 89.60 | 79.30 | 18.03 | 102.49
DSTC8 Track 1 Winner (Ham et al. 2020) | 77.00 | 69.70 | 16.11 | 89.46
DLGNet-Task (Ours) | 75.15 | 57.31 | 18.34 | 87.52
Table 4: End-to-end evaluation on MultiWOZ Model |  $\uparrow$ Inform |  $\uparrow$ Success |  $\uparrow$ BLEU |  $\uparrow$ Combined
---|---|---|---|---
Structured Fusion (Mehri, Srinivasan, and Eskenazi 2019) | 73.80 | 58.60 | 16.90 | 83.10
SOLOIST (Peng et al. 2020) | 85.50 | 72.90 | 16.54 | 95.74
SimpleTOD (Hosseini-Asl et al. 2020) | 84.40 | 70.10 | 15.01 | 92.26
DSTC8 Track 1 Winner (Ham et al. 2020) | 73.00 | 62.40 | 16.00 | 83.50
DLGNet-Task (Ours) | 72.65 | 56.81 | 15.40 | 80.13

## Experiments

In this section, we describe how the DLGNet-Task models, trained using the
proposed framework performs on the Multiwoz2.1 dataset under different data
and model inference conditions.

### Evaluation Metrics

Following Budzianowski et al. (2018), we evaluate the DGLNet-Task models based
on the Inform, Success, and BLEU scores. Inform measures if the system
provides a correct entity (inform rate). Success measures the exact matching
of answering all the requested information (success rate) and if the answered
information matches users’ goal . BLEU evaluates how natural the generated
utterance is compared with human readers. A combined score (Combined) is also
reported using Combined = (Inform + Success) x 0.5 + BLEU as an overall
quality measure, as suggested in Budzianowski et al. (2018).

### Model Inference Variants

For the purpose of evaluation, we consider three different model inference
variants.

#### Context-Result-to-Response

Under this condition, we provided the ground truth dialogue history, language
understanding, belief states, and action policy. The model generates the
dialogue actions, delexicalized outputs as well as the final response. This
helps to evaluate the combined performance of the dialogue act policy and the
system response generation modules. This variant is similar to the Context-to-
Response evaluation in the existing work (Peng et al. 2020).

#### Context-State-to-Response

Under this condition, we provided the ground truth dialogue history, language
understanding, and the belief states. The model undergoes a 2-stage generation
here. First, the model generates the plan and the API actions. The generated
API action is combined with the ground truth belief state and converted to
backend API calls. If the generated API action is search-execute, a database
query result is returned. However, if the generated API action is either of
booking-execute or booking-retrieve, we just copy from the ground truth
booking results, since we are not able to reproduce the booking experience in
real time. If the corresponding ground truth booking result does not exist,
the we just use the model generated results (we hope to explore real time
booking experience in the future). In the second stage, the model generates
the dialogue actions, delexicalized outputs as well as the final response
similar to Context-Result-to-Response above. This evaluation variant helps to
evaluate the combined performance of the action policy (API and dialogue) and
the system response modules, which was not explored in the existing work.

#### End-to-end

This evaluation variant also involves performing a 2-stage generation similar
to the Context-State-to-Response. However, the model input only consists of
ground truth dialogue history and the current user utterance. The model first
generates the NLU outputs and everything up to the API actions. The generated
belief state and the API action is then used to make backend API calls using
the procedure described for the Context-State-to-Response variant. In the
second stage, the model generates the dialogue actions, delexicalized outputs
as well as the final response similar to Context-Result-to-Response above.
This evaluation variant helps to evaluate the combined performance of NLU,
DST, POL and NLG modules, which is similar to the end-to-end evaluation in the
existing work.

### Dataset Variants

We used three different datasets to explore the support for controllability
and explainability. The full version, FULL contains all the nodes shown in the
dialogue flow graph. The second version, MED excludes the all_entities and
all_domain variables. The final version, MIN further excludes the plans from
the MED version. We also tried excluding the delex variable, but the model
consistently performed worse, showing that intermediate template generation
contributes positively to response generation performance.

Table 5: Ablation Studies on the Dataset Variations $(k=5)$ Model |  $\uparrow$ Inform |  $\uparrow$ Success |  $\uparrow$ BLEU |  $\uparrow$ Combined
---|---|---|---|---
Context-result-to-response |  |  |
FULL | 72.90 | 54.10 | 18.58 | 82.08
MED | 71.20 | 56.00 | 18.28 | 81.88
MIN | 65.90 | 51.30 | 18.07 | 76.67
Context-state-to-response |  |  |
FULL | 67.40 | 48.50 | 16.34 | 74.30
MED | 67.10 | 50.10 | 16.42 | 75.02
MIN | 65.40 | 51.20 | 16.54 | 75.92
End-to-end |  |  |
FULL | 63.30 | 37.50 | 13.97 | 64.37
MED | 62.20 | 41.50 | 14.34 | 66.20
MIN | 62.50 | 43.00 | 15.29 | 68.04

Table 6: Ablation Studies on the Search Result Sample Size (Dataset variant MIN) Model |  $\uparrow$ Inform |  $\uparrow$ Success |  $\uparrow$ BLEU |  $\uparrow$ Combined
---|---|---|---|---
Context-result-to-response |  |  |
$k=1$ | 75.15 | 63.23 | 18.34 | 87.53
$k=3$ | 71.20 | 56.00 | 18.29 | 81.89
$k=5$ | 85.50 | 72.90 | 16.54 | 95.74
Context-state-to-response |  |  |
$k=1$ | 75.15 | 63.43 | 17.19 | 86.48
$k=3$ | 67.10 | 50.10 | 16.42 | 75.02
$k=5$ | 85.50 | 72.90 | 16.54 | 95.74
End-to-end |  |  |
$k=1$ | 72.65 | 56.81 | 15.40 | 80.13
$k=3$ | 67.70 | 52.50 | 15.39 | 75.48
$k=5$ | 62.50 | 43.00 | 15.29 | 68.04

## Results and Discussion

This section describes the experimental results and discusses some of the
unique patterns observed when applying the DLGNet-Task framework and model to
the MultiWoz dataset.

### Quantitative Evaluation

The performance of DLGNet-Task is reported in Tables 3 and 4 for context-
result-to-response and end-to-end evaluations respectively. DLGNet-Task models
produce performance comparable with relevant existing work, although the
models do not outperform state-of-art models based on the quantitative
measure. The performance gap can mostly be attributed to the noise introduced
to the dataset during our processing. The additional control and verification
signals introduced to training data might have contributed to the model’s
reduced performance since they are based on heuristics which might not always
be accurate. Also, there are lots of mismatches between the search result and
the actual ground truth dialogue actions. These mismatches requires non-
trivial effort to fix as also reported by Hosseini-Asl et al. (2020). To
obtain the best result, Hosseini-Asl et al. (2020) did not include the search
results in the downstream generation, which is not feasible for practical
applications. Also, to obtain the state-of-the-art SOLOIST models, the authors
excluded the entire action policy from their framework. However, in practical
applications, at least the API action policy is required to know when to
execute search, execute booking and retrieve booked information, which is not
considered in any of the existing work. DLGNet-Task, on the other hand, does
not sacrifice practical usefulness for the purpose of performance on a noisy
dataset.

When compared to other practical approaches, such as DSTC8 Track 1 Winner (Ham
et al. 2020), DLGNet-Task performs favorably, in addition to the capturing of
the API action policy.

### Qualitative Evaluation

A random conversation sample with the end-to-end model (with $k=1$ and dataset
variant MIN) and ground truth outputs are shown in Tables 1 and 2. The tables
show DLGNet-Task model being able to follow the multi-turn conversation,
correctly executing the dialogue state tracking, and the action policy, and
language generation function at each conversation turn. One striking
observation is the high level of coherence and relevance in the generated
responses from the DLGNet-Task model. The model is also able to capture both
short- and long-term temporal dependencies over multiple turns of
conversation.

### Ablation Studies

In our ablation studies we explored the impact of varying the training signals
in the dataset, and the size of the search results from the backend API, on
the model performance.

#### Effect of training signals with dataset variation

Table 6 shows the performance of DLGNet-Task model on the dataset with varying
amount of training signals with a fixed search result sample size $k=5$. It
could be observed that for all dataset varations, the models performs worse
with decreasing ground truth information from context-result-to-response,
context-state-to-response, to end-to-end as expected. We also observe that the
more training signal we have, the better the model performance with more
ground truth information during inference (e.g., context-result-to-response).
However, with less ground truth information (e.g., end-to-end),the dataset
with the least training signal (MIN) gives the best results. We attribute this
surprising behavior to the noise in the original dataset and our data
processing. Therefore, another way to look at this is that the model gives the
best end-to-end performance with the least noisy data.

#### Effect of Search Result Sample Size

We also compare the model performance with different sizes of the search
result sample. The results with best performing the MIN dataset variation are
reported in Table 6. The results show that the model performs worse with
increasing result sample size. Although this is also surprising, the behavior
could be attributed to the disparity in the informed entities in the ground
truth dialogue action policy and the ones in the search result. Even when the
informed entities are present in the search result, they are in random
positions, which makes it difficult to obtain any useful training signal for
the model. This observation shows the need for a consistent training dataset
in order to evaluate both the system architecture and the proposed neural
network models for end-to-end TOD systems.

## Conclusion

In this paper, we have proposed DLGNet-Task, an end-to-end neural network
framework for modeling multi-turn multi-domain task-oriented dialogue. The
DLGNet-Task model learns the joint distribution of the nodes (variables) of a
dialogue flow graph capable of representing both task-oriented and open-domain
dialogue systems. For TOD specific applications, DLGNet-Task is also capable
of learning the action policy towards the back-end API. Experimental results
show that DLGNet-Task gives comparable performance with existing approaches
with practical focus. The results also showed that performance of DLGNet is
hampered by the errors in the original MultiWoz dataset as well as the noise
introduced during the data processing. This also shows need for consistent TOD
datasets with properly defined dialogue flow graph. We hope to explore this
direction as part of our future work both in terms of dataset generation and
data processing pipeline. We also hope to improve DLGNet-Task model
performance with adversarial and reinforcement learning.

## References

  * Budzianowski and Vulic (2019) Budzianowski, P.; and Vulic, I. 2019.  Hello, It’s GPT-2 - How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems.  In _arXiv preprint arXiv:1907.05774_.
  * Budzianowski et al. (2018) Budzianowski, P.; Wen, T.-H.; Tseng, B.-H.; Casanueva, I.; Ultes, S.; Ramadan, O.; and Gasic, M. 2018.  MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling.  In _EMNLP_.
  * Chen et al. (2019) Chen, W.; Chen, J.; Qin, P.; Yan, X.; and Wang, W. Y. 2019.  Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention.  In _ACL_.
  * Ham et al. (2020) Ham, D.; Lee, J.; Jang, Y.; and Kim, K. 2020.  End-to-End Neural Pipeline for Goal-Oriented Dialogue Systems using GPT-2.  In _ACL_ , 583–592.
  * Holtzman et al. (2019) Holtzman, A.; Buys, J.; Forbes, M.; and Choi, Y. 2019.  The Curious Case of Neural Text Degeneration.  In _arXiv preprint arXiv:1904.09751_.
  * Hosseini-Asl et al. (2020) Hosseini-Asl, E.; McCann, B.; Wu, C.; Yavuz, S.; and Socher, R. 2020.  A Simple Language Model for Task-Oriented Dialogue.  In _arXiv preprint arXiv:2005.00796_.
  * Kim, Lee, and Stratos (2017) Kim, Y.-B.; Lee, S.; and Stratos, K. 2017.  OneNet: Joint Domain, Intent, Slot Prediction for Spoken Language Understanding.  In _IEEE Automatic Speech Recognition and Understanding Workshop_.
  * Lee, Lee, and Kim (2019) Lee, H.; Lee, J.; and Kim, T.-Y. 2019.  SUMBT: Slot-utterance matching for universal and scalable belief tracking.  In _ACL_.
  * Lee et al. (2019) Lee, S.; Zhu, Q.; Takanobu, R.; Li, X.; Zhang, Y.; Zhang, Z.; Li, J.; Peng, B.; Li, X.; Huang, M.; and Gao, J. 2019.  ConvLab: Multi-Domain End-to-End Dialog System Platform.  In _ACL_.
  * Mehri, Srinivasan, and Eskenazi (2019) Mehri, S.; Srinivasan, T.; and Eskenazi, M. 2019.  Structured fusion networks for dialog.  In _20th Annual Meeting of the Special Interest Group on Discourse and Dialogue_.
  * Olabiyi et al. (2019) Olabiyi, O.; Khazan, A.; Salimov, A.; and Mueller, E. 2019.  An Adversarial Learning Framework For A Persona-Based Multi-Turn Dialogue Model.  In _NAACL NeuralGen Workshop_.
  * Olabiyi and Mueller (2019) Olabiyi, O.; and Mueller, E. T. 2019.  DLGNet: A Transformer-based Model for Dialogue Response Generation.  In _arXiv preprint arXiv:1908.01841_.
  * Olabiyi et al. (2018) Olabiyi, O.; Salimov, A.; Khazane, A.; and Mueller, E. 2018.  Multi-turn Dialogue Response Generation in an Adversarial Learning Framework.  In _arXiv preprint arXiv:1805.11752_.
  * Pei, Ren, and de Rijke (2019) Pei, J.; Ren, P.; and de Rijke, M. 2019.  A modular task-oriented dialogue system using a neural mixture-of-experts.  In _WCIS: SIGIR 2019 Workshop on Conversational Interaction Systems._
  * Peng et al. (2020) Peng, B.; Li, C.; Li, J.; Shayandeh, S.; Liden, L.; and Gao, J. 2020.  SOLOIST: Few-shot Task-Oriented Dialog with A Single Pre-trained Auto-regressive Model.  In _arXiv preprint arXiv:2005.05298_.
  * Radford et al. (2019) Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; and Sutskever, I. 2019.  Language Models are Unsupervised Multitask Learners.  In _https://d4mucfpksywv.cloudfront.net/better-language-models_.
  * Ramadan, Budzianowski, and Gasic (2018) Ramadan, O.; Budzianowski, P.; and Gasic, M. 2018.  Large-Scale Multi-Domain Belief Tracking with Knowledge Sharing.  In _ACL_.
  * Serban et al. (2016) Serban, I.; Sordoni, A.; Bengio, Y.; Courville, A.; and Pineau, J. 2016.  Building end-to-end dialogue systems using generative hierarchical neural network models.  In _Proceedings of The Thirtieth AAAI Conference on Artificial Intelligence (AAAI 2016)_ , 3776–3784.
  * Serban et al. (2017) Serban, I. V.; Sordoni, A.; Lowe, R.; Charlin, L.; Pineau, J.; Courville, A.; and Bengio, Y. 2017.  A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogue.  In _Proceedings of The Thirty-first AAAI Conference on Artificial Intelligence (AAAI 2017)_.
  * Sutskever, Vinyals, and Le (2014) Sutskever, I.; Vinyals, O.; and Le, Q. 2014.  Sequence to sequence learning with neural networks.  In _Proceedings of Advances in Neural Information Processing Systems (NIPS)_ , 3104––3112.
  * Wen et al. (2015) Wen, T.-H.; Gasic, M.; Mrksic, N.; Su, P.-H.; Vandyke, D.; and Young, S. 2015.  Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems.  In _EMNLP_.
  * Williams et al. (2013) Williams, J.; Raux, A.; Ramachandran, D.; ; and Black, A. 2013.  The Dialog State Tracking Challenge.  In _SIGDIAL_.
  * Zhang et al. (2019) Zhang, Y.; Sun, S.; Galley, M.; Chen, Y.; Brockett, C.; Gao, X.; Gao, J.; Liu, J.; and Dolan, B. 2019.  DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation.  In _arXiv preprint arXiv:1911.00536_.
  * Zhao, Xie, and Eskenazi (2019) Zhao, T.; Xie, K.; and Eskenazi, M. 2019.  Rethinking action spaces for reinforcement learning in end-to-end dialog agents with latent variable models.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies._

Table A1: Example of training data for DLGNet-Task models based on the
MultiWoz dataset

Prompt |  Outputs
---|---
$<$conversation_sep$>$
usr: |  I am looking for information in Cambridge.
all_domains: |  general
domains: |  general
plans: |  domain:general, reqmore
dlg_acts: |  domain:general, reqmore-noerror-inform
delex: |  sure , what do you need to know ?
sys: |  Sure, what do you need to know?
$<$turn_sep$>$
usr: |  I would like to book a reservation at Caffe Uno.
intents: |  request_booking
entities: |  name:caffe uno
all_entities: |  name:caffe uno
all_domains: |  restaurant, general
domains: |  restaurant
slots_search: |  domain:restaurant, food:?, pricerange:?, area:?, name:caffe uno
slots_booking: |  domain:restaurant, time:?, day:?, people:?
plans: |  domain:restaurant, booking
dlg_acts: |  domain:restaurant, booking-noerror-request:[[’Day’, ’?’], [’Time’, ’?’], [’People’, ’?’]]
delex: |  what day and time do you need the reservation for ? and how many people in your group ?
sys: |  What day and time do you need the reservation for? And how many people in your group?
$<$turn_sep$>$
usr: |  I need the reservation for 1 person for Friday at 10:15.
entities: |  people:1, day:friday, time:10:15
all_entities: |  people:1, day:friday, time:10:15, name:caffe uno
all_domains: |  restaurant, general
domains: |  restaurant
slots_search: |  domain:restaurant, food:?, pricerange:?, area:?, name:caffe uno
slots_booking: |  domain:restaurant, time:10:15, day:friday, people:1
plans: |  domain:restaurant, booking
api_acts: |  domain:restaurant, booking-execute
results: |  domain:restaurant, booked:[’name’: ’caffe uno’, ’reference’: ’3UH2KQDP’]
dlg_acts: |  domain:restaurant, booking-noerror-book:[[’Ref’, ’3UH2KQDP’]]
delex: |  your booking was successful and your table has been reserved ! your reference number is [restaurant_reference] . can i do anything else for you today ?
sys: |  Your booking was successful and your table has been reserved! Your reference number is 3UH2KQDP. Can I do anything else for you today?
$<$turn_sep$>$
usr: |  Yes, I need a train on Friday also, going into Cambridge.
intents: |  request_train
entities: |  destination:cambridge, day:friday
all_entities: |  destination:cambridge, day:friday, people:1, time:10:15, name:caffe uno
all_domains: |  train, restaurant, general
domains: |  train
slots_search: |  domain:train, destination:cambridge, day:friday, departure:?, arriveBy:?, leaveAt:?
slots_booking: |  domain:train, confirm:?, people:?
plans: |  domain:train, search
dlg_acts: |  domain:train, search-noerror-request:[[’Arrive’, ’?’], [’Depart’, ’?’]]
delex: |  ok , where from and what time do you want to arrive by ?
sys: |  Ok, where from and what time do you want to arrive by?
$<$turn_sep$>$
usr: |  I am leaving peterborough and want to get to cambridge by 08:15
intents: |  request_booking
entities: |  arriveBy:08:15, departure:peterborough
all_entities: |  arriveBy:08:15, departure:peterborough, destination:cambridge, day:friday, people:1, time:10:15, name:caffe uno
all_domains: |  train, restaurant, general
domains: |  train
slots_search: |  domain:train, destination:cambridge, day:friday, departure:peterborough, arriveBy:08:15, leaveAt:?
slots_booking: |  domain:train, confirm:?, people:?
plans: |  domain:train, search, booking
api_acts: |  domain:train, search-execute
results: |  domain:train, [’Choice’, 5], [’Sample’, [’trainID’: ’TR0035’, ’arriveBy’: ’08:09’, ’day’: ’friday’, ’departure’: ’peterborough’, ’destination’: ’cambridge’, ’duration’: ’50 minutes’, ’leaveAt’: ’07:19’, ’price’: ’16.50 pounds’, ’trainID’: ’TR8286’, ’arriveBy’: ’07:38’, ’day’: ’friday’, ’departure’: ’peterborough’, ’destination’: ’cambridge’, ’duration’: ’50 minutes’, ’leaveAt’: ’06:48’, ’price’: ’16.50 pounds’, ’trainID’: ’TR3144’, ’arriveBy’: ’07:09’, ’day’: ’friday’, ’departure’: ’peterborough’, ’destination’: ’cambridge’, ’duration’: ’50 minutes’, ’leaveAt’: ’06:19’, ’price’: ’16.50 pounds’, ’trainID’: ’TR1799’, ’arriveBy’: ’06:38’, ’day’: ’friday’, ’departure’: ’peterborough’, ’destination’: ’cambridge’, ’duration’: ’50 minutes’, ’leaveAt’: ’05:48’, ’price’: ’16.50 pounds’, ’trainID’: ’TR1662’, ’arriveBy’: ’06:09’, ’day’: ’friday’, ’departure’: ’peterborough’, ’destination’: ’cambridge’, ’duration’: ’50 minutes’, ’leaveAt’: ’05:19’, ’price’: ’16.50 pounds’]]
dlg_acts: |  domain:train, booking-noerror-offerbook:[[’Arrive’, ’08:09’], [’Id’, ’tr0035’]], booking-noerror-request:[[’People’, ’?’]]
delex: |  i can put you on the [train_id] it will get you there by [value_time] . how many tickets do you need ?
sys: |  I can put you on the tr0035 it will get you there by 08:09. How many tickets do you need?
$<$turn_sep$>$

Prompt |  Outputs
---|---
usr: |  Just one. Can I get the reference number for that?
entities: |  confirm:yes, people:1
all_entities: |  confirm:yes, people:1, arriveBy:08:15, departure:peterborough, destination:cambridge, day:friday, time:10:15, name:caffe uno
all_domains: |  train, restaurant, general
domains: |  train
slots_search: |  domain:train, destination:cambridge, day:friday, departure:peterborough, arriveBy:08:15, leaveAt:?
slots_book: |  domain:train, confirm:yes, people:1
plans: |  domain:train, booking
api_acts: |  domain:train, booking-execute
results: |  domain:train, booked:[[’Ref’, ’E75VHN9I ’], [’Ticket’, ’16.5 pounds’]]
dlg_acts: |  domain:train, booking-noerror-inform:[[’Ref’, ’E75VHN9I ’], [’Ticket’, ’16.5 pounds’]]
delex: |  your reference number is [train_reference] and [value_price] pounds will be payable at the station . anything else i can do for you ?
sys: |  Your reference number is E75VHN9I and 16.5 pounds will be payable at the station. Anything else I can do for you?
$<$turn_sep$>$
usr: |  Yes, could you please email or fax me the fare amount, as well as the reference number?
all_domains: |  train, restaurant, general
domains: |  train
all_entities: |  confirm:yes, people:1, arriveBy:08:15, departure:peterborough, destination:cambridge, day:friday, time:10:15, name:caffe uno
slots_search: |  domain:train, destination:cambridge, day:friday, departure:peterborough, arriveBy:08:15, leaveAt:?
slots_book: |  domain:train, confirm:yes, people:1
slots_requestable: |  domain:train, booked:[’trainID’: ’TR0035’, ’reference’: ’E75VHN9I’]
plans: |  domain:train, booking
api_acts: |  domain:train, booking-retrieve
results: |  domain:train, booked:[[’Ticket’, ’16.50 pounds’], [’Ref’, ’E75VHN9I’]]
dlg_acts: |  domain:train, booking-noerror-inform:[[’Ticket’, ’16.50 pounds’], [’Ref’, ’E75VHN9I’]]
delex: |  it will cost you [value_price] pounds and the reference number is [train_reference] .
sys: |  it will cost you 16.50 pounds and the Reference number is : E75VHN9I.
$<$turn_sep$>$
usr: |  Thank you. That is all that I need.
all_entities: |  confirm:yes, people:1, arriveBy:08:15, departure:peterborough, destination:cambridge, day:friday, time:10:15, name:caffe un
all_domains: |  general, train, restaurant
domains: |  general
plans: |  domain:general, welcome
dlg_acts: |  domain:general, welcome-noerror-inform
delex: |  welcome , it was a pleasure serving you .
sys: |  Welcome, it was a pleasure serving you.
$<$conversation_sep$>$