# TuGeBiC – A Turkish German Bilingual Code-Switching Corpus

###### Abstract

In this paper we describe the process of collection, transcription, and
annotation of recordings of spontaneous speech samples from Turkish-German
bilinguals, and the compilation of a corpus called tugebic. Participants in
the study were adult Turkish-German bilinguals living in Germany or Turkey at
the time of recording in the first half of the 1990s. The data were manually
tokenised and normalised, and all proper names (names of participants and
places mentioned in the conversations) were replaced with pseudonyms. Token-
level automatic language identification was performed, which made it possible
to establish the proportions of words from each language. The corpus is
roughly balanced between both languages. We also present quantitative
information about the number of code-switches, and give examples of different
types of code-switching found in the data. The resulting corpus has been made
freely available to the research community.
Keywords: code-switching, Turkish, German, spoken corpus

languageresourceLanguage Resources

TuGeBiC – A Turkish German Bilingual Code-Switching Corpus

Jeanine Treffers-Daller^1, Özlem Çetinoğlu^2
---
^1Department of English Language and Applied Linguistics, University of
Reading
^2Institute for Natural Language Processing (IMS), University of Stuttgart
<EMAIL_ADDRESS><EMAIL_ADDRESS>

Abstract content

## 1\. Introduction

In many bilingual speech communities around the world, bilinguals switch
between languages within one conversation and even within one utterance. This
behaviour, which is generally called code-switching111Some researchers prefer
the term translanguaging [Creese and Blackledge, 2010] for pedagogical
practices which allow teachers and learners to make use of all their language
resources in classroom settings. As only a few transcripts in the TuGeBic data
were recorded in classrooms, the term translanguaging is less appropriate for
our purposes. presents a challenge for linguists trying to explain where
switching is possible within one sentence, as well as for psychologists
interested in bilingual speech processing. For linguists, it is very important
to obtain information about the variability in code-switching patterns: it has
become clear that there is a wide range of different code-switching patterns
across communities, and this variability cannot be accounted for exclusively
on the basis of linguistic variables, such as language typology and linguistic
distance between contact languages [Muysken, 2000, Muysken, 2013].
Psycholinguists and neuroscientists are also increasingly interested in
studying code-switching in experimental settings [Pfaff, 2000, Treffers-
Daller, 2020], because it is then easier to control for a range of background
variables. Computational approaches to code-switching have also gained
momentum in the last decade [Solorio et al., 2014, Molina et al., 2016,
Çetinoglu et al., 2016]. The rise of social media and the ubiquitous presence
of code-switching as found among multilingual online communities raised the
profile of code-switching in the research agenda of computational linguistics
and natural language processing.

For researchers from all fields, it is crucially important to have access to
code-switching corpora that have been carefully compiled and annotated, as
these can provide the necessary information about the frequency of different
types of code-switching in a particular language pair and social setting.
While some code-switching corpora are already in the public domain (see
Talkbank RN51, LinCE aguilar:2020, GLUECoS
khanuja:2020),222https://talkbank.org/
https://ritual.uh.edu/lince/datasets
https://microsoft.github.io/GLUECoS/333Bilingual corpora on the Talkbank
include the following language pairs: English-Chinese; German-Greek; Greek-
English, Hebrew-English, Japanese-Spanish, Spanish-English, Turkish-Danish,
Welsh-Spanish; the LinCE corpus contains four language pairs: Spanish-English,
Nepali-English, Hindi-English, and Modern Standard Arabic-Egyptian Arabic.
Finally, the GLUECos comprises English-Hindi and English-Spanish data. there
is a great need for more such corpora. This is because frequency patterns are
more likely to be valid if they are drawn from a large database. Making data
from a variety of sources accessible to the research community is also
important because code-switching patterns can vary depending on the generation
speakers belong to [Backus, 1996], or the type of bilinguals and the contexts
they live in, such as, for example, heritage speakers of Turkish living in
Germany versus returnees who live in Turkey [Treffers-Daller, 1997]. Less is
known about differences between patterns found in oral conversations versus
social media such as Twitter RN371 or about the ways in which patterns can
change over time. The greater the number of different corpora that are made
available to the research community, the more likely it is that we can draw
valid conclusions about the variability in code-switching patterns and the
explanations for these patterns. This aim can only be achieved if corpora are
presented in a computer-readable, standardised format, so that the data can be
searched with computational tools. Both labelled and unlabelled data are
valuable in this regard to conduct experiments, evaluate results as well as
computational aid for linguistic and psycholinguistic research questions.

The aim of the current paper is to provide information about the compilation
of a Turkish-German bilingual corpus, called tugebic. While some of the data
have been analysed in ?; ?) so far the entire corpus has not been available to
the research community. In this paper, we describe the process of collection,
transcription, and annotation of the corpus. In addition, we provide token-
level language IDs as automatic predictions, and describe quantitative and
qualitative aspects of code-switching in the corpus.

## 2\. Data

In this section we describe the collection of the corpus and give an overview
of the transcription process that was initially carried out.

### 2.1. Collection

The data for the tugebic corpus were collected in Germany and Turkey between
1993 and 1995 by a Turkish-German bilingual research assistant, who was born
in Germany, but had returned to Turkey with his parents at the age of twelve.
Free conversations were recorded with other undergraduate students at two
different universities in Turkey as well as with master students from a
university in Germany, and with family members and friends of the research
assistant. Twelve males and 24 females were audio recorded for the study. They
were between the ages of 18 and 50 years old, and the majority was in their
early twenties.

The audio recordings are no longer available and can therefore not be made
available to the research community. Participants provided consent for being
recorded, and for the analysis of the data for the purposes of the project.
Permission to make the data available was obtained from the University of
Reading, after consultation with the Ethics Committee and a data protection
officer at this institution. It has been agreed that data would only be
donated after they had been carefully anonymised and sensitive passages had
been deleted. No metadata will be released about the participants to protect
their identities.

All participants were Turkish-German bilinguals in that they had grown up with
both languages and used them on a daily basis. No measurements of their
knowledge of each language are available, but analyses of data from similar
groups of Turkish-German bilinguals [Daller, 1996, Daller et al., 2011] show
that the Turkish language proficiency of the heritage speakers (that is
speakers of Turkish who were born in Germany and still live there) is lower
than that of the returnees who had returned to Turkey in late adolescence and
still live in Turkey. For German, the reverse is true. The same contrast can
also be observed with respect to the amount of German and Turkish that is
spoken by heritage speakers and returnees in the tugebic corpus in that
conversations the research assistant held with heritage speakers contain more
German words than conversations held with returnees [Treffers-Daller, 1997].
The topics discussed during the recordings can be grouped into three main
groups: a) students who discuss the content of classes and exams, as well as
leisure activities. b) members of staff (two in total) who give a linguistics
lecture; c) conversations among family members and friends of the research
assistants, who talk about food, travel abroad, childhood experiences, leisure
activities such as films they watched and sports they played, as well as
differences between life in Turkey and Germany. Sensitive passages where
participants discussed details of their private life were omitted.

### 2.2. Transcription

The transcription was carried out by the research assistant who was collecting
the data, between 1993 and 1995. MS Word is chosen as the electronic format
the transcripts are preserved. Here we outline the basic principles of the
original transcription format.

  * •

Metadata All files had metadata, indicating time and place of recording as
well as participants in the conversation and others who were present but did
not contribute to the conversation.

  * •

Comments The transcripts contained comments on the situation or on the start
of a conversation.

  * •

Speaker ID All speakers had a speaker ID, generally just a single capital
letter, which represented the first letter of their first name, which was
followed by a colon. Speaker turns were represented with this ID. If the
identity of a speaker was unknown, they were represented as `X:'

  * •

Multiple speakers The speaker IDs were used in combination (e.g. A+B or A&B)
to indicate multiple speakers. Sometimes actual speaker names or kinship
terms, such as Mum, or generic descriptive terms, such as alle ``all''were
used.

  * •

Translations In a few cases German translations of Turkish expressions were
provided by the transcriber to make it more accessible to readers with higher
German proficiency than Turkish.

  * •

Colloquial forms Words were sometimes transcribed in colloquial forms, e.g.,
gidiyom ``I am going" instead of the standard form gidiyorum

  * •

Backchannels Backchannels (brief verbal messages from listeners signaling that
they are paying attention), such as mh ``mm" or echt? ``really?" etc., were
put in parentheses in between the utterances of the main speaker in the
original transcript.

  * •

Incomprehensible passages Incomprehensible passages were marked with XXX
(these were sometimes in upper and sometimes in lower case; the number of
letters varied too).

  * •

Pauses Three or more dots were used to indicate pauses.

  * •

Hyphens Between one and four hyphens are sometimes found in the middle of an
utterance or at the end of an utterance, probably to indicate hesitations or
incomplete sentences. A sequence of hyphens was used to indicate that a new
conversation had started.

## 3\. Machine-Readability and Consistency Improvements

In the 1990s, corpus data were rarely given to online databases, as these were
still under development. To be able to publish the data, we revisited the
transcriptions in 2021 and applied a series of manual and automatic
improvements. A Turkish-German bilingual Masters student who studied Turcology
at a German university worked as an annotator in the manual first stage, then
semi-automatic corrections were made by a Computational Linguistics student
(native German speaker) and the second author (native Turkish speaker) in the
second stage.

### 3.1. Manual Revisions

We formulated detailed guidelines to facilitate manual revisions. In this
section we present the major points of these guidelines.

#### Format Conversion

The very first step of the revision was moving the files from MS Word to .txt
format to ensure machine-readibility and encoding consistency. All files were
moved to UTF-8 encoding, and incorrect encodings of Turkish- and German-
specific characters were replaced with the correct ones. At a later stage, we
also unified the filenames and removed recurring content from files. The
resulting collection contains 25 files.

#### Sentence Boundaries and Tokenisation

Transcribing oral data is notoriously complex [MacWhinney, 2019]. There are no
full stops in spoken language, for example, which makes it very difficult to
determine where the boundary between sentences (or rather utterances) is.
Whatever the direction of research one adopts, it remains important to
determine where the boundaries between utterances are, which is never easy
with oral data, because participants do not always speak in complete
sentences: they may stop in the middle of an utterance or may decide to change
the structure of an utterance during processing, for example, which means that
rules for written language are not always observed in oral data.

For tugebic we have adopted the choices of the transcriber, who indicated
whether an utterance was finished by adding a full stop or another utterance
terminator at the end of an utterance, albeit in the awareness that other
transcribers might have made other choices.444As the recordings are no longer
available it is not possible to ask for a second opinion on this issue.
Whenever possible, we stayed as close as possible to the original. In this
case an utterance could contain multiple sentences even they are separated by
sentence-final punctuation. We nevertheless applied tokenisation, including
adding spaces before punctuation marks excepts ones that are part of tokens
(e.g., apostrophes in inflected Turkish proper names).

#### Intermediary Annotations

We introduced two intermediary annotation markers in the updated version of
the corpus. These are

  * •

$<$e$>$ for normalisation. For instance, sonar is a typo of the Turkish word
sonra ``later". The annotator marked this correction as sonar$<$e$>$sonra.

  * •

$<$-$>$ for anonymisation. For instance, if Ali is a name to be anonymised and
the replacement is Veli, the annotator indicated it in the corpus as
Ali$<$-$>$Veli.

We chose annotated corrections instead of directly correcting them in the
corpus for multiple reasons. When the annotator was in doubt they used a
question mark as the replacement string. This way it was possible to easily
reach all instances of unresolved cases. Moreover, the corrections are
countable and reversible (e.g. for the debatable case of normalising
colloquial use). The normalisation annotation could as well serve as a dataset
for automatic normalisation tasks in the future. In the second stage, the
string before the annotation marker is replaced with the string after the
marker automatically via a script.

#### Normalisation

We corrected capitalisation where necessary, and inserted apostrophes before
suffixes attached to proper names, as in İstanbul'da ``in Istanbul". Restoring
Turkish-specific characters, e.g. bunlarin$<$e$>$bunların, is also part of
normalisation. Non-standard orthography (e.g. gidiyom) was replaced with
standard orthography (gidiyorum ``I am going"). We also corrected spelling
errors and other typos, for example when an interrogative suffix was attached
to the verb root, as in biliyormusun ``do you know". According to standard
spelling rules the question suffix needs to be separated as in biliyor musun.
The same type of corrections were applied to writing de and ki clitics, which
are common mistakes also in Turkish social media.

The remaining revision steps could be summarised as follows:

  * •

Intra-word Code-switching When a token contains morphemes from two languages,
we call it intra-word code-switching. To facilitate the study of such mixed
tokens, these were marked with a paragraph mark §555This sign is chosen by
RN371 for practical purposes, that is, the Twitter corpus that contains a
diverse list of characters did not contain this particular character so that
it could be used as a unique identifier. As our corpus also did not contain
this character we follow the existing convention. at the boundary between the
two languages, as in Stadt§tan (city§abl meaning ``from the city"), where §
indicates the boundary between the German root Stadt ``city" and the Turkish
ablative suffix -tan.

  * •

Pauses Where in the original transcript three or more dots were used to
indicate pauses, these were replaced with the string [pause].

  * •

Backchannels Backchannels were put on separate lines and provided with a
speaker ID.

  * •

Unintelligible material The various representations of XXX were replaced with
the annotation [unintelligible].

  * •

Translations Translations were omitted from the final transcript, as
translations were not given for complete sentences, and creating glosses for
the entire corpus was beyond the scope our project.

### 3.2. Sanity and Consistency Checks

An automatic check was done to verify the manual revisions of the first phase
and to find out remaining mistakes. The corrections that need decisions are
resolved manually, the remaining ones are addressed automatically. This phase
can be summarised as follows:

  * •

During manual revision we asked the annotator to use a question mark as the
replacement string when they are uncertain about a normalisation or
anonymisation case. All such cases were resolved with appropriate
replacements.

  * •

In the original files there were a few instances of X used instead of person
names in utterances (e.g. X war doch schon verheiratet ``X was already
married"). They were replaced with the string [anon] so that they can be
distinguished from the speaker ID X.

  * •

Remaining human errors were corrected such as annotation mistakes (e.g. typo
in [unintelligible]), tokenisation, unmatched parentheses, idiosyncratic
characters.

  * •

Replacements for the intermediate anonymisation and normalisation annotations
were carried out.

  * •

Speaker IDs were standardised to a single character for a speaker followed by
a column sign, and there is a space before the utterance starts.

  * •

The representation of multiple speakers were unified. Two or more speakers
were represented their respective single characters combined with a `&' sign.

  * •

The hyphens in the middle of a sentence (between one and three) were left in
place. The number of the hyphens at the end was standardised to three. Four or
more hyphens on an otherwise empty line were converted to twenty `=', denoting
conversation separators.

  * •

Empty lines, trailing spaces were removed, and multiple spaces between tokens
were reduced to one.

### 3.3. Anonymisation

All names of speakers were replaced with names which started with the same
letter as the original name, and ensured vowel harmony rules would remain the
same for the pseudonym. Thus, for example, Gülsen was chosen as the pseudonym
for a female name which started with a G and for which the last syllable
contained the front vowel ``e". For names mentioned in the transcripts (not
participants in the project) we kept the format the original transcripts had
for names. When the name was fully transcribed (e.g. Ali), we anonymised it,
but when only the first letter was transcribed (e.g. A.) we did not anonymise
the names further. We did not anonymise celebrities, e.g. Mustafa Sandal, a
Turkish pop singer either.

Place names were replaced with alternative place names which sounded similar
in that they contained the vowels with the same articulatory properties in the
last syllable as the original, so that any suffixes on the names could be
maintained and vowel harmony rules were respected.

Furthermore, words and phrases that could reveal the identity of the speaker
due to context were also be anonymised. For instance, a course name was
anonymised in transcriptions where students talked about the degree course
they were taking. Finally, passages which revealed sensitive personal
information about the speaker(s) were deleted.

All anonymisation is done manually. In the sanity check phase we extracted the
list of anonymisation pairs and automatically checked if any references in the
list were left out. We then corrected the mistakes or kept the proper name as
is depending on the context (e.g. the Turkish female name Ayşe is not a
sensitive reference when used in Ayşe fasulye, a type of string beans).

### 3.4. Language ID Prediction

The original version of the corpus is not annotated with language IDs.
Instead, we provide predicted language IDs for each token. We trained the
STEPS tool grunewald:2021 that utilises XLM-RoBERTaconneau:2020 as a language
identifier.

As training data we used the 2.8 version of the Turkish-German SAGT Treebank
cetinoglu:2019.666github.com/UniversalDependencies/
UD_Turkish_German-SAGT The size of the training data in the standard split is
quite small (578 sentences), which might affect the performance in a new
dataset. Since our intention is to annotate the tugebic corpus as accurately
as possible, we increased the training data size by combining the standard
training and test splits. The model performs with 98.8% accuracy on the
development set of the treebank. As there are no gold language IDs on the
tugebic corpus, it is not possible to evaluate the accuracy of the model on
this dataset. Nevertheless the in-domain accuracy of the language identifier
is quite high and both tugebic and SAGT are transcriptions of bilingual
speaker conversations, hence it is likely that the same model can perform well
on the new dataset too.

The SAGT Treebank employs the following language ID tagset:

  * •

TR for Turkish

  * •

DE for German

  * •

LANG3 for tokens that belong to a third language other than Turkish and German

  * •

MIXED for tokens with intra-word code-switching

  * •

OTHER for punctuation, symbols, and any other tokens that do not fall into the
categories above

Before predicting language IDs, we did automatic sentence segmentation. We
used dot, exclamation mark, and question mark as sentence delimiters if they
are separate tokens in the middle of a line. If the sentence-final punctuation
was inside quotation marks, we assumed that they are part of a larger
sentence, e.g., reported speech, and did not apply the split. If segmentation
resulted in a sentence consisting only of [pause] token followed by a
sentence-final punctuation, we removed this sentence. We also removed all
[pause] tokens, speaker IDs and conversation separators.

After predicting language IDs we automatically applied minor postprocessing to
the output based on heuristics. We made sure that all [unintelligible] and
[anon] tokens, and all punctuation are assigned the OTHER label, and all
intra-word code-switching tokens with the § annotation have the MIXED language
ID.

## 4\. Corpus Characteristics

In this section we give a quantitative and qualitative analysis of the corpus.
The quantitative analysis includes the basic statistics of the corpus as well
as language ID and code-switching point distributions according to the
predicted language IDs. Although the statistics based on predictions are not
100% correct, we believe they are useful instruments to portray the overview
of the corpus with a close approximation of the real distribution.

As qualitative analysis we present a set of observations of code-switching
patterns from the corpus. In all examples German words are shown in bold font
and Turkish words in regular font. A morpheme-by-morpheme analysis is given
when the interpretation of the examples required it.777The abbreviations used
in the examples: 1PL = first person plural; 1SG = first person singular; 3SG =
third person singular; AOR = aorist; COND = conditional; DAT = dative; GEN =
genitive; LOC = locative; NMLZ = nominalization; PASS = passive, PL = plural;
PROG = progressive; PST = past; PTCP = participle; REL = relative. In
providing these interlinear glosses, we follow the Leipzig Glossing rules
[Comrie et al., 2015]. The numbered `Tugebic' string that is presented in
parentheses after the translation denotes the file the example is taken from.

### 4.1. Statistics

Basics |
---|---
# of tokens | $116\,688$ |
# of sentences | $14\,651$ |
# of monolingual sentences | $10\,141$ |
# of bilingual sentences | $4510$ |
# of CS points in bilingual sentences | $8180$ |
Avg sentence length | $7.96$ |
Avg CS points per bilingual sentence | $1.82$ |
Language ID Distribution |
TR | $43\,785$ | (37.52%)
DE | $43\,210$ | (37.03%)
OTHER | $28\,656$ | (24.56%)
MIXED | $686$ | (0.59%)
LANG3 | $351$ | (0.30%)
Table 1: The basic statistics of the tugebic corpus and the language ID
distribution according to predicted language IDs. CS denotes code-switching.

The basic statistics on the corpus and the language ID distribution are given
in Table 1. The number of sentences is based on the automatic segmentation.
The minimum sentence length is 1 and the maximum is 202 tokens. The majority
of the corpus consists of monolingual sentences. Only the sentences that
contain intrasentential or intra-word code-switching (i.e., bilingual
sentences) are considered in calculating the code-switching points.888For
instance, there are three code-switching points in (4.2) and two code-
switching points in (4.2). There are a total of 8180 code-switching points,
which correspond to an average of 1.82 points per bilingual sentence. This
figure is in line with the Turkish-German Twitter Corpus RN371 and the SAGT
Corpus cetinoglu:2019 that both consist of only bilingual sentences and that
both have averages slightly over 2 points.

In the language ID distribution, however, these corpora show different
behaviour. In the Twitter corpus TR tokens are dominant, and in the SAGT
Corpus, the DE tokens take approximately half of the corpus. In our case the
Turkish and German tokens are almost balanced. The differences are likely due
to the fact that the tugebic corpus also contains monolingual sentences, and
not just a selection of utterances with code-switching.

5565 of the monolingual sentences are in Turkish and 4323 are in German. They
are followed by 1582 sentences that start in German and continue in Turkish,
and for 853 sentences it is the other way around. They constitute the majority
of the sentences with a single code-switching point (the rest are sentences
with the combinations of LANG3 with TR or DE). 1246 sentences contain two
switches, 329 sentences contain three switches, and 216 sentences contain four
switches. Sentences with five or more switches are 234 in total. The highest
number of switches is 20, with only one instance.

### 4.2. Observations

Transcriptions contain a large number of switches between utterances
(intersentential code-switching), as in (4.2), where Speaker K makes a
statement in German, and Speaker O replies in German, after which Speaker K
answers in Turkish.999One reviewer suggests it would have been preferable to
give exact numbers of different types of switches. This is only possible if
the corpus is manually annotated with switch points and ideally with syntactic
structures. Thus this is beyond the scope of the current paper.

K: Bei uns kannst du nicht so ohne Jacke. [pause].
O: Warum?
K: Buz gibi. `K: At our place you cannot go without a jacket. O: Why? K: Cold
as ice.' (21Tugebic)

While in (4.2) the switch takes place between interlocutors, intersentential
code-switching can also take place between utterances of the same speaker, as
in (4.2), where Speaker S switches to Turkish after her first utterance in
German.101010Note that we take proper names as language-specific, hence they
take part in code-switching as exemplified with Seçil in (4.2). While many
proper names keep their original form across languages, there are also many
language-specific ones as in Munich in English, München in German, and Münih
in Turkish. And even if the proper name does not change, it is still an
unknown word for the other language in many cases, e.g., there will be no
instances of Seçil in monolingual resources of German.

A: Und wie haben wir angefangen, Seçil?
S: Na ja, weiß ich nicht. Yok bunun babası ilk defa beni şeye bıraktı. `A: And
how did we start, Seçil? S: Well, I don't know. No, his father first of all
left me at thingie.' (21Tugebic)

Switching within utterances (intrasentential code-switching) takes place very
frequently in the dataset. For the current purposes, we use the four-way
typology of [Muysken, 2013] to describe the different types of code-switching
in the data. One of the most frequent types of switching in the data is
insertion: informants often insert a content word or fixed expression from
German into Turkish as in (4.2), where the German compound Einkaufsstraße
``shopping road" is inserted into a Turkish sentence, or vice versa, as in
(4.2), where a Turkish noun zayıf ``low mark" is inserted into a German
sentence.

Böyle Einkaufsstraße gibi bir şey var, weißt du in Darmstadt! `There is a kind
of shopping street or something there, you know, in Darmstadt.' (19Tugebic,
Speaker A)

Es gibt keine zayıf§s. `There are no low marks.' (20Tugebic, Speaker M).

In (4.2) and (4.2) there is a clear matrix language [Myers-Scotton, 1997],
which sets the grammatical frame. In (4.2) the matrix language is Turkish, and
in (4.2) it is German. When the matrix language is Turkish, German nouns that
are inserted into Turkish can receive Turkish inflection, as in (4.2), where
German Wohnheim receives the Turkish plural -lar, which is followed by the
Turkish genitive -ın.

Mittlerweile bu şey-ler-in Wohnheim§-lar-ın hepsi bir sistem-e bağla-n-dı.
in the meantime that thing-PL-GEN dormitory§-PL-GEN all one system-DAT link-
PASS-PST.
`In the meantime, those things, dormitories, were all connected to one
system.' (14Tugebic, speaker S)

It is interesting that vowel harmony rules apply across language boundaries.
In (4.2), for example, the second syllable of Wohnheim ``dormitory" contains
the diphthong /aI/,which means that a plural suffix with the back vowel a,
namely -lar, needs to be chosen to mark this noun for number. The opposite
(German inflection on Turkish nouns) can be seen in 4.2, where zayıf ``low
(mark)" receives a German -s plural, although this is much rarer in the data
than Turkish marking of plurals on German nouns.

Another fairly rare type of code-switch can be found in (4.2). Here die
Zeitung ``the newspaper" is probably the subject of the verb ilan ediyor
``advertises", although this switch point is untypical because switching
between the subject and the verb is severely constrained in most language
combinations [MacSwan, 2000]. Another reason why this example is exceptional
is that in Turkish-German code-switching data we rarely find switches of full
Determiner Phrases, such as the one in 4.2: the German determiner die "the"
accompanies the German noun Zeitung ``newspaper".

Gazete-de ilan aaa die Zeitung ilan, ilan ed-iyor.
Newspaper-LOC advertise ehm the newspaper advertise, advertise do-PROG
`He advertises in the newspaper ehm, the newspaper advertises (it).'
(18Tugebic, Speaker Ö)

While informants frequently produce insertions such as those in 4.2, switches
can also occur at the periphery of a sentence, for example for an adverbial
phrase, such as mittlerweile ``in the mean time" in (4.2). This type of
switching is called alternation. Alternations are often longer and can, for
example, consist of an entire adverbial clause, as in (4.2), which begins with
a Turkish adverbial clause marked with -ken ``when/while", whereas the main
clause hab' ich mich gefreut ``I was happy" is in German. It is also
remarkable that the Turkish adverbial clause can trigger Verb Second in
German, as can be seen in the occurrence of the inflected verb hab' ``have"
directly after the adverbial clause in (4.2).

Ben bura-ya gel-ir-ken hab' ich mich gefreut.
I here-DAT come-AOR-when have I myself PTCP-be.happy-PTCP
`When I came here, I was happy.' (03Tugebic, Speaker H)

A special case of alternation can be found in (4.2), where the switch takes
place in the middle of a conditional clause: the German conditional
subordinate conjunction wenn ``if" is used at the start of the sentence,
followed by the German pronoun wir ``we" but the inflected verb olsaydık also
contains the information that this is a conditional clause, in the suffix sa
``if" and it is also inflected for person. The doubling of grammatical
information in German and Turkish is typical for alternational types of
switches [Muysken, 2013, cetinoglu:2019].

Wenn wir yakın ol-say-dı-k.
If we close be-COND-PST-1PL
`If we were close.' (01Tugebic, Speaker U)

The third type, congruent lexicalisation, is one where both German and Turkish
contribute grammatical and lexical items and the structure becomes a hybrid
between both languages, as in (4.2), where a German copula ist ``is" links the
subject okutman ``teacher" and the subject complement öǧretmen ``teacher".

Okutman ist sonuç-ta öǧretmen yani, normal öǧretmen ohne Dings.
Lecturer ist end-LOC teacher you know, normal teacher without thingie.
`A lecturer is a teacher after all, you know, a normal teacher without a
what's it called again.' (03Tugebic, Speaker K)

The fourth type of code-switching, called backflagging, involves the use of a
discourse marker from the heritage language (Turkish) in discourse that is
otherwise entirely in German. This is called backflagging because this
strategy can be used to flag up one's ethnic identity [Muysken, 2013, p. 713],
as in (4.2), which begins with the Turkish discourse marker işte ``well."

İşte, ich bin ausgestiegen. `Well, I got off (the bus).' (19Tugebic, Speaker
A)

Backflagging appears to be particularly frequent among the Germany-based
speakers in the tugebic corpus possibly because they mainly speak more German
than Turkish in daily life.

Another very frequent type of code-switching is one where the Turkish verb
yap- ``to do/make" is used as a light verb and combined with a German verb in
the infinitive form, as in (4.2), where German anmelden ``sign in" and
auflisten ``to list" are used in combination with the Turkish light verb yap-
``to do/make".

Ora-da bekle-me şey-i var, liste-si var, işte anmelden yap-an-lar-ın hepsi,
ora-ya auflisten yap-ıl-ıyor.
There-LOC wait-NMLZ thing-POSS there is, list-POSS, there is there sign.in do-
REL-PL-GEN all, there-DAT list do-PASS-PROG.3SG
`There was a waiting thing, a list, there were people signing in and lists
were made.' (14Tugebic, Speaker S)

In examples with mixed verbal compounds, the matrix verb yap- carries the
inflection, and the embedded German verb is in the infinitive form. This type
of bilingual compound verb construction is found in many Turkish-speaking
immigrant communities in Europe, e.g. in the Netherlands [Backus, 1996],
Germany [Treffers-Daller et al., 2016, Pfaff, 2000]RN371,cetinoglu:2019 or
Norway [Türker, 2000]. According to [Muysken, 2000], these forms can be seen
as insertions or alternations, depending on the type of element that is
embedded in this construction, and the relationship between the embedded verb
and the matrix verb. Interestingly, in the tugebic corpus, one of the teachers
avoided the light verb construction using yap-, and employed et- ``do/make" as
the matrix verb instead to create a mixed verbal compound, as in (4.2). This
form is frequently used in standard Turkish nominal verb compounds [Kerslake
and Goksel, 2014], but less common in mixed verbal compounds as found among
heritage speakers.

Hatta widersprechen ed-iyor-lar.
In fact contradict do-PROG-PL
`In fact, they contradict (that).' (02Tugebic, Speaker G)

The tutor may have preferred et- because compound verbs with yap- are
typically used by heritage speakers of Turkish but less often by Turkish
people who have always lived in Turkey [Pfaff, 2000, Treffers-Daller et al.,
2016], such as the tutor who produced this utterance.

Finally, there were a few exceptional cases of switches within words, as in
4.2, where Turkish tense -iyo and person marking -m is attached to the German
root komm ``come". This is a quite surprising use as the preferred way in such
constructions is employing light verbs as exemplified in (4.2) and (4.2).

Ich komm-iyo-m$<$e$>$komm§iyorum
I come-PROG-1SG
`I am coming.' (01Tugebic, Speaker U)

The other rare type of intra-word switch is (4.2), where a Turkish noun kayın
``in-law" and a German noun Sohn ``son" are combined into a mixed compound.
The speaker probably meant son-in-law as they repeated the same word in full
German as the next token. There are some kinship terms in Turkish that start
with kayın, e.g., kayınpeder ``father-in-law", kayınvalide ``mother-in-law",
kayınbirader ``brother-in-law" but son-in-law is not one of them. It seems the
speaker notices this mid-way and switches to German.

Almanya'da ben-im kayın§sohn, Schwieger-sohn
Germany-LOC I-1SG.POSS in-law§son, in-law-son.
`My son-in-law is in Germany." (01Tugebic, Speaker A)

Further research on the corpus will need to be carried out to determine the
frequency of the different code-switching types in the corpus, and the
sociolinguistic variables associated with this variability.

## 5\. Conclusion

In this paper, we have described the characteristics of a corpus of
spontaneous conversations among Turkish-German bilinguals in Turkey and in
Germany called tugebic. We have have summarised the steps taken for the data
collection, and given an overview of the transcription and further data
treatment of the corpus. Furthermore, we have presented quantitative details
about the number of monolingual and bilingual sentences and provided examples
of different types of code-switching in the data. To the best of our
knowledge, this is the largest Turkish-German dataset that is made available
to the research community to date.

As the tugebic data were collected in the 1990s, they offer an historical
perspective on code-switching as it was common in this bilingual speech
community about 30 years ago. A possible avenue for future research could be
to compare this dataset to more recent datasets RN352,cetinoglu:2019 to
establish whether code-switching patterns found currently differ from those in
the 1990s, or to compare code-switching patterns from spontaneous
conversations with those found in social media, such as Twitter RN371.

The corpus is also valuable as an additional resource in computational
studies, especially given that no other unlabelled Turkish-German code-
switching dataset is available. Both the standard corpus files and the files
with predicted language IDs can be obtained from
https://github.com/ozlemcek/TuGeBiC.

## 6\. Acknowledgements

We are very grateful to Kubilay Yalçın who collected and transcribed the data,
to Mustafa Cem Güneş who tokenised and normalised the data, and to Marina Haid
who implemented automatic sanity checks and corrections, and ran language ID
experiments. The first author also gratefully acknowledges sponsoring from the
Language Contact Fund of the University of Amsterdam for the data collection.
The second author is funded by DFG via project CE 326/11 Computational
Structural Analysis of German Turkish Code-Switching (SAGT). This paper was
accepted for LREC 2022 but subsequently withdrawn. We are thankful to the
three anonymous reviewers for their helpful comments.

## 7\. Bibliographical References

## References

  * Backus, 1996 Backus, A.  (1996).  Two in one. Bilingual speech of Turkish immigrants in the Netherlands.  Studies in multilingualism. Tilburg University Press, Tilburg.
  * Çetinoglu et al., 2016 Çetinoglu, Ö., Schulz, S., and Vu, N. T.  (2016).  Challenges of computational processing of code-switching.  In Mona T. Diab, et al., editors, Proceedings of the Second Workshop on Computational Approaches to Code Switching@EMNLP 2016, Austin, Texas, USA, November 1, 2016, pages 1–11. Association for Computational Linguistics.
  * Comrie et al., 2015 Comrie, B., Haspelmath, M., and Bickel, B.  (2015).  The leipzig glossing rules: Conventions for interlinear morpheme-by-morpheme glosses.  Report, Department of Linguistics of the Max Planck Institute for Evolutionary Anthropology and the Department of Linguistics of the University of Leipzig.
  * Creese and Blackledge, 2010 Creese, A. and Blackledge, A.  (2010).  Translanguaging in the bilingual classroom: A pedagogy for learning and teaching?  The modern language journal, 94(1):103–115.
  * Daller et al., 2011 Daller, M. H., Yıldız, C., de Jong, N. H., Kan, S., and Başbağı, R.  (2011).  Language dominance in turkish–german bilinguals: Methodological aspects of measurements in structurally different languages.  International Journal of Bilingualism, pages 215–236.
  * Daller, 1996 Daller, H.  (1996).  Migration und Mehrsprachigkeit. Der Sprachstand türkischer Rückkehrer aus Deutschland.  Peter Lang, Frankfurt am Main.
  * Kerslake and Goksel, 2014 Kerslake, C. and Goksel, A.  (2014).  Turkish: an essential grammar.  Routledge, London.
  * MacSwan, 2000 MacSwan, J.  (2000).  The architecture of the bilingual language faculty: Evidence from intrasentential code switching.  Bilingualism: language and cognition, 3(1):37–54.
  * MacWhinney, 2019 MacWhinney, B.  (2019).  Understanding spoken language through talkbank.  Behavior research methods, 51(4):1919–1927.
  * Molina et al., 2016 Molina, G., AlGhamdi, F., Ghoneim, M., Hawwari, A., Rey-Villamizar, N., Diab, M., and Solorio, T.  (2016).  Overview for the second shared task on language identification in code-switched data.  In Proceedings of the Second Workshop on Computational Approaches to Code Switching, pages 40–49, Austin, Texas, November. Association for Computational Linguistics.
  * Muysken, 2000 Muysken, P.  (2000).  Bilingual speech: A typology of code-mixing, volume 11.  Cambridge University Press.
  * Muysken, 2013 Muysken, P.  (2013).  Language contact outcomes as the result of bilingual optimization strategies.  Bilingualism: Language and cognition, 16(04):709–730.
  * Myers-Scotton, 1997 Myers-Scotton, C.  (1997).  Duelling languages: Grammatical structure in codeswitching.  Oxford University Press.
  * Pfaff, 2000 Pfaff, C. W., (2000).  Development and use of et-and yap-by Turkish/German bilingual children, pages 365–373.  Harrassowitz, Wiesbaden.
  * Solorio et al., 2014 Solorio, T., Blair, E., Maharjan, S., Bethard, S., Diab, M., Ghoneim, M., Hawwari, A., AlGhamdi, F., Hirschberg, J., Chang, A., and Fung, P.  (2014).  Overview for the first shared task on language identification in code-switched data.  In Proceedings of the First Workshop on Computational Approaches to Code Switching, pages 62–72, Doha, Qatar, October. Association for Computational Linguistics.
  * Treffers-Daller et al., 2016 Treffers-Daller, J., Daller, M., Furman, R., and Rothman, J.  (2016).  Ultimate attainment in the use of collocations among heritage speakers of turkish in germany and turkish–german returnees.  Bilingualism: Language and Cognition, 19(03):504–519.
  * Treffers-Daller, 1997 Treffers-Daller, J., (1997).  Variability in codeswitching styles: Turkish-German code-switching patterns, volume 1, pages 277–297.  Mouton de Gruyter, Berlin.
  * Treffers-Daller, 2020 Treffers-Daller, J., (2020).  Turkish-German code-switching patterns revisited: what naturalistic data can(not) tell us. John Benjamins, Amsterdam/Philadelphia.
  * Türker, 2000 Türker, E.  (2000).  Turkish-Norwegian codeswitching: Evidence from intermediate and second generation Turkish immigrants in Norway.  Thesis.

## 8\. Language Resource References

lrec2022-bib languageresource


# Sequence-to-Sequence Learning for Task-oriented Dialogue with Dialogue State
Representation

Haoyang Wen, Yijia Liu, Wanxiang Che*, Libo Qin, Ting Liu
Research Center for Social Computing and Information Retrieval,
Harbin Institute of Technology, China
{hywen, yjliu, car, lbqin<EMAIL_ADDRESS>

###### Abstract

Classic pipeline models for task-oriented dialogue system require explicit
modeling the dialogue states and hand-crafted action spaces to query a domain-
specific knowledge base. Conversely, sequence-to-sequence models learn to map
dialogue history to the response in current turn without explicit knowledge
base querying. In this work, we propose a novel framework that leverages the
advantages of classic pipeline and sequence-to-sequence models. Our framework
models a dialogue state as a fixed-size distributed representation and use
this representation to query a knowledge base via an attention mechanism.
Experiment on Stanford Multi-turn Multi-domain Task-oriented Dialogue Dataset
shows that our framework significantly outperforms other sequence-to-sequence
based baseline models on both automatic and human evaluation.

Title and Abstract in Chinese 面向任务型对话中基于对话状态表示的序列到序列学习

  * 面向任务型对话中，传统流水线模型要求对对话状态进行显式建模。这需要人工定义对领域相关的知识库进行检索的动作空间。相反地，序列到序列模型可以直接学习从对话历史到当前轮回复的一个映射，但其没有显式地进行知识库的检索。在本文中，我们提出了一个结合传统流水线与序列到序列二者优点的模型。我们的模型将对话历史建模为一组固定大小的分布式表示。基于这组表示，我们利用注意力机制对知识库进行检索。在斯坦福多轮多领域对话数据集上的实验证明，我们的模型在自动评价与人工评价上优于其他基于序列到序列的模型。

00footnotetext:  This work is licenced under a Creative Commons Attribution
4.0 International Licence. Licence details:
http://creativecommons.org/licenses/by/4.0/ 00footnotetext:  * Email
corresponding.

##  1 Introduction

Address | Distance | POI type | POI | Traffic info
---|---|---|---|---
638 Amherst St | 3 miles | grocery store | Sigona Farmers Market | car collision nearby
269 Alger Dr | 1 miles | coffee or tea place | Cafe Venetia | car collision nearby
5672 barringer street | 5 miles | certain address | 5672 barringer street | no traffic
200 Alester Ave | 2 miles | gas station | Valero | road block nearby
899 Ames Ct | 5 miles | hospital | Stanford Childrens Health | moderate traffic
481 Amaranta Ave | 1 miles | parking garage | Palo Alto Garage R | moderate traffic
145 Amherst St | 1 miles | coffee or tea place | Teavana | road block nearby
409 Bollard St | 5 miles | grocery store | Willows Market | no traffic

Driver: | Address to the gas station.
---|---
Car: | Valero is located at 200 Alester Ave.
Driver: | OK , please give me directions via a route that avoids all heavy traffic.
Car: | Since there is a road block nearby, I found another route for you and I sent it on your screen.
Driver: | Awesome thank you.

Figure 1: An example of a task-oriented dialogue that incorporates a knowledge
base. The knowledge base will be changed based on different dialogue
environment setting. Agents need to generate response based on current
knowledge base.

Task-oriented dialogue system attracts more and more attention with the
success of commercial systems such as Siri, Cortana and Echo. It helps users
complete specific tasks with natural language. Figure 1 shows a typical
example of a task-oriented dialogue, where an agent provides with location
information for a user. The requirements for the agents to accomplish users’
demands usually involve querying the knowledge base (KB), like acquiring
address from location information KB in Figure 1.

Typical machine learning approaches model the problem as a partially
observable Markov Decision Process (POMDP) [Williams and Young (2007, Young et
al. (2013], where a pipeline system is introduced. The pipeline system
consists of four components: natural language understanding (NLU, Tur and De
Mori, 2011) , dialogue state tracking [Williams et al. (2013, Williams (2012],
dialogue policy learning [Young et al. (2010] and natural language generation
[Wen et al. (2015]. Taking the utterance in Figure 1 for example, NLU maps the
utterance “Address to the gas station” into semantic slot “POI type”. Dialogue
state tracker keeps the probability of “gas station” close to 1 against other
values of slot “POI type”. Given a semantic frame as a dialogue state, which
is the combination of distributions of these slots, dialogue policy learning
generates the next pre-defined system action, which usually involves querying
the knowledge base. The action is then converted to its natural language
expression using natural language generation. Both natural language
understanding and dialogue state tracking require a large amount of domain
specific annotation for training, which is expensive to obtain. Besides, the
design of actions and the explicit forms of semantic frames require a lot of
knowledge from human experts, which are domain-specific as well.

Neural generative models, typically Seq2Seq models, have achieved success on
machine translation [Sutskever et al. (2014, Bahdanau et al. (2014, Luong et
al. (2015]. This success spurs the interests to apply Seq2Seq models into
dialogue systems. Seq2Seq models map dialogue history directly into the
response in current turn, while requires a minimum amount of hand-crafting.
However, conventional Seq2Seq doesn’t model the exterior data retrieval
explicitly, which makes it hard for Seq2Seq to generate information stored in
KB like meeting time and address, but this kind of retrieval is easy to
achieve for classic pipeline. To tackle with the problem, ?) use an additional
copy mechanism to retrieve entities that occurs in both KB and dialogue
history. ?) further introduced retrieval from key-value KB where the model
uses key representations to retrieve the corresponding values. However, not
all KBs are presented in key-value forms. Besides, an important component of
classic pipeline, dialogue state tracker, is not properly modeled, making it
difficult to precisely retrieve from KB.

In this paper, we propose a novel framework that takes the advantages from
both classic pipeline models and Seq2Seq models. We introduce dialogue states
into Seq2Seq learning, but in a implicit way. Distributions in classic state
tracking are modeled as a group of representation vectors computed by an
attention-based network [Britz et al. (2017], which can be considered as a
dialogue state representation that aggregates information for each slot. And
training this representation doesn’t require annotation of dialogue state
tracking. Our model queries the KB entries in an attention-based method as
well, so that the querying is differentiable, without domain-specific pre-
defined action spaces. Meanwhile we compute the representation for KB using
entry-level attention and aggregate the representation with dialogue state
representation to form a memory matrix of dialogue history and KB information.
While decoding, we perform an attention over memory and an attention over
input, incorporating copying mechanism [Gu et al. (2016] that allows model to
copy words from KBs to enhance the capability of retrieving accurate entities.

We evaluate the proposed framework on Stanford Multi-turn, Multi-domain
Dialogue Dataset [Eric et al. (2017], to test the effectiveness of our
framework and flexibility to apply to different domains. We compare our model
with other Seq2Seq models and discovered that our model has outperformed other
models on both automatic evaluation and human evaluation.

##  2 Proposed Framework

In this section, we describe a framework for task-oriented dialogue system.
Our framework first encodes previous dialogue history, and computes dialogue
state representation. Then our framework queries the table by attention and
computes a matrix to represent information from previous history and KB. At
last, the responses are generated using copying mechanism. The general
architecture is demonstrated in Figure 2.

Figure 2: Proposed framework.

###  2.1 Encoder

Given a dialogue consisting of utterances from a user and an agent, our
encoder encodes the whole dialogue history. We represent the dialogue history
as a sequence of utterances. We encode the previous dialogue history as one
single sequence consisting of each word in previous dialogue history and use
$\left(x_{1},x_{2},\ldots,x_{n^{\text{IN}}}\right)$ to denote the whole
dialogue history word by word, where $n^{\text{IN}}$ is the length of this
sequence. Words are mapped to word embeddings and a long short-term memory
network (LSTM) aggregates hidden representation over the sentence, denoted as
$\bm{H}^{\text{ENC}}=\left[\bm{h}^{\text{ENC}}_{1},\ldots,\bm{h}^{\text{ENC}}_{n^{\text{IN}}}\right]$
($\bm{H}^{\text{ENC}}\in\mathbb{R}^{d\times n^{\text{IN}}}$, $d$ is the
dimensionality of a hidden state).

###  2.2 Dialogue State Representation

The dialogue state tracking component of a dialogue system interprets the
previous dialogue history to a belief state [Williams et al. (2013], which
consists of a group of probability distributions over values for each slot.
The dialogue state tracking is the core component of pipeline model. It helps
with retrieving values from KB and generating accurate entities. To introduce
dialogue state into Seq2Seq learning, in this framework, we model
representation of belief state, motivated by ?). We do not compute the
explicit probability distribution for each slot. Instead, a group of
distributed representations is computed. In this paper, we assume each turn of
the dialogue is associated with $m$ slots and its dialouge state
representation is a matrix
$\bm{U}^{\text{IN}}=\left[\bm{u}^{\text{IN}}_{1},\ldots,\bm{u}^{\text{IN}}_{m}\right]\in\mathbb{R}^{d\times
m}$ whose columns represent corresponding distribution. We further assume that
$m$ equals to the number of columns in our KB. Each state representation
$\bm{u}_{k}^{\text{IN}}$ is calculated by an attention over the whole
representation of history:

$\bm{u}^{\text{IN}}_{k}\quad=\quad\sum_{t=1}^{n^{\text{IN}}}a^{\text{IN}}_{k}\left(t\right)\bm{h}^{\text{ENC}}_{t},\\\
$

where we assign each hidden state $m$ different kinds of scores and perform a
weighted average. The scores are computed by the following equations:

$\displaystyle\textsc{ScoreIn}\left(\bm{w}^{\text{A}}_{k},\bm{h}^{\text{ENC}}_{t}\right)\quad$
$\displaystyle=\quad\bm{w}^{\textit{A}^{\textit{T}}}_{k}\bm{h}^{\text{ENC}}_{t},$
$\displaystyle a^{\text{IN}}_{k}\left(t\right)\quad$
$\displaystyle=\quad\frac{\exp\left(\textsc{ScoreIn}\left(\bm{w}^{\text{A}}_{k},\bm{h}^{\text{ENC}}_{t}\right)\right)}{\sum_{t^{\prime}}\exp\left(\textsc{ScoreIn}\left(\bm{w}^{\text{A}}_{k},\bm{h}^{\text{ENC}}_{t^{\prime}}\right)\right)},$

$\bm{W}^{\textit{A}}=[\bm{w}^{\textit{A}}_{1},\ldots,\bm{w}^{\textit{A}}_{m}]$
is a parameter matrix in $\mathbb{R}^{d\times m}$.

###  2.3 Soft KB Attention

##### Table Encoder.

In our framework, we compile the process of querying a KB entry into an
attention network. The first step is to encode the tables. Each KB cell is
represented as the concatenation of the column name embedding
$\phi^{\textit{EMB}}(f)$ and the cell value embedding
$\phi^{\textit{EMB}}(c)$. This representation is further fed into a $\tanh$
non-linearity and the final representation can be formalized as
$\bm{c}=\tanh\left(\bm{W}^{\text{C}}\left[\phi^{\textit{EMB}}\left(c\right),\phi^{\textit{EMB}}\left(f\right)\right]\right)$.
The representation of the KB entry $\bm{C}_{k}$ is denote as
$\bm{C}_{k}=[\bm{c}_{k,1},...,\bm{c}_{k,m}]$, consisting all its cell
representations.

##### Entry and KB Representation.

Conventionally, task-oriented dialogue systems interact with KB via carefully
hand-crafted API calls, which are usually domain-specific and break the
differentiability [Wen et al. (2017b]. To make it differentiable, our
framework applies a soft-attention over the KB entries and the attention value
can be interpreted as the possibility that an entry will be used for decoding.
We use the similarity between $\bm{C}_{k}$ and $\bm{U}^{\text{IN}}$ to
represent attention score for the $k^{\rm th}$ entry. The similarity is
computed by the following equation:

$\text{sim}\left(\bm{C}_{k},U^{\text{IN}}\right)\quad=\quad\sum_{t=1}^{m}{\bm{c}_{k,t}\cdot\bm{u}^{\text{IN}}_{t}},$

where we sum the dot product between vectors in $\bm{C}_{k}$ and
$\bm{U}^{\text{IN}}$ respectively.

This similarity distribution is then converted into a probability distribution
naturally using a softmax function. This probability distribution indicates
the possibility to use entry $e_{k}$ in the further response generation given
previous dialogue history $\bm{x}_{<i}$ :

$p(e=e_{k}\mid\bm{x}_{<i})\quad=\quad\frac{\exp\left(\text{sim}\left(\bm{C}_{k},U^{\text{IN}}\right)\right)}{\sum_{k^{\prime}}\exp\left(\text{sim}\left(\bm{C}_{k^{\prime}},U^{\text{IN}}\right)\right)}.$

The information matrix from KB that is used for future generation can be
computed as a weighted summation:

$\bm{U}^{\textit{KB}}\quad=\quad\sum_{t=1}^{|T|}p(e=e_{t}\mid\bm{x}_{<i})\bm{C}_{t},$

where $\bm{U}^{\text{KB}}$ is denoted as the information matrix, $|T|$ is the
number of entries in this KB.

Since the dimensionalities of all parameters are not related to the size of
knowledge base. it allows changing the KB on-the-fly. Besides, there is no
need to define specific operations, which is required for using API calls to
extract information. Since we model the entries directly, it is appropriate to
extract information from non-entity-centric knowledge bases as well.

Finally, we combine these two kinds of information by concatenating
corresponding vectors and feeding them into a linear layer with an activation
function. Formally, we denote $\bm{U}^{\text{CAT}}$ as the concatenation of
$\bm{U}^{\text{IN}}$ and $\bm{U}^{\text{KB}}$. The two matrices are
concatenated by concatenated corresponding vectors respectively. The process
can be formulated as:
$\bm{U}=\tanh\left(\bm{W}^{\textit{CAT}}\bm{U}^{\textit{CAT}}\right)$. The
matrix $\bm{U}$ can be considered as a fix-sized memory representation over
dialogue history and knowledge base.

###  2.4 Decoder

In this section, we will discuss the decoder that takes all previously
calculated information to generate sentences. The vanilla Seq2Seq decoder
generates a sequence of words recurrently based on the last hidden state of a
encoder. We denote
$\left(\bm{h}^{\text{DEC}}_{1},\ldots,\bm{h}^{\text{DEC}}_{n^{\text{OUT}}}\right)$
as the hidden states of the decoder and
$\left(y_{1},\ldots,y_{n^{\text{OUT}}}\right)$ as the output sentence. We will
consider two kinds of information, that are information of dialogue history
and information from KB. The model aggregates information via attention over
KB representation and history representation.

##### Input Attention.

The conventional attention mechanism is introduced to extend the decoder,
where each hidden state in the encoder is assigned a score based on the
current hidden state $\bm{h}^{\text{OUT}}_{t}$ at time step $t$, and then the
context vector is computed by the weight summation [Luong et al. (2015]. This
process can be described by the following equations:

$\displaystyle\bm{c}^{\text{IN}}_{t}\quad$
$\displaystyle=\quad\sum_{i=1}^{n^{\text{IN}}}a^{\text{OUT}}_{t}\left(i\right)\bm{h}^{\text{ENC}}_{i},$
$\displaystyle\textsc{ScoreOut}\left(\bm{h}^{\text{ENC}}_{i},\bm{h}^{\text{DEC}}_{t}\right)\quad$
$\displaystyle=\quad\bm{v}^{\text{OUT}}\tanh\left(\bm{W}^{\text{OUT}}\left[\bm{h}^{\text{ENC}}_{i},\bm{h}^{\text{DEC}}_{t}\right]\right).$
$\displaystyle a^{\text{OUT}}_{t}\left(i\right)\quad$
$\displaystyle=\quad\frac{\exp\left(\textsc{ScoreOut}\left(\bm{h}^{\textit{ENC}}_{i},\bm{h}^{\text{DEC}}_{t}\right)\right)}{\sum_{i^{\prime}}\exp\left(\textsc{ScoreOut}\left(\bm{h}^{\text{ENC}}_{i^{\prime}},\bm{h}^{\text{DEC}}_{t}\right)\right)}.$

##### Memory Attention.

Besides the context vector through input attention, we also use another
context vector from the attention over the fix-size memory matrix $\bm{U}$ and
it’s computed as:

$\displaystyle\bm{c}^{\text{MEM}}_{t}\quad$
$\displaystyle=\quad\sum_{i=1}^{m}a^{\text{MEM}}_{t}\left(i\right)\bm{u}_{i},$
$\displaystyle\textsc{ScoreMem}\left(\bm{u}_{i},\bm{h}^{\text{DEC}}_{t}\right)\quad$
$\displaystyle=\quad\bm{v}^{\text{MEM}}\tanh\left(\bm{W}^{\text{MEM}}\left[\bm{u}_{i},\bm{h}^{\text{DEC}}_{t}\right]\right),$
$\displaystyle a^{\text{MEM}}_{t}\left(i\right)\quad$
$\displaystyle=\quad\frac{\exp\left(\textsc{ScoreMem}\left(\bm{u}_{i},\bm{h}^{\text{DEC}}_{t}\right)\right)}{\sum_{i^{\prime}}\exp\left(\textsc{ScoreMem}\left(\bm{u}_{i^{\prime}},\bm{h}^{\text{DEC}}_{t}\right)\right)}.$

In practice, we use an additional context vector from encoder to calculate the
dialogue state representation, which is different from the one that is used
for the start of decoding.

The two kinds of context vectors and the current hidden state are used for
decoding. We introduce a variant of copying mechanism [Gu et al. (2016] in
order to retrieve entities from KB directly. We first compute a probability
distribution over output vocabulary $\mathcal{V}$ and slot types
$\mathcal{V}^{\text{SLOT}}$, given previous dialogue history $\bm{x}_{<i}$ and
previous generated words $\bm{y}_{<t}$, which can be described as:
$p\left(\widetilde{y_{t}}\mid\bm{x}_{<i},\bm{y}_{<t}\right)=\mathrm{softmax}\left(W^{\text{O}}\left[\bm{h}^{\text{DEC}},\bm{c}^{\text{IN}},\bm{c}^{\text{MEM}}\right]\right).$
The probability of generating a slot type represents the sum of probability of
generating a slot value for this slot from KB. Since we have calculated the
probability of using an entry in section 2.3, the probability of generating a
word can be described as:

$\displaystyle
p\left(y_{t}=y\mid\widetilde{y_{t}},\bm{x}_{<i},\bm{y}_{<t}\right)\quad=\quad$
$\displaystyle p\left(\widetilde{y_{t}}=y\mid\bm{x}_{<i},\bm{y}_{<t}\right)$
$\displaystyle+\sum_{y^{\text{S}}\in\mathcal{V}^{\text{SLOT}}}p\left(\widetilde{y_{t}}=y^{\text{S}}\mid\bm{x}_{<i},\bm{y}_{<t}\right)\sum_{k=1}^{|T|}\mathbbm{1}\left\\{e_{k}\left(y^{\text{S}}\right)=y\right\\}p(e=e_{k}\mid\bm{x}_{<i}),$

where $e_{k}\left(y^{\text{S}}\right)$ is the cell that is in slot (i.e.
column) $y^{\text{S}}$. Note that the summation of probability over
$\mathcal{V}$ is exactly 1 after the model copies entities from KB.

###  2.5 Training

Conventionally, we use negative log likelihood (NLL) for training to train a
Seq2Seq model. Since there is no supervision for soft KB attention, and it is
easy to get over-fitting when we only use NLL, we apply policy gradient to
improve the performance of soft KB attention as well. We consider the KB and
fix-size memory representation from input as the environment in a
reinforcement learning setting. There is only one action, which is defined as
choosing a single entry from KB to help with generating response.
Heuristically, the more entities in an entry appear in dialogue context, the
higher possibility that this entry is used for generating response. Therefore,
we consider the number of entities from an entry $e$ that appear in previous
dialogue history or current gold response as reward $R\left(e\right)$, and
apply REINFORCE with baseline [Williams (1992]:
$J_{\textit{RL}}=-\mathbb{E}_{p(e\mid\bm{x}_{<i})}\left[R\left(e\right)-b\right]$,
where $b$ is a hyperparameter denoting the baseline reward. The joint loss is
the combination of the NLL and loss from reinforcement learning, which is:

$\displaystyle J\quad=\quad J_{\textit{NLL}}+\lambda J_{\textit{RL}}$

where $\lambda$ is a hyperpramater balancing the two factors. In practice, we
first use $J_{\textit{RL}}$ only to train the soft KB attention and its
encoder, without training for response generation, which will accelerate the
convergence of Seq2Seq learning. We apply data augmentation to the original
dataset as well, where we add delexicalised form responses into training data
in order to force our model to generate slot types first and then retrieve
entity from KB using copying mechanism. The delexicalised responses are
generated by simple matching and replacing.

##  3 Experiment

In this section, we first introduce the details of experiment setting. Then we
provide results and analyses of automatic evaluation and human evaluation in
order to compare with other baseline models. Besides, we present ablation test
to evaluate and analyze the function of different components in our framework.
Finally, we provide visualization of dialogue state representation and case
study.

| Weather | Navigation
---|---|---
Slot Types |  location, date, highest temperature, lowest temperature and weather attribute |  POI name, traffic info, POI type, address, distance
Table 1: Slot types for different domains.

###  3.1 Experiment Setting

We choose two KB-rich domains from Stanford Multi-turn Multi-domain Task-
oriented Dialogue Dataset ?), which are weather information retrieval and
point-of-interest (POI) navigation (navigation). We first change the form of
KB in weather information retrieval domain (weather). ?) integrates the
highest temperature, lowest temperature and weather information into a single
weekday column due to their incapability of utilizing non-entity-centric KB.
In this paper, we separate these information into three different column, and
the slots of these two domains are provided in Table 1.

Our framework is trained separately in these two domains, using the same
train/validation/test split sets as ?). We do not map the entities in dialogue
into its canonical form as what ?) have done, since our framework extract
entities directly from KB. And we evaluate our framework on exact entities as
well.

Our framework is trained using the Adam optimizer [Kingma and Ba (2014]. The
learning rate is $10^{-3}$, $\lambda$ for balancing two loss functions is
$10^{-1}$. We applied dropout [Srivastava et al. (2014] to the input and the
output of LSTM, with a dropout rate at $0.75$. We add the weight decay on the
model. The coefficient of weight decay is $5\times 10^{-6}$. The embedding
size and all hidden size are 200. The number of epochs for training soft KB
attention is 30 for both navigation and weather. Baseline $b$ is 1.5 for both
navigation and weather.

####  3.1.1 Baseline Models

We compare our model with two additional baselines beyond the Seq2Seq with
attention, which includes:

  * •

Copy-augmented Sequence-to-Sequence Network. This model is adapted from [Eric
and Manning (2017]. It utilizes entities that appear in both previous dialogue
history and KB. A hard copy mechanism for these entities is applied in this
model.

  * •

Key-value Retrieval Network. This model is adapted from ?). It utilizes key-
value forms to represent KBs. Key representations are used for an attention-
based value retrieval. In weather information retrieval domain, although we
have changed the KB into an non-entity-centric form, we still designate
“location” slot as subject slot and we allow a key representation to retrieval
multiple values. We convert inputs into canonical forms, while the outputs
remain the same in order to compare with our model.

###  3.2 Automatic Evaluation

In this section, we provide two different automatic evaluations to compare
with other baseline models. The results and analyses are provide in the
following sections.

####  3.2.1 Evaluation Metrics

Entity F1 and BLEU score are used to evaluate our model. The entity F1 scores
in both micro-average and macro-average manners are used to measure the
difference between entities in the system and gold responses. Besides, we use
BLEU to evaluate the quality of responses. ?) showed that BLEU has a
comparatively strong correlation with human evaluation on task-oriented
dialogue dataset. The BLEU score is computed from results with highest micro
F1. To evaluate macro F1, we delete instances that neither gold nor generated
response contains an entity.

####  3.2.2 Results and Analyses

Experiment results are illustrated in Table 2. The results show that our model
outperforms other models in most of automatic evaluation metrics. In the
navigation domain, compared to KV Net, we achieve 5.0 improvement on BLEU
score, 37.1 improvement on Macro F1 and 27.4 improvement on Micro F1. Compared
to Copy Net, we achieve 5.0 improvement on BLEU score, 41.2 improvement Macro
F1 and 27.4 improvement on Micro F1. The results in navigation show our
model’s capability to generate more natural and accurate response than the
Seq2Seq baseline models. In the weather domain, our model generates more
accurate responses than our baseline models as well. The BLEU score is a
little bit lower than Copy Net and Seq2Seq with attention. This is because the
forms of responses are relatively limited in weather domain. Besides, the
entities in inputs are highly probable to be mentioned in responses, such as
“location”. These two reasons indicate that the simpler models can capture
this pattern more smoothly. The results that Seq2Seq with Attention performs
better than Copy Net and KV Net also confirm this.

We also find that the KV Net’s results are lower than that reported by ?). We
address this to the differences in the preprocessing, model training and
evaluation metrics. In spite of the difference of evaluation metrics that we
evaluate on exact entities rather than their canonical forms, the Micro F1
score of our model still outperforms what ?) reported, which is 41.3 in
navigation domain and which is evaluated on canonical forms. Our changes of
the weather domain into non-entity-centric also influence its performance.
This differences in results also indicate the robustness of our model when
facing non-entity-centric KBs.

| Navigation | Weather
---|---|---
Model | BLEU | Macro F1 | Micro F1 | BLEU | Macro F1 | Micro F1
Seq2Seq with Attention | 8.3 | 15.6 | 17.5 | 19.6 | 56.0 | 53.5
Copy Net | 8.7 | 20.8 | 23.7 | 17.5 | 52.4 | 53.1
KV Net | 8.7 | 24.9 | 29.5 | 12.4 | 37.7 | 39.4
our model | 13.7 | 62.0 | 56.9 | 14.9 | 58.5 | 56.3
Table 2: Automatic evaluation on test data. Best results are shown in bold. Generally, our framework outperforms other models in most automatic evaluation metrics.  Model | BLEU | Macro F1 | Micro F1
---|---|---|---
our model | 13.7 | 62.0 | 56.9
-copying | 9.6 | 35.2 | 41.3
-RL | 9.3 | 38.2 | 46.0
Table 3: Ablation experiment on navigation domain. -copy refers to a framework
without copying. -RL refers to a framework without RL loss.

####  3.2.3 Ablation

In this section, we perform several ablation experiments to evaluate different
components in our framework on the navigation domain. Results are shown in
Table 3. The results demonstrate effectiveness of components of our model to
the final performance.

Copying mechanism enables our framework to retrieve entities directly from
KBs. Without copying mechanism, such retrieval is infeasible and our framework
cannot produce values in KBs. The results show that it introduces more
variability to the generation process if we do not use copying mechanism.

The reinforcement learning loss helps our framework to use correct KB entries
so that improve the performance of generation. Without this reinforcement
learning loss, the item selection process is only supervised by log likelihood
loss. We address the drop in performance to that our model overfits to the
training data.

Model | Correct | Fluent | Humanlike
---|---|---|---
Copy Net | 3.52 | 4.47 | 4.17
KV Net | 3.61 | 4.50 | 4.20
our model | 4.21 | 4.65 | 4.38
agreement | 41.0 | 55.0 | 43.0
Table 4: Human evaluation of responses based on random selected previous
dialogue history in test dataset. The agreement scores indicate the percentage
of responses to which all three human experts give exactly the same scores.

###  3.3 Human Evaluation

In this section, we provide human evaluation on our framework and other
baseline models. We randomly generated 200 responses. These response are based
on distinct dialogue history in navigation test data. We hire three different
human experts to evaluate the quality of responses. Three dimensions are
involved, which are correctness, fluency, and humanlikeness. Three human
experts judged each dimension on a scale from 1 to 5. And each judgment
indicates a relative score compared to standard response from test data. The
results are illustrated in Table 4. The results show that our framework
outperforms other baseline models on all metrics. The most significant
improvement is from correctness, indicating that our model generates more
accurate information that the users want to know.

###  3.4 Visualization and Case Study

We provide a visualization example to demonstrate the effectiveness of
dialogue state representation. The visualization illustrates attention scores
over the sentence for each slot. The blue-level of a cell indicates the
attention score it represents. From this visualization, we can discover that
our dialogue state representation matches slots with correct entities in
sentence. For example, “pizza_restaurant” matches “poi” and “poi_type”
correctly, “4_miles” matches “address” correctly. The visualization indicates
the capability of our framework to track accurate information and integrate
them into a fix-size matrix representation.

Finally, we provide a case study that consists of two conversations which are
generated between our framework and a human and between Seq2Seq with Attention
with human respectively. In this case, we find that our framework is able to
generate correct information such as address and point-of-interest.
Conversely, Seq2Seq with Attention is generated in a more random way. The
comparison between these two dialogues illustrates the capability of our model
to retrieve accurate entities and while in the same time generate natural
response.

Figure 3: Visualization of dialogue state representation.

Address | Distance | POI type | POI | Traffic info
---|---|---|---|---
783 Arcadia Pl | 5 miles | gas station | Chevron | moderate traffic
383 University Ave | 5 miles | shopping center | Town and Country | no traffic
864 Almanor Ln | 5 miles | friends house | jacks house | no traffic
5671 barringer street | 6 miles | home | home | heavy traffic
657 Ames Ave | 4 miles | rest stop | The Clement Hotel | no traffic
638 Amherst St | 1 miles | grocery store | Sigona Farmers Market | heavy traffic
830 Almanor Ln | 6 miles | chinese restaurant | tai pan | no traffic

our model
---
Driver: |  what gas stations are here ?
Car: |  there is a chevron 5 miles away .
Driver: |  that ’s good ! please show me the address and pick the quickest route to get there !
Car: |  the address is 783 arcadia pl , i sent on your screen the best route to reach there .
Driver: |  thank you !
Car: |  you ’re welcome !

Seq2Seq with Attention
---
Driver: |  what gas stations are here ?
Car: |  there is a chevron 4 miles away .
Driver: |  that ’s good ! please show me the address and pick the quickest route to get there !
Car: |  the address to safeway is 452 arcadia pl .
Driver: |  thank you !
Car: |  you ’re welcome !

Figure 4: Case study. The dialogue in the left is generated from our
framework, and that in the right is generated from Seq2Seq with Attention
baseline.

##  4 Related Work

The recent successes on neural networks spread across many natural language
processing tasks, which also stimulate research on task-oriented dialogue
system. The powerful distributed representation ability of neural networks
makes task-oriented dialogue system end-to-end possible. Recently, ?) built a
system that connects classic pipeline modules by a policy network training
with a user simulator. ?) further introduced latent intention into end-to-end
learning. However, their modules like belief tracker still needs to be trained
separately before end-to-end training. In contrast to their work, our
framework trained the state tracker jointly with the end-to-end dialogue
training. ?) built a turn-level LSTM to model the dialogue state and generate
probability distribution for each slot. ?) built a system by applying memory
network to store the previous dialogue history. But the responses are
retrieved from templates, which is significantly different from our neural
generative responses. Another type of work tried to build an end-to-end system
as a task completion dialogue system [Li et al. (2016, Li et al. (2017, Peng
et al. (2017]. These modeled are trained through an end-to-end fashion via
reinforcement learning algorithm using the reward from user simulators.
However, their dialogue responses are not generated from the dialogue history
directly but from a set of pre-defined action and explicit semantic frames.

Our soft KB attention can be considered as a process to retrieve entries,
which has been explored in many QA and dialogue work. One line of this
research includes creating well-defined API calls to query the KB [Williams et
al. (2017, Wen et al. (2017a]. And another line of research tried to directly
retrieve entities from knowledge base. ?) has built a system to encode all
table cells and assign a score vector to each row. Our framework resembles the
second line of research, but can generate multiple entities to form natural
language responses. ?) has built two symmetric dialogue agents with private
knowledge, and has applied knowledge graph reasoning into Seq2Seq learning,
which is distantly related with our framework. In the sense of the KB forms,
?) retrieved entities based on (subject, relation, object) triples. While ?)
applied a soft-KB lookup on an entity-centric knowledge base to compute the
probability of that the user knows the values of slots, and has tried to model
the posterior distributions over all slots. However, our framework doesn’t
require entity-centric knowledge base.

##  5 Conclusion

In this paper, we proposed a framework that leverages dialogue state
representation, which is tracked by an attention-based methods. Our framework
performed an entry-level soft lookup over the knowledge base, and applied
copying mechanism to retrieve entities from knowledge base while decoding.
This framework was trained in an end-to-end fashion with only the dialogue
history, and get rid of other annotation. Experiments showed that our model
outperformed other Seq2Seq models on both automatic and human evaluation. The
visualization and case study demonstrated the effectiveness of dialogue state
representation and entity retrieval.

##  6 Acknowledgments

We thank the anonymous reviewers for their helpful comments and suggestions.
This work was supported by the National Key Basic Research Program of China
via grant 2014CB340503 and the National Natural Science Foundation of China
(NSFC) via grant 61632011 and 61772153.

## References

  * [Bahdanau et al. (2014]  Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio.  2014\.  Neural machine translation by jointly learning to align and translate.  arXiv preprint arXiv:1409.0473.
  * [Bordes and Weston (2017]  Antoine Bordes and Jason Weston.  2017\.  Learning end-to-end goal-oriented dialog.  In Proceedings of International Conference on Learning Representations.
  * [Britz et al. (2017]  Denny Britz, Melody Guan, and Minh-Thang Luong.  2017\.  Efficient attention using a fixed-size memory representation.  In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 392–400.
  * [Dhingra et al. (2017]  Bhuwan Dhingra, Lihong Li, Xiujun Li, Jianfeng Gao, Yun-Nung Chen, Faisal Ahmed, and Li Deng.  2017\.  Towards end-to-end reinforcement learning of dialogue agents for information access.  In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 484–495.
  * [Eric and Manning (2017]  Mihail Eric and Christopher Manning.  2017\.  A copy-augmented sequence-to-sequence architecture gives good performance on task-oriented dialogue.  In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: (Volume 2, Short Papers), volume 2, pages 468–473.
  * [Eric et al. (2017]  Mihail Eric, Lakshmi Krishnan, Francois Charette, and Christopher D Manning.  2017\.  Key-value retrieval networks for task-oriented dialogue.  In Proceedings of the 18th Annual SIGDial Meeting on Discourse and Dialogue, pages 37–49.
  * [Gu et al. (2016]  Jiatao Gu, Zhengdong Lu, Hang Li, and Victor OK Li.  2016\.  Incorporating copying mechanism in sequence-to-sequence learning.  In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1631–1640.
  * [He et al. (2017]  He He, Anusha Balakrishnan, Mihail Eric, and Percy Liang.  2017\.  Learning symmetric collaborative dialogue agents with dynamic knowledge graph embeddings.  In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1766–1776.
  * [Kingma and Ba (2014]  Diederik P Kingma and Jimmy Ba.  2014\.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980.
  * [Li et al. (2016]  Xiujun Li, Zachary C Lipton, Bhuwan Dhingra, Lihong Li, Jianfeng Gao, and Yun-Nung Chen.  2016\.  A user simulator for task-completion dialogues.  arXiv preprint arXiv:1612.05688.
  * [Li et al. (2017]  Xiujun Li, Yun-Nung Chen, Lihong Li, Jianfeng Gao, and Asli Celikyilmaz.  2017\.  End-to-end task-completion neural dialogue systems.  In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), volume 1, pages 733–743.
  * [Liu and Lane (2017]  Bing Liu and Ian Lane.  2017\.  An end-to-end trainable neural network model with belief tracking for task-oriented dialog.  In Interspeech 2017.
  * [Luong et al. (2015]  Thang Luong, Hieu Pham, and Christopher D. Manning.  2015\.  Effective approaches to attention-based neural machine translation.  In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1412–1421, Lisbon, Portugal, September. Association for Computational Linguistics.
  * [Peng et al. (2017]  Baolin Peng, Xiujun Li, Lihong Li, Jianfeng Gao, Asli Celikyilmaz, Sungjin Lee, and Kam-Fai Wong.  2017\.  Composite task-completion dialogue policy learning via hierarchical deep reinforcement learning.  In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2231–2240.
  * [Sharma et al. (2017]  Shikhar Sharma, Layla El Asri, Hannes Schulz, and Jeremie Zumer.  2017\.  Relevance of unsupervised metrics in task-oriented dialogue for evaluating natural language generation.  arXiv preprint arXiv:1706.09799.
  * [Srivastava et al. (2014]  Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov.  2014\.  Dropout: A simple way to prevent neural networks from overfitting.  The Journal of Machine Learning Research, 15(1):1929–1958.
  * [Sutskever et al. (2014]  Ilya Sutskever, Oriol Vinyals, and Quoc V Le.  2014\.  Sequence to sequence learning with neural networks.  In Advances in neural information processing systems, pages 3104–3112.
  * [Tur and De Mori (2011]  Gokhan Tur and Renato De Mori.  2011\.  Spoken language understanding: Systems for extracting semantic information from speech.  John Wiley & Sons.
  * [Wen et al. (2015]  Tsung-Hsien Wen, Milica Gašic, Dongho Kim, Nikola Mrkšic, Pei-Hao Su, David Vandyke, and Steve Young.  2015\.  Stochastic language generation in dialogue using recurrent neural networks with convolutional sentence reranking.  In 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, page 275.
  * [Wen et al. (2017a]  Tsung-Hsien Wen, Yishu Miao, Phil Blunsom, and Steve Young.  2017a.  Latent intention dialogue models.  In International Conference on Machine Learning, pages 3732–3741.
  * [Wen et al. (2017b]  Tsung-Hsien Wen, David Vandyke, Nikola Mrkšić, Milica Gasic, Lina M. Rojas Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young.  2017b.  A network-based end-to-end trainable task-oriented dialogue system.  In EACL, pages 438–449, Valencia, Spain, April. Association for Computational Linguistics.
  * [Williams and Young (2007]  Jason D Williams and Steve Young.  2007\.  Partially observable markov decision processes for spoken dialog systems.  Computer Speech & Language, 21(2):393–422.
  * [Williams et al. (2013]  Jason D Williams, Antoine Raux, Deepak Ramachandran, and Alan Black.  2013\.  The dialog state tracking challenge.  In Proceedings of the 14th Annual SIGDial Meeting on Discourse and Dialogue, pages 404–413.
  * [Williams et al. (2017]  Jason D Williams, Kavosh Asadi, and Geoffrey Zweig.  2017\.  Hybrid code networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning.  In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 665–677.
  * [Williams (1992]  Ronald J Williams.  1992\.  Simple statistical gradient-following algorithms for connectionist reinforcement learning.  In Reinforcement Learning, pages 5–32. Springer.
  * [Williams (2012]  Jason D Williams.  2012\.  A belief tracking challenge task for spoken dialog systems.  In NAACL-HLT Workshop on future directions and needs in the spoken dialog community: tools and data, pages 23–24. Association for Computational Linguistics.
  * [Yin et al. (2016a]  Jun Yin, Xin Jiang, Zhengdong Lu, Lifeng Shang, Hang Li, and Xiaoming Li.  2016a.  Neural generative question answering.  In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16), pages 2972–2978. AAAI Press.
  * [Yin et al. (2016b]  Pengcheng Yin, Zhengdong Lu, Hang Li, and Ben Kao.  2016b.  Neural enquirer: learning to query tables in natural language.  In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16). AAAI Press.
  * [Young et al. (2010]  Steve Young, Milica Gašić, Simon Keizer, François Mairesse, Jost Schatzmann, Blaise Thomson, and Kai Yu.  2010\.  The hidden information state model: A practical framework for pomdp-based spoken dialogue management.  Computer Speech & Language, 24(2):150–174.
  * [Young et al. (2013]  Steve Young, Milica Gašić, Blaise Thomson, and Jason D Williams.  2013\.  Pomdp-based statistical spoken dialog systems: A review.  Proceedings of the IEEE, 101(5):1160–1179.