# Recent Advancements and Challenges of Turkic Central Asian Language
Processing

Yana Veitsman
Department of Language Science and Technology
Saarland University, Germany
<EMAIL_ADDRESS>
###### Abstract

Research in the NLP sphere of the Turkic counterparts of Central Asian
languages, namely Kazakh, Uzbek, Kyrgyz, and Turkmen, comes with the typical
challenges of low-resource languages, like data scarcity and a general lack of
linguistic resources. However, in the recent years research has greatly
advanced via collection of language-specific datasets and development of
downstream task technologies. Aiming to summarize this research up until May
2024, this paper also seeks to identify potential areas of future work. To
achieve this, the paper gives a broad, high-level overview of the linguistic
properties of the languages, the current coverage and performance of already
developed technology, application of transfer learning techniques from higher-
resource languages, and availability of labeled and unlabeled data for each
language. Providing a summary of the current state of affairs, we hope that
further research will be facilitated with the considerations we provide in the
current paper.

Recent Advancements and Challenges of Turkic Central Asian Language Processing

Yana Veitsman Department of Language Science and Technology Saarland
University, Germany<EMAIL_ADDRESS>

##  1 Introduction

Turkic languages are spoken by around 200 million people worldwide. Among
them, native speakers of Kazakh, Uzbek, Kyrgyz, and Turkmen, living both
inside and outside their respective countries, make over 60 million of those
speakers. Given their geographical proximity, historical background, and
closely related linguistic topology, these languages are prone to share the
same challenges in the development of NLP technologies, which range anywhere
from a lack of quality data to limited research opportunities in the field.
Just like other low-resource language speakers, the speakers of Central Asian
languages would benefit immensely from having trustworthy and reliable
language technology, from simple spell checkers to virtual assistants. The
technologies would also uphold newly adopted language policies and cement the
role of local languages in the region. This necessity calls for the
development of large, freely available open-source datasets, up-to-date
language models, cutting edge research, yet requires tremendous efforts for
such low-resource languages. Therefore, being bound by the data availability,
researchers also explore alternative techniques, like transfer learning or
data augmentation, to compensate for the lack of thereof. However, both
approaches have limitations in terms of their applicability to different tasks
and architectures as well as effectiveness (Chen et al. 2021; Raffel et al.
2019).
Thus, the paper aims to provide an overview of the already available resources
along with the directions for potential future research. It also aims to
identify the priorities and actions to be taken for the successful movement of
the low-resource languages up the classification ladder, towards the bright
future of high resource. Overall, we hope that our work will paint a general
picture for anyone interested in the research on the topic.

Figure 1: Distributions of numbers of Kazakh, Uzbek, Kyrgyz, and Turkmen
native speakers among all Turkic language speakers. Source: Wikipedia
contributors 2024.

##  2 Related Work

Recently, a significant amount of work directed at unifying the domain
knowledge of Turkic languages has emerged. Frameworks for their evaluation
have been created (Akın and Akın, 2007) alongside suites for their linguistic
analysis Abdurakhmonova et al. (2022) or overall assessments of the latest
technology Mirzakhalov et al. (2021). However, there has been little to no
comprehensive research summarizing the state of all the available data and
developed language processing tools for Turkic languages neither in general
nor for the Central Asian subset of those. While speech recognition, machine
translation, and other technologies have been developed and reached state-of-
the-art for some languages in the area, no cross-linguistic comparison was
conducted ((Bekarystankyzy et al., 2024); (Yeshpanov et al., 2024b)). Thus, a
comprehensive survey could prove crucial for both establishing a comparative
base for this language subset and identifying areas for future improvements
and development.

Feature | Turkish | Kazakh | Kyrgyz | Uzbek | Turkmen
---|---|---|---|---|---
Number of vowels | 8 | 12 | 8 | 6 | 9
Number of plural suffixes | 2 | 12 | 12 | 4 | 4
Number of pronouns | 6 | 8 | 8 | 8 | 6
Number of noun cases | 6 | 7 | 6 | 6 | 6
Number of personal verb suffixes | 5 | 9 | 6 | 9 | 5
Word order | SOV | SOV | SOV | SOV | SOV
Table 1: High-level overview of the Turkic Central Asian languages’
differences. Sources: Adali , Johanson .

##  3 Difficulties in Processing Turkic Languages

###  3.1 Overview

With the expansion of the application of neural networks, exact linguistic
topology became less relevant. However, transfer learning from similar
language families has been one of the most prominent solutions in the sphere
of low-resource language processing Zoph et al. (2016). Thus, a summarization
of grammatical similiarities and differences might prove beneficial in this
context. While in depth linguistic comparison of each language with their
richer counterpart, Turkish, lies beyond the scope of the paper, a basic
summary is provided to identify potential difficulties in the application of
transfer learning from Turkish.
Turkic languages are primarily described as morphologically rich languages.
Over the past years, several studies revolving around specifics of processing
these languages’ complex morphological structures and morpho-syntactic
features have been conducted Ciddi (2013). It has been shown that their highly
agglutinative nature leads to difficulties in the tasks of machine translation
(Alkim and Çebi 2019; Mirzakhalov et al. 2021) and named entity recognition
Küçük et al. (2017). On top of that, it is not exactly clear how these
pecularities will be reflected in the transfer learning applications.

###  3.2 Similarities and Differences

Main grammatical features that are most relevant for language processing (like
verb tense systems and word order) are common for all the languages. However,
there exist minor differences including heavier reliance on the underlying
vowel harmony in Kazakh, Uzbek, Kyrgyz, and Turkmen – e.g. additional noun
cases and plural suffixes, with the exact choice depending on the preceding
syllable’s vowel.
The number of vowels and pronouns in those languages differs as well. As can
be seen in Table 1, one can also infer that Kazakh and Kyrgyz are
typologically closer to each other than Kazakh and Uzbek and Uzbek and Kyrgyz
respectively, while Turkmen is the closest to Turkish grammatically. This
might hint not only at potential differences in transfer learning from
Turkish, but also a potential for transfer learning with the group itself,
presumably making transfer learning application between certain pairs (like
Kazakh and Kyrgyz) more successful than others (Kazakh and Turkmen, for
example).
There is also another substantial difference between some of the languages:
the script and inconsistency of its usage. For example, while Uzbek is written
using Latin script, Kazakh, despite the Kazakhstani government’s best efforts
to also employ Latin script for the language, is still primarily written in
Cyrillic. It is possible that these script inconsistencies might prevent
transfer learning application from reaching its full potential (Gheini and May
2019; Amrhein and Sennrich 2020). Thus, additional steps might be required at
the level of data cleaning or pre-processing before running the models.
Overall, with this in mind, there is great potential for exploiting the
similiarity of the languages for the purpose of transfer learning either from
Turkish or from within the Turkic Central Asian language.

##  4 Datasets Availability

Availability of open-source data enables researchers to scale and reuse data,
reducing overall time spent on data collection and evaluation. However,
corpora access varies from low to almost non-existent for the languages
considered in the paper.

###  4.1 Sources of Data and Stakeholders

While developing a comprehensive dataset classification system would lie
beyond the scope of the current paper, it is important to identify potential
data collection sources and main stakeholders and actors of the process. While
a significant effort has been made by the researchers within the Central Asian
countries themselves (e.g., most notable research on Kazakh language is
contributed by Institute of Smart and Intelligent Systems (ISSAI) of
Nazarbayev University), outside effort is driven primarily by the studies of
NLP applications for Chinese minority languages Du et al. (2023) and for
Turkic languages in general Mirzakhalov et al. (2021). Additionally, while the
researchers focus on collection of handcrafted high-quality datasets, other,
potentially lower quality data, is available via multilingual datasets such as
Common Crawl (CC) or OSCAR Ortiz Suárez et al. (2020). Thus, there is a
greater variety of data sources and stakeholders than might initially seem.

###  4.2 Kazakh Language Datasets

Linguistic Features Datasets. Out of all four languages, the most diverse and
largest in quantity datasets are available for Kazakh. Besides collections of
morphologically and syntactically-annotated data like Almaty Corpus of Kazakh
Language and UD treebank (Tyers and Washington 2015; Makazhanov et al. 2015),
there are also more task-specific text corpora.
Task-specific Datasets. For example, there exists the KazNERD corpus, spanning
over 100,000 sentences with 25 entity classes for the task of Named Entity
Recognition Yeshpanov et al. (2022), the KazQAD corpus for open-domain
question answering with over 6000 questions Yeshpanov et al. (2024a), the
KazParC parallel corpus of Kazakh, English, Russian, and Turkish for machine
translation Yeshpanov et al. (2024b), and the KazSAnDRA dataset, containing
over 180 thousand reviews to be used for the task of sentiment analysis
(Yeshpanov and Varol, 2024).
Unannotated Datasets. Additionally, there is a substantial amount of
unannotated data available, including the KazNews corpus Yakunin et al.
(2021), which includes over 4 million publications from 36 local media
sources.
Multimodal Datasets. There are also numerous collections of multimodal data
for the language. Most notable and largest language dataset up until now is
Kazakh Speech Corpus 2 (KSC2), collected by Mussakhojayeva et al. (2022b),
which contains over 1,200 hours of transcribed audios, and which also
integrates research work on previously collected datasets, such as Kazakh
Speech Corpus (KSC), compiled by Khassanov et al. (2021). Another prominent
resource is the KazakhTTS2 dataset Mussakhojayeva et al. (2022a), which
extended its previous version, KazakhTTS Mussakhojayeva et al. (2021), to more
than 270 hours of recorded audio. Drawing from this work on speech, a speech
commands dataset was also recently collected Kuzdeuov et al. (2023).
Additionally, even more multimodal data is available in Kazakh for even more
peculiar applications; for example, Mukushev et al. (2022) have gathered over
40,000 video samples of 50 signers of Kazakh-Russian Sign Language, which was
also an expansion of a previous project by the same authors in 2020. Further
work around the topic includes recent development of the KazEmoTTS dataset
Abilbekov et al. (2024) that collects more than 54 thousands audio-text pairs
in over 5 emotional states. Several datasets were also collected for the
purpose of hand-written text recognition, some of them combining Kazakh and
Russian languages Abdallah et al. (2020), and some containing text inclusively
in Kazakh, for example, Kazakh Offline Handwritten Text Dataset Toiganbayeva
et al. (2022), with over 3000 exam papers available.
This variety of multimodal data can potentially cover a wide range of the data
needs, enabling efficient data reuse and eliminating the necessity to use
precious resources for additional data collection. For example, audios from
the speech corpus can be transcribed and used as text for the tasks of text
classification, text generation, information retrieval, and others. Thus, this
datasets contribute strongly to the future of Kazakh NLP.

###  4.3 Uzbek Language Datasets

Second place in terms of data availability belongs to Uzbek language, however,
the existence of data solely for this Central Asian language is much scarcer.
Linguistic Features Datasets. A distinctive feature of most of the Uzbek
datasets is their orientation at purely linguistic tasks; for example, there
are several datasets, such as UzWordnet Agostini et al. (2021) and SimRelUz
Salaev et al. (2022) that capture prominent semantic features of the language,
and others, such as a dataset collected by Sharipov et al. (2023), that
includes a variety of POS tags and syntactic features of this Central Asian
language.
Task-specific Datasets. There is substantial data for specific tasks, such as
sentiment analysis. For example, there exist a sentiment analysis dataset
collected by Kuriyozov and Matlatipov (2019) and one based on restaurant
reviews by Matlatipov et al. (2022). There also exists data for text
classification, mainly scraped from the news sources in Uzbek, one collected
by Rabbimov and Kobilov (2020) and another one by Kuriyozov et al. (2023).
Multimodal and Unannotated Datasets. Most prominent multimodal and/or
unannotated Uzbek datasets include an open-source Uzbek Speech Corpus Musaev
et al. (2021), Uzbek corpus, which includes a large collection of educational,
scientific, official, and artistic texts together with a morphological
database and dictionaries, and an Uzbek Community corpus, collected by Leipzig
University and containing over 660 thousand sentences and 9 million tokens.

###  4.4 Kyrgyz Language Datasets

Unannotated Datasets and Corpora Collections. The of the largest open-source
datasets for Kyrgyz is the Manas-UdS corpus of over 84 literary texts in 5
genres, marked with lemmas and parts of speeches Kasieva et al. (2020). For
most other datasets it is not possible to link them directly to publications,
however, they are available on Github, for example, KyrgyzNews dataset with
over 250 thousand scraped news and Some work has also been done by Leipzig
University, who collected over 3 million sentences from publicly available web
resources.
Task-specific datasets. Again, most of the task-specific datasets for Kyrgyz
language are only available via Github. For example, one of the NER is
currently under development by the researchers from Kyrgyz State Technical
University (KSTU). There is also an instruction following dataset, derived
from Stanford Alpaca and translated into Kyrgyz using ChatGPT and Google
Translate, available for research purposes Khakim Davurov (2023).

###  4.5 Turkmen Language Datasets

The data landscape of Turkmen language is even more scarce. Besides the
Leipzig University’s corpora of over 270 thousand sentences of web-scraped
data, other resources are practically non-existent. With only a few
dictionaries and poetry collections collected by enthusiasts and available in
an open-source fashion on Github, Turkmen language falls largely behind every
other language in Central Asia in terms of data availability.

###  4.6 Web-Scraped Datasets

All the above-mentioned languages are also available in the multilingual
datasets which are predominantly scraped from the web. In particular, all the
languages are available in CC100, WikiAnn Pan et al. (2017), and OSCAR
datasets. Most of other web-scraped datasets present online are either subsets
of the bigger multilingual datasets (like CC100 or OSCAR), or are scrapes from
newspapers and websites available on the Internet, with many of them not being
open-source, for example, the kkWaC dataset with over 139 million words
collected in Kazakh language. However, according to Kreutzer et al. (2022),
such datasets should be treated with caution, since there is a potential that
the human-judged quality of low-resource language data in them is
significantly lower. With only a few hundred labels available, having even
5-10 percent of mislabeled or grammatically incorrect sentences can
significantly impact performance of the models in those languages.

###  4.7 Multilingual Datasets

Another important source of data in Central Asian languages are the handpicked
multilingual datasets for Turkic languages. For example, a large corpus for
Turkic languages was collected by Baisa and Suchomel (2012) for training
morphological analyzers and disambiguators. Another example is the xSID
(Cross-lingual Slot and Intent Detection) dataset by van der Goot et al.
(2021), which, among other language families, covered Turkic languages. More
task-specific multilingual datasets featuring Central Asian languages include
the Common Voice dataset containing Kyrgyz audio recordings (Ardila et al.,
2020), the Belebele dataset for machine reading comprehension containing
Kazakh, Uzbek, and Kyrgyz (Bandarkar et al., 2023), the MuMiN dataset
(Multimodal Fact-Checked Misinformation Dataset) based on the scraped tweets
featuring Kazakh language (Nielsen and McConville, 2022), and the AM2iCo
dataset for the evaluation of word meaning in context (Liu et al., 2021).
Thus, Kazakh, Uzbek, and Kyrgyz make a prominent appearance in research
related to Turkic languages as well as in the web-scraped datasets, thanks to
their presence in the web domain.

###  4.8 Parallel Corpora

Additionally, we would like to comment on the availability of parallel corpora
for the studied languages. Besides the already mentioned Kazakh-Russian sign
language corpora, only a few parallel text datasets exist for Uzbek-Kazakh
(Allaberdiev et al., 2024) and Kazakh-Russian (Kozhirbayev and Islamgozhayev,
2023) machine translation and ASR. Parallel data, however, is crucial for
advancing one of the most prominent NLP technologies - translation tools.

###  4.9 Classifying Languages by Data Availability

The resources available for each language are far from abundant. If one was to
categorize them according to the classification suggested by Joshi et al.
(2020), Kazakh would most likely fit into “The Rising Star” category, with its
substantial presence in the web and good variety of multimodal datasets.
However, the research on this language is pushed back by a lack of labeled
data for the downstream tasks. Uzbek, given its recent developments and
greater variability in terms of the linguistic resources available, would be
categorized as “The Hopeful,” given that in the next years the efforts for
collecting the datasets will not fade. Kyrgyz and Turkmen, unfortunately, not
being sufficiently backed up by streamlined research efforts, would be
classified as “The Scraping-Bys,” with the future of their data collection
processes yet unclear.

##  5 Reasons of Data Scarcity

Reasons of Central Asian languages’ data scarcity include the wide-spread
usage of Russian language, limited access of the population to the internet,
and lack of educational and technological initiatives in the sphere of
artificial intelligence.

###  5.1 Russian Language in Central Asia

In the Soviet times Russian language was the dominant one across all the
Soviet republics. Since the fall of the Soviet Union, most of the republics
have embarked on the journey of rediscovering their national identities, which
led to significant changes in their language policies. However, according to
the studies of Fierman (2012), despite its substantial decline, Russian still
enjoys its privileged position in the spheres of science, education, politics,
and others. On top of that, Russian media, TV channels, and Internet resources
are freely available in Central Asia, unifying the information and
communication medium between the countries. While beneficial for the purpose
of intercultural communication, this situation is somewhat detrimental to the
development of natural language processing technologies for regional
languages. A fair share of contemporary datasets come from the web-scraped
data, and users from Central Asia have to heavily rely on the web sources in
Russian language, which undermines the development of their own national
languages online. The situation is identical in terms of media and news, which
leads to a limited variety of textual and multimedia data available in Kazakh,
Uzbek, Kyrgyz, and Turkmen.

###  5.2 Internet Access

Limited Internet access is the second reason for data scarcity in the region.
As stated before, while a substantial amount of data has been gathered by NLP
researchers from the Internet based sources, only 38 and 21 percent of users
in Kyrgyzstan and Turkmenistan respectively have access to the global net.
While the situation is somewhat better in Uzbekistan (with 55 percent of
population being able to access the medium) and significantly better in
Kazakhstan (around 79 percent of population), limited access to the Internet
prevents potential contribution of those users to the contents of open-source
encyclopedias, blogs, news sites, etc. Additionally, a lack of digitalization
in terms of electronic copies of books and journals, audio transcripts, and
video recordings limits the potential of information sharing within the
region.

###  5.3 Lack of Initiatives

Being a demanding and resource-greedy field, the area of machine learning and
natural language processing requires substantial and long-term financial
investments. Thus, only a few exclusively AI-dedicated initiatives have been
launched by the governments: for example, ISSAI (Institute of Smart Systems
and Artificial Intelligence) in Kazakhstan or High Technology Park in
Kyrgyzstan. However, these institutions are not solely dedicated to NLP
research, and cover a wide range of topics in the AI domain, including
robotics, IoT, computer vision technologies, and others. Thus, without a
tailored initiative, it is difficult for the researchers to specialize fully
in NLP.

##  6 Application of Transfer Learning

The situation when one of the languages in a language family is more
resourceful than others is not unique to the Turkic language family. This
opens great potential for the application of transfer learning from Turkish or
one of the Central Asian languages to other languages of the same family.
Usually, this is more attainable than collecting large task-specific datasets
from scratch.
Transfer learning was substantially studied in the domain of machine
translation, and the choice of parent language has been highlighted as an
important criteria for the technique application Zoph et al. (2016). Combining
transliteration and byte-pair encoding, Nguyen and Chiang (2017) proved that
this transfer learning approach might be suitable beyond high-resource to low-
resource pairs, extending to low-resource pairs, especially the ones belonging
to the agglutinative languages. Given greater availability of NLP tools for
Kazakh, transfer of this knowledge has been evaluated on some Turkic languages
lying beyond the scope of the current paper, like Tatar. Other studies on
transfer learning from Kazakh have also been conducted on the task of ASR
(Orel et al., 2023). Some of them also aimed at using Russian as a source
language, but the efforts proved to be less successful in the end (N. et al.,
2020).

##  7 Data Augmentation, Transliteration, and R-Drop Regularization

Apart from transfer learning, data augmentation, has also been somewhat
substantially researched for a part of Turkic languages. Other techniques,
like R-Drop regularization or transliteration, also proved useful for
performance enhancement.
A recent study on the topic of sentence augmentation using large language
models for Kazakh (Bimagambetova et al., 2023) showed that data augmentation
works well for already resourceful languages and does less successfully in the
low-resource domain, which might seem somewhat obvious. However, practical
applications using other data augmentation techniques, including phrase
replacement, proved to significantly improve the BLEU score of certain
language pairs, for example, Kazakh-Chinese (Wu and Ma, 2023). Data
augmentation with R-Drop regularization has also proved useful for the same
Kazakh-Chinese translation task in other studies (Liu et al., 2023).
Other techniques, like dropout or transliteration, are usually applied
together with transfer learning and data augmentation. Additionally, a
multilingual model can arguably also perform better on certain tasks than a
model used via transfer learning approach (Nugumanova et al., 2022). All these
numerous variants and techniques leave researchers a great space for
experimenting in the Central Asian language processing without the necessity
to collect additional data.

Task | Kazakh | Uzbek | Kyrgyz | Turkmen
---|---|---|---|---
Automatic Speech Recognition | ✓ | ✓ | ✓ | ✗
Machine Translation | ✓ | ✓ | ✓ | ✓
Named-Entity Recognition | ✓ | ✓ | ✓ | ✗
Text Generation | ✓ | ✗ | ✗ | ✗
Sentiment Analysis | ✓ | ✓ | ✗ | ✗
Text Classification | ✓ | ✓ | ✓ | ✗
POS Tagging | ✓ | ✓ | ✓ | ✗
Text Summarization | ✓ | ✓ | ✗ | ✗
Question Answering | ✓ | ✗ | ✗ | ✗
Table 2: Existence of downstream task research per language. Existence is
defined as at least one published research paper and/or dataset for the
specific language and/or the specific language in conjunction with other
closely related languages.

##  8 Current State of Technologies

###  8.1 Kazakh Language Technologies

As with the data availability, the first place in terms of the technology
available is taken by Kazakh language.
Linguistic Analysis. A variety of tools have been developed for linguistic
analysis and normalization of texts in Kazakh (Yessenbayev et al., 2020). This
research paved the way for first advances in the most prominent areas of NLP,
like machine translation and ASR, that are well represented in Kazakh
language.
Machine Translation. Almost all the datasets mentioned in the previous
sections have been released together with the evaluation benchmarks for
certain tasks on either already pre-trained models like mBERT (Yeshpanov et
al., 2022) or together with completely new systems like Tilmash, which enables
a two-way translation between for 4 languages, including Kazakh and Turkish
(Yeshpanov et al., 2024b). The latter has proved to be comparable or even
better at translating language pairs involving Kazakh than translation
technologies developed by Google and Yandex, which dominate the scene of
machine translation in the region.
ASR. Another important advancement in processing of Kazakh language lies in
the sphere of automatic speech recognition. For that purpose, researchers have
been actively leveraging the KazakhTTS and KazakhTTS2 datasets as well as the
recently available KazEmoTTS dataset. For example, a Turkic ASR system that
leverages the KazakhTTS, USC, and Common Voice data and covers, among other
Turkic languages, Kazakh, Uzbek, and Kyrgyz, has been developed by the authors
of the KazakhTTS dataset (Mussakhojayeva et al., 2022a). The most recent
research on exclusively Kazakh language managed to bring down the word error
rate to just 7.2 percent (Bekarystankyzy et al., 2023) as well as to summarize
difficulties and explore deep learning techniques on end-to-end speech
recognition of agglutinative languages (Bekarystankyzy et al., 2024).
Additionally, research on speech commands recognition has been conducted
(Kuzdeuov et al., 2023).
Fine-tuning Existing Models. Regarding the usage and fine-tuning of already
existing models, several tasks have been evaluated for Kazakh. For example, on
the KazNERD dataset (Yeshpanov et al., 2022), the fine-tuned XLM-RoBERTa
demonstrated a micro average precision score of an impressive 97.09 percent,
and on the task of sentiment analysis the same model gained an F1 score of
0.87.
Understudied Areas. However, despite the above-mentioned efforts, certain
areas of Kazakh NLP remain significantly understudied. For example, the
particularly hot field of text generation. Only a few but hopeful experiments
have been conducted in recent years (Tolegen et al., 2023) regarding the usage
of large language models for the benefit of Kazakh language.

###  8.2 Uzbek Language Technologies

Alike in terms of data availability, Uzbek language gains the second place in
terms of available technology.
ASR. While Uzbek does not enjoy the same variety of machine translation tools
as Kazakh, presence of automatic speech recognition technology is somewhat
comparable, with the most recent work contributed by Musaev et al. (2021),
with their model performing at the level of 14.3 percent word error rate.
Fine-Tuning Existing Models. Uzbek also enjoys a better variety of pre-trained
and fine-tuned models, including UzBERT (Mansurov and Mansurov, 2021), capable
of outperforming mBERT by leveraging at least 11 times more of language
specific data; UzRoberta, pre-trained on roughly 2 million news articles
(Davronov and Adilova, 2024); BERTBek, a model, improving on the UzBERT
decision by training on Latin script (Kuriyozov et al., 2024), and a compact
and fine-tuned variation of a mT5 model. In general, with Uzbek still catching
up on the data availability, there is potential for the language to soon enjoy
at least the same status of the NLP technology available as for Kazakh.

###  8.3 Kyrgyz and Turkmen Languages Technology

Unfortunately, the situation for Kyrgyz and Turkmen looks less promising. With
little to no work in terms of machine translation or automatic speech
recognition Kyrgyz enjoys little variety of technology for text classification
(Alekseev et al., 2023)) and NER fine-tuned on WikiAnn data. Turkmen offers
even less language-specific technology, since it is mainly studied within the
scope of comparative studies of Turkic languages. For example, some work on
machine translation evaluation was done in the effort to build the
infrastructure for Turkic languages, but now this approach seems already
outdated (Alkim and Çebi, 2019). Overall, neither further fine-tuning or pre-
training of already available models or exploitation of statistical approaches
seems to have been researched for the majority of the basic NLP tasks both in
Kyrgyz and Turkmen.

##  9 Future Work Areas

Kazakh Language. Due to its variety of multimodal data, Kazakh language can
potentially already expand on the existing work and move onto fine-tuning and
developing more advanced models for other downstream tasks, e.g. text
generation or question answering. Additionally, transfer learning from Kazakh
should be further explored as a potential workaround for the problems of its
less resourceful brothers, with more focus on linguistically closer languages
of the Turkic family, like Kyrgyz.
Uzbek Language. In contrast, Uzbek language clearly requires more data to be
collected for the creation of the systems like the ones built on top of
datasets like KazakhTTS. With already developed pre-trained models like
UzBERT, there is a need for research into their further application and
comparison with other technologies available. Generally, leveraging the rich
Uzbek linguistic features datasets in combination with further efforts of
large-scale data collection paint a bright future for the language.
Kyrgyz and Turkmen languages. The current situation for Kyrgyz and Turkmen
requires significant efforts for data collection and aggregation first. With
the amount of data available for both languages, it seems barely useful not
only to pre-train models like BERT, but also to research potential
applications of statistical algorithms. In parallel with data collection,
studies on transfer learning from Kazakh or Turkish might prove beneficial,
given the grammatical similarity between the corresponding language pairs.
Additionally, given the incorporation of LLMs in the field, potential of using
them for data augmentation or annotation might happen to be another important
direction of research in comparison with more expensive methods of human data
collection and annotation.

##  10 Conclusion

Significant improvements in the processing of Turkic Central Asian languages
have been achieved in the recent years. However, there still exists an
imbalance in the amount of available data and technology, with Kazakh and
Uzbek dominating the scene, and Kyrgyz and Turkmen requiring significant
efforts in terms of data collection. Besides leveraging already existing
datasets or web-sources for curating new data using data augmentation, other
potentially successful options for technological advancements include transfer
learning from Kazakh to Uzbek, Kyrgyz, and Turkmen. Already existing data in
Kazakh might prove useful for the studies of other Central Asian languages,
given their close linguistic relatedness.
While significant efforts are yet to emerge and be maintained for Kazakh,
Uzbek, Kyrgyz, and Turkmen to reach the level of winners or underdogs (Joshi
et al., 2020), a substantial work has already been developed in the most
crucial areas of natural language processing, including the automatic speech
recognition systems and machine translation. With the potential of
transferring this experience to other tasks, such as information retrieval,
text generation, and question answering, there is a greater hope for progress
towards the state-of-the-art technology for these Central Asian languages.

##  11 Acknowledgements

We thank Dr. Mareike Hartmann for valuable feedback and discussion about the
paper.

## References

  * Abdallah et al. (2020) Abdelrahman Abdallah, Mohamed Hamada, and Daniyar Nurseitov. 2020.  Attention-based fully gated cnn-bgru for russian handwritten text.  _Journal of Imaging_ , 6(12):141.
  * Abdurakhmonova et al. (2022) Nilufar Z. Abdurakhmonova, Alisher S. Ismailov, and Davlatyor Mengliev. 2022.  Developing nlp tool for linguistic analysis of turkic languages.  In _2022 IEEE International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON)_ , pages 1790–1793.
  * Abilbekov et al. (2024) Adal Abilbekov, Saida Mussakhojayeva, Rustem Yeshpanov, and Huseyin Atakan Varol. 2024.  Kazemotts: A dataset for kazakh emotional text-to-speech synthesis.  _Preprint_ , arXiv:2404.01033.
  * (4) Eşref Adali.  Similarities and differences of turkic languages.
  * Agostini et al. (2021) Alessandro Agostini, Timur Usmanov, Ulugbek Khamdamov, Nilufar Abdurakhmonova, and Mukhammadsaid Mamasaidov. 2021.  UZWORDNET: A lexical-semantic database for the Uzbek language.  In _Proceedings of the 11th Global Wordnet Conference_ , pages 8–19, University of South Africa (UNISA). Global Wordnet Association.
  * Akın and Akın (2007) Ahmet Afsin Akın and Mehmet Dündar Akın. 2007.  Zemberek, an open source nlp framework for turkic languages.  _Structure_ , 10(2007):1–5.
  * Alekseev et al. (2023) Anton M. Alekseev, Sergey I. Nikolenko, and Gulnara Kabaeva. 2023.  Benchmarking multilabel topic classification in the kyrgyz language.  _ArXiv_ , abs/2308.15952.
  * Alkim and Çebi (2019) Emel Alkim and Yalçin Çebi. 2019.  Machine translation infrastructure for turkic languages (mt-turk).  _Int. Arab J. Inf. Technol._ , 16:380–388.
  * Allaberdiev et al. (2024) Bobur Allaberdiev, Gayrat Matlatipov, Elmurod Kuriyozov, and Zafar Rakhmonov. 2024.  Parallel texts dataset for uzbek-kazakh machine translation.  _Data in Brief_ , 53:110194.
  * Amrhein and Sennrich (2020) Chantal Amrhein and Rico Sennrich. 2020.  On Romanization for model transfer between scripts in neural machine translation.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 2461–2469, Online. Association for Computational Linguistics.
  * Ardila et al. (2020) Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber. 2020.  Common voice: A massively-multilingual speech corpus.  In _Proceedings of the Twelfth Language Resources and Evaluation Conference_ , pages 4218–4222, Marseille, France. European Language Resources Association.
  * Baisa and Suchomel (2012) Vít Baisa and Vít Suchomel. 2012.  Large corpora for turkic languages and unsupervisedmorphological analysis.
  * Bandarkar et al. (2023) Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, and Madian Khabsa. 2023.  The belebele benchmark: a parallel reading comprehension dataset in 122 language variants.  _Preprint_ , arXiv:2308.16884.
  * Bekarystankyzy et al. (2024) Akbayan Bekarystankyzy, Orken Mamyrbayev, and Tolganay Anarbekova. 2024.  Integrated end-to-end automatic speech recognition for languages for agglutinative languages.  _ACM Trans. Asian Low-Resour. Lang. Inf. Process._ Just Accepted.
  * Bekarystankyzy et al. (2023) Akbayan Bekarystankyzy, Orken Zh. Mamyrbayev, DO Oralbekova, and Bagashar Zhumazhanov. 2023.  Transfer learning for an integrated low-data automatic speech recognition system.
  * Bimagambetova et al. (2023) Zhamilya Bimagambetova, Dauren Rakhymzhanov, Assel Jaxylykova, and Alexander Pak. 2023.  Evaluating large language models for sentence augmentation in low-resource languages: A case study on kazakh.  _2023 19th International Asian School-Seminar on Optimization Problems of Complex Systems (OPCS)_ , pages 14–18.
  * Chen et al. (2021) Jiaao Chen, Derek Tam, Colin Raffel, Mohit Bansal, and Diyi Yang. 2021.  An empirical survey of data augmentation for limited data learning in nlp.  _Preprint_ , arXiv:2106.07499.
  * Ciddi (2013) Sibel Ciddi. 2013.  _Processing of Turkic Languages_.  Ph.D. thesis.
  * Davronov and Adilova (2024) Rifkat Davronov and Fatima Adilova. 2024.  UzRoberta: A pre-trained language model for Uzbek.  _AIP Conference Proceedings_ , 3004(1):050001.
  * Du et al. (2023) Wenqiang Du, Yikeremu Maimaitiyiming, Mewlude Nijat, Lantian Li, Askar Hamdulla, and Dong Wang. 2023.  Automatic speech recognition for uyghur, kazakh, and kyrgyz: An overview.  _Applied Sciences_ , 13(1).
  * Fierman (2012) W. Fierman. 2012.  Russian in post-soviet central asia: A comparison with the states of the baltic and south caucasus.  _Europe-Asia Studies_ , 64:1077 – 1100.
  * Gheini and May (2019) Mozhdeh Gheini and Jonathan May. 2019.  A universal parent model for low-resource neural machine translation transfer.  _Preprint_ , arXiv:1909.06516.
  * (23) Lars Johanson.  https://www.britannica.com/topic/Turkic-languages. Accessed: 2024-7-5.
  * Joshi et al. (2020) Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020.  The state and fate of linguistic diversity and inclusion in the NLP world.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 6282–6293, Online. Association for Computational Linguistics.
  * Kasieva et al. (2020) Aida Kasieva, Jörg Knappen, Stefan Fischer, and Elke Teich. 2020.  A new kyrgyz corpus: sampling, compilation, annotation.
  * Khakim Davurov (2023) Ulan Abdurazakov Khakim Davurov, Timur Turatali. 2023.  Kyrgyz alpaca: Models and datasets.  https://github.com/Akyl-AI/kyrgyz-alpaca.
  * Khassanov et al. (2021) Yerbolat Khassanov, Saida Mussakhojayeva, Almas Mirzakhmetov, Alen Adiyev, Mukhamet Nurpeiissov, and Huseyin Atakan Varol. 2021.  A crowdsourced open-source kazakh speech corpus and initial speech recognition baseline.  _Preprint_ , arXiv:2009.10334.
  * Kozhirbayev and Islamgozhayev (2023) Zhanibek Kozhirbayev and Talgat Islamgozhayev. 2023.  Cascade speech translation for the kazakh language.  _Applied Sciences (Switzerland)_ , 13(15).  Publisher Copyright: © 2023 by the authors.
  * Kreutzer et al. (2022) Julia Kreutzer, Isaac Caswell, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii-Orshikh, Allahsera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, Monang Setyawan, Supheakmungkol Sarin, Sokhar Samb, Benoît Sagot, Clara Rivera, Annette Rios, Isabel Papadimitriou, Salomey Osei, Pedro Ortiz Suarez, Iroro Orife, Kelechi Ogueji, Andre Niyongabo Rubungo, Toan Q. Nguyen, Mathias Müller, André Müller, Shamsuddeen Hassan Muhammad, Nanda Muhammad, Ayanda Mnyakeni, Jamshidbek Mirzakhalov, Tapiwanashe Matangira, Colin Leong, Nze Lawson, Sneha Kudugunta, Yacine Jernite, Mathias Jenny, Orhan Firat, Bonaventure F. P. Dossou, Sakhile Dlamini, Nisansa de Silva, Sakine Çabuk Ballı, Stella Biderman, Alessia Battisti, Ahmed Baruwa, Ankur Bapna, Pallavi Baljekar, Israel Abebe Azime, Ayodele Awokoya, Duygu Ataman, Orevaoghene Ahia, Oghenefego Ahia, Sweta Agrawal, and Mofetoluwa Adeyemi. 2022.  Quality at a glance: An audit of web-crawled multilingual datasets.  _Transactions of the Association for Computational Linguistics_ , 10:50–72.
  * Kuriyozov and Matlatipov (2019) Elmurod Kuriyozov and Sanatbek Matlatipov. 2019.  Building a new sentiment analysis dataset for uzbek language and creating baseline models.  _Proceedings_.
  * Kuriyozov et al. (2023) Elmurod Kuriyozov, Ulugbek Salaev, Sanatbek Matlatipov, and Gayrat Matlatipov. 2023.  Text classification dataset and analysis for uzbek language.  _Preprint_ , arXiv:2302.14494.
  * Kuriyozov et al. (2024) Elmurod Kuriyozov, David Vilares, and Carlos Gómez-Rodríguez. 2024.  BERTbek: A pretrained language model for Uzbek.  In _Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024_ , pages 33–44, Torino, Italia. ELRA and ICCL.
  * Kuzdeuov et al. (2023) Askat Kuzdeuov, Shakhizat Nurgaliyev, Diana Turmakhan, Nurkhan Laiyk, and Huseyin Atakan Varol. 2023.  Speech Command Recognition: Text-to-Speech and Speech Corpus Scraping Are All You Need.
  * Küçük et al. (2017) Dogan Küçük, Nursal Arıcı, and Dilek Küçük. 2017.  Named entity recognition in turkish: Approaches and issues.  In _International Conference on Applications of Natural Language to Data Bases_.
  * Liu et al. (2023) Canglan Liu, Wushouer Silamu, and Yanbing Li. 2023.  A chinese–kazakh translation method that combines data augmentation and r-drop regularization.  _Applied Sciences_ , 13(19).
  * Liu et al. (2021) Qianchu Liu, Edoardo Maria Ponti, Diana McCarthy, Ivan Vulić, and Anna Korhonen. 2021.  AM2iCo: Evaluating word meaning in context across low-resource languages with adversarial examples.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 7151–7162, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Makazhanov et al. (2015) Aibek Makazhanov, Aitolkyn Sultangazina, Olzhas Makhambetov, and Zhandos Yessenbayev. 2015.  Syntactic annotation of kazakh: Following the universal dependencies guidelines. a report.  In _3rd International Conference on Turkic Languages Processing, (TurkLang 2015)_ , pages 338–350.
  * Mansurov and Mansurov (2021) B. Mansurov and A. Mansurov. 2021.  Uzbert: pretraining a bert model for uzbek.  _Preprint_ , arXiv:2108.09814.
  * Matlatipov et al. (2022) Sanatbek Matlatipov, Hulkar Rahimboeva, Jaloliddin Rajabov, and Elmurod Kuriyozov. 2022.  Uzbek sentiment analysis based on local restaurant reviews.  _Preprint_ , arXiv:2205.15930.
  * Mirzakhalov et al. (2021) Jamshidbek Mirzakhalov, Anoop Babu, Duygu Ataman, Sherzod Kariev, Francis Tyers, Otabek Abduraufov, Mammad Hajili, Sardana Ivanova, Abror Khaytbaev, Antonio Laverghetta Jr., Bekhzodbek Moydinboyev, Esra Onal, Shaxnoza Pulatova, Ahsan Wahab, Orhan Firat, and Sriram Chellappan. 2021.  A large-scale study of machine translation in Turkic languages.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 5876–5890, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Mukushev et al. (2022) Medet Mukushev, Aigerim Kydyrbekova, Vadim Kimmelman, and Anara Sandygulova. 2022.  Towards large vocabulary kazakh-russian sign language dataset: Krsl-onlineschool.  _10th Workshop on the Representation and Processing of Sign Languages: Multilingual Sign Language Resources, sign-lang 2022_.
  * Musaev et al. (2021) Muhammadjon Musaev, Saida Mussakhojayeva, Ilyos Khujayorov, Yerbolat Khassanov, Mannon Ochilov, and Huseyin Atakan Varol. 2021.  Usc: An open-source uzbek speech corpus and initial speech recognition experiments.  _Preprint_ , arXiv:2107.14419.
  * Mussakhojayeva et al. (2021) Saida Mussakhojayeva, Aigerim Janaliyeva, Almas Mirzakhmetov, Yerbolat Khassanov, and Huseyin Atakan Varol. 2021.  Kazakhtts: An open-source kazakh text-to-speech synthesis dataset.  _ArXiv_ , abs/2104.08459.
  * Mussakhojayeva et al. (2022a) Saida Mussakhojayeva, Yerbolat Khassanov, and Huseyin Atakan Varol. 2022a.  KazakhTTS2: Extending the open-source Kazakh TTS corpus with more data, speakers, and topics.  In _Proceedings of the Thirteenth Language Resources and Evaluation Conference_ , pages 5404–5411, Marseille, France. European Language Resources Association.
  * Mussakhojayeva et al. (2022b) Saida Mussakhojayeva, Yerbolat Khassanov, and Huseyin Atakan Varol. 2022b.  Ksc2: An industrial-scale open-source kazakh speech corpus.  In _Interspeech_.
  * N. et al. (2020) Amirgaliyev E. N., Kuanyshbay D. N., and Baimuratov O. 2020.  Development of automatic speech recognition for kazakh language using transfer learning.  _Preprint_ , arXiv:2003.04710.
  * Nguyen and Chiang (2017) Toan Q. Nguyen and David Chiang. 2017.  Transfer learning across low-resource, related languages for neural machine translation.  _Preprint_ , arXiv:1708.09803.
  * Nielsen and McConville (2022) Dan Saattrup Nielsen and Ryan McConville. 2022.  Mumin: A large-scale multilingual multimodal fact-checked misinformation social network dataset.  _Preprint_ , arXiv:2202.11684.
  * Nugumanova et al. (2022) Aliya Nugumanova, Yerzhan Baiburin, and Yermek Alimzhanov. 2022.  Sentiment analysis of reviews in kazakh with transfer learning techniques.  _2022 International Conference on Smart Information Systems and Technologies (SIST)_ , pages 1–6.
  * Orel et al. (2023) Daniil Orel, Rustem Yeshpanov, and Huseyin Atakan Varol. 2023.  Speech recognition for turkic languages using cross-lingual transfer learning from kazakh.  _2023 IEEE International Conference on Big Data and Smart Computing (BigComp)_ , pages 174–182.
  * Ortiz Suárez et al. (2020) Pedro Javier Ortiz Suárez, Laurent Romary, and Benoît Sagot. 2020.  A monolingual approach to contextualized word embeddings for mid-resource languages.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1703–1714, Online. Association for Computational Linguistics.
  * Pan et al. (2017) Xiaoman Pan, Boliang Zhang, Jonathan May, Joel Nothman, Kevin Knight, and Heng Ji. 2017.  Cross-lingual name tagging and linking for 282 languages.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1946–1958, Vancouver, Canada. Association for Computational Linguistics.
  * Rabbimov and Kobilov (2020) Ilyos Rabbimov and S S Kobilov. 2020.  Multi-class text classification of uzbek news articles using machine learning.  _Journal of Physics: Conference Series_ , 1546.
  * Raffel et al. (2019) Colin Raffel, Noam M. Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _J. Mach. Learn. Res._ , 21:140:1–140:67.
  * Salaev et al. (2022) Ulugbek Salaev, Elmurod Kuriyozov, and Carlos Gómez-Rodríguez. 2022.  Simreluz: Similarity and relatedness scores as a semantic evaluation dataset for uzbek language.  _Preprint_ , arXiv:2205.06072.
  * Sharipov et al. (2023) Maksud Sharipov, Elmurod Kuriyozov, Ollabergan Yuldashev, and Ogabek Sobirov. 2023.  Uzbektagger: The rule-based pos tagger for uzbek language.  _ArXiv_ , abs/2301.12711.
  * Toiganbayeva et al. (2022) Nazgul Toiganbayeva, Mahmoud Kasem, Galymzhan Abdimanap, Kairat Bostanbekov, Abdelrahman Abdallah, Anel Alimova, and Daniyar Nurseitov. 2022.  Kohtd: Kazakh offline handwritten text dataset.  _Signal Processing: Image Communication_ , 108:116827.
  * Tolegen et al. (2023) Gulmira Tolegen, Alymzhan Toleu, Rustam Mussabayev, Bagashar Zhumazhanov, and Gulzat Ziyatbekova. 2023.  Generative pre-trained transformer for kazakh text generation tasks.  _2023 19th International Asian School-Seminar on Optimization Problems of Complex Systems (OPCS)_ , pages 1–5.
  * Tyers and Washington (2015) Francis M. Tyers and Jonathan N. Washington. 2015.  Towards a free/open-source universal-dependency treebank for kazakh.  In _3rd International Conference on Turkic Languages Processing, (TurkLang 2015)_ , pages 276–289.
  * van der Goot et al. (2021) Rob van der Goot, Ibrahim Sharaf, Aizhan Imankulova, Ahmet Üstün, Marija Stepanovic, Alan Ramponi, Siti Oryza Khairunnisa, Mamoru Komachi, and Barbara Plank. 2021.  From masked-language modeling to translation: Non-English auxiliary tasks improve zero-shot spoken language understanding.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , Mexico City, Mexico. Association for Computational Linguistics.
  * Wikipedia contributors (2024) Wikipedia contributors. 2024.  Languages of asia — Wikipedia, the free encyclopedia.  [Online; accessed 5-July-2024].
  * Wu and Ma (2023) Hao Wu and Beiqiang Ma. 2023.  Kazakh-chinese neural machine translation based on data augmentation.  In _Conference on Computer Graphics, Artificial Intelligence, and Data Processing_.
  * Yakunin et al. (2021) Kirill Yakunin, Maksat N. Kalimoldayev, Ravil I. Mukhamediev, Rustam Mussabayev, Vladimir B. Barakhnin, Yan I. Kuchin, Sanzhar Murzakhmetov, Timur Buldybayev, Ulzhan Ospanova, Marina Yelis, Akylbek Zhumabayev, Viktors I. Gopejenko, Zhazirakhanym Meirambekkyzy, and A. Abdurazakov. 2021.  Kaznewsdataset: Single country overall digital mass media publication corpus.  _Data_ , 6:31.
  * Yeshpanov et al. (2024a) Rustem Yeshpanov, Pavel Efimov, Leonid Boytsov, Ardak Shalkarbayuli, and Pavel Braslavski. 2024a.  Kazqad: Kazakh open-domain question answering dataset.  _Preprint_ , arXiv:2404.04487.
  * Yeshpanov et al. (2022) Rustem Yeshpanov, Yerbolat Khassanov, and Huseyin Atakan Varol. 2022.  KazNERD: Kazakh named entity recognition dataset.  In _Proceedings of the Thirteenth Language Resources and Evaluation Conference_ , pages 417–426, Marseille, France. European Language Resources Association.
  * Yeshpanov et al. (2024b) Rustem Yeshpanov, Alina Polonskaya, and Huseyin Atakan Varol. 2024b.  Kazparc: Kazakh parallel corpus for machine translation.  _Preprint_ , arXiv:2403.19399.
  * Yeshpanov and Varol (2024) Rustem Yeshpanov and Huseyin Atakan Varol. 2024.  Kazsandra: Kazakh sentiment analysis dataset of reviews and attitudes.  _Preprint_ , arXiv:2403.19335.
  * Yessenbayev et al. (2020) Zhandos Yessenbayev, Zhanibek Kozhirbayev, and Aibek Makazhanov. 2020.  Kaznlp: A pipeline for automated processing of texts written in kazakh language.  In _Speech and Computer_ , pages 657–666, Cham. Springer International Publishing.
  * Zoph et al. (2016) Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016.  Transfer learning for low-resource neural machine translation.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 1568–1575, Austin, Texas. Association for Computational Linguistics.