# NusaCrowd: Open Source Initiative for Indonesian NLP Resources

Samuel Cahyawijaya♠,1,2, Holy Lovenia♠,1,2, Alham Fikri Aji♠,3, Genta Indra
Winata♠,4,
Bryan Wilie♠,1,2, Fajri Koto♠,3,2, Rahmad Mahendra5,2, Christian Wibisono6,
Ade Romadhony7,2, Karissa Vincentio8,2, Jennifer Santoso9, David Moeljadi10,
Cahya Wirawan12, Frederikus Hudi11,20, Muhammad Satrio Wicaksono13,,
Ivan Halim Parmonangan14, Ika Alfina5, Ilham Firdausi Putra13, Samsul
Rahmadani15,
Yulianti Oenang13, Ali Akbar Septiandri16, James Jaya13, Kaustubh D. Dhole17,
Arie Ardiyanti Suryani7, Rifki Afina Putri18, Dan Su1, Keith Stevens19,
Made Nindyatama Nityasya13, Muhammad Farid Adilazuarda6, Ryan Ignatius13,
Ryandito Diandaru6,Vito Ghifari6, Tiezheng Yu1, Wenliang Dai1, Yan Xu1,
Dyah Damapuspita5, Haryo Akbarianto Wibowo13, Cuk Tho14,
Ichwanul Muslim Karo Karo21, Tirana Noor Fatyanosa22, Ziwei Ji1, Graham
Neubig23,
Timothy Baldwin3, Sebastian Ruder24, Pascale Fung1, Herry Sujaini25,2,
Sakriani Sakti26,11, Ayu Purwarianti6,27,2
♠Main Authors
1HKUST 2INACL 3MBZUAI 4Bloomberg 5Universitas Indonesia
6Institut Teknologi Bandung 7Telkom University 8JULO 9University of Tsukuba
10Kanda University of International Studies 11NAIST 12AI-Research.id
13Independent Researcher 14BINUS 15Bahasa.ai 16Universitas Al Azhar Indonesia
17Emory University 18KAIST 19Surface Data 20 Works Applications
21State University of Medan 22Kumamoto University 23CMU 24Google
25Tanjungpura University 26JAIST 27Prosa.ai

###### Abstract

We present NusaCrowd, a collaborative initiative to collect and unify existing
resources for Indonesian languages, including opening access to previously
non-public resources. Through this initiative, we have brought together 137
datasets and 118 standardized data loaders. The quality of the datasets has
been assessed manually and automatically, and their value is demonstrated
through multiple experiments. NusaCrowd’s data collection enables the creation
of the first zero-shot benchmarks for natural language understanding and
generation in Indonesian and the local languages of Indonesia. Furthermore,
NusaCrowd brings the creation of the first multilingual automatic speech
recognition benchmark in Indonesian and the local languages of Indonesia. Our
work strives to advance natural language processing (NLP) research for
languages that are under-represented despite being widely spoken.

##  1 Introduction

Indonesia is one of the most linguistically diverse and populous countries in
the world, with over 270 million people living across 18,000+ islands. It
covers more than 700 spoken languages, making up $\sim$10% of all languages in
the world Grimes (2000); Lewis (2009); Cohn and Ravindranath (2014). However,
the progress of NLP research in Indonesian languages has been held back by
factors including language diversity Anderbeck (2008); Haryono (2012); Siregar
et al. (2014); Fauzi and Puspitorini (2018), orthographic variation Soeparno
(2015), resource limitation Wilie et al. (2020); Koto et al. (2020b), and
other societal challenges Nurjanah (2018); Jahang and Meirina (2021); Aji et
al. (2022).

Figure 1: System architecture of NusaCrowd. Open access to the datasheets is
provided through NusaCatalogue, while dataloader scripts to access the
resources are implemented in NusaCrowd Data Hub.

Existing NLP research mainly focuses on high-resource languages Wang et al.
(2018); Xu et al. (2020); Ruder (2022), while the vast majority of languages
with limited data—including most languages spoken in Indonesia—are neglected
Joshi et al. (2020). Specifically, many Indonesian NLP resources are
scattered, undocumented, and not publicly available. These issues cause a
severe data scarcity problem, which hinders NLP research in Indonesian and
other local languages spoken in Indonesia from progressing.

In this work, we introduce NusaCrowd,111NusaCrowd is a portmanteau of the
words Nusantara and Crowd. The word Nusantara is derived from an old Javanese
term referring to the territories of the Majapahit empire that corresponds to
present-day Indonesia. an open collaborative effort to gather and unify
existing resources in Indonesian languages for public use, and liberate non-
public resources. This initiative has successfully collected a total of 137
datasheets with 118 standardized data loaders in NusaCrowd Data Hub222We
publicly release NusaCrowd’s data hub at https://github.com/IndoNLP/nusa-crowd
and the NusaCatalogue at https://indonlp.github.io/nusa-catalogue/. The
datasets were manually assessed for data quality by multiple native speakers
and experts in NLP. Utilizing the datasets collected in NusaCrowd, we
introduce the first zero-shot NLU benchmark (NusaNLU), zero-shot NLG benchmark
(NusaNLG), and multilingual ASR benchmark (NusaASR) for Indonesian languages.
We evaluate various Indonesian and multilingual models on the benchmarks.

Our contributions can be summarized as follows:

  * •

We introduce the first large-scale resource hub of standardized Indonesian
corpora, covering 100+ datasets and 200+ tasks, spanning 19 Indonesian
languages in text, speech, and image modalities. As part of this, we provide
first-time access to 14 previously private datasets.

  * •

We develop the first Indonesian multilingual zero-shot benchmarks for natural
language understanding (NusaNLU) and natural language generation (NusaNLG),
which cover 40 NLU and NLG tasks in 12 languages.

  * •

We conduct a comprehensive analysis of the collected datasets across various
factors. Our analysis reflects the quality and diversity of existing NLP
datasets in Indonesian and other languages spoken in the region.

  * •

For speech, our initiative opens up access to a wide variety of ASR corpora
($\sim$800 hours) covering 10 Indonesian languages. Using these resources, we
build NusaASR and develop various Indonesian monolingual and multilingual ASR
models.

##  2 Related Work

#### Indonesian NLP Resources

The lack of labeled datasets for training and evaluation has impeded the
advancement of NLP research in Indonesian languages Aji et al. (2022). As a
result, research has focused on using unlabeled data by building large
language models (LLMs) to enable zero-shot and few-shot transfer learning. In
recent years, multiple efforts have worked on language models (LMs) in
Indonesian languages by exploring and developing different LM structures.
Several efforts have focused on encoder-only LMs, such as IndoBERT Wilie et
al. (2020); Koto et al. (2020b), SundaBERT Wongso et al. (2022), and IndoBERT-
Tweet Koto et al. (2021). Elsewhere, a number of generative models have been
proposed, i.e., IndoBART and IndoGPT, along with the generation task
benchmark, IndoNLG Cahyawijaya et al. (2021b).

#### Open and Community-based Initiatives

Open source/open science initiatives are a core part of the motivation behind
this paper. Large-scale collaborations have made their mark in various
research areas through developing a variety of resources, e.g., LMs Scao et
al. (2022); Muennighoff et al. (2022), datasets Ardila et al. (2020); Adelani
et al. (2021); Mager et al. (2021), catalogues Alyafeai et al. (2022); Altaher
et al. (2022); McMillan-Major et al. (2022), and benchmarks Srivastava et al.
(2022); Dhole et al. (2021); Fries et al. (2022).

##  3 NusaCrowd

In this section, we provide an overview of NusaCrowd, a detailed description
of the NusaCrowd framework, the dataset curation process, as well as a
detailed summary and statistics of the datasets contained in NusaCrowd.

###  3.1 Overview of NusaCrowd

NusaCrowd is a crowdsourcing initiative to collect, open-source, and
standardize access to datasets in Indonesian and 700+ local languages in
Indonesia. NusaCrowd aims to address the resource limitation problem in
Indonesian NLP across three dimensions: (1) complete datasheets for each
curated, ready-to-use dataset; (2) an open-access and centralized data hub for
accessing datasets through standardized data loading scripts; and (3)
promoting public data access for published non-public datasets. Through
promoting public data access, NusaCrowd provides access to 14 previously non-
public datasets, some of which are multilingual, covering a total of $\sim$40
tasks over 12 languages. It also serves as a portal for retrieving and loading
a wide variety of Indonesian NLP datasets, in text and other modalities (e.g.,
speech and images). NusaCrowd does not store or copy any of the hosted
datasets, and control and ownership of the hosted datasets belong to the
original owners.

Figure 2: Distribution of dataset curation approaches used in datasets
contained in NusaCrowd.

###  3.2 NusaCrowd Framework

As shown in Figure 1, NusaCrowd consists of two platforms: NusaCatalogue and
NusaCrowd Data Hub. The two platforms interact to support dataset registration
and provide a standardized pipeline for NusaCrowd. In general, NusaCatalogue
stores the datasheets (metadata) of all datasets, and NusaCrowd Data Hub
stores the standardized data loaders for all of the datasets. The two systems
share information about the datasheets and the data loaders, enabling users to
seamlessly explore and use the datasets.333All code in NusaCrowd will be made
publicly available under Apache License 2.0.

#### NusaCrowd Workflow

The dataset registration and standardization pipeline in NusaCrowd consists of
four stages: (1) submission of datasheet information through an online form;
(2) manual curation of the datasheet information by an expert in NLP, which,
once approved (Section 3.3), is made available via the NusaCatalogue portal
and a data loader implementation request is submitted to NusaCrowd Data Hub;
(3) implementation of a data loader; and (4) review and approval of the
implemented data loader by two maintainers, which is then published on
NusaCrowd Data Hub. In addition to the datasheets, we also provide
instructions on how to use the data in NusaCatalogue.

###  3.3 Dataset Standardization and Curation

We standardize the tasks from the datasets in NusaCrowd into several
categories according to a specific schema, defined as a common set of
attributes required to perform the task. We use the schema to cover similar
tasks across the datasets. We define 13 schemas to cover all the tasks and
modalities in the datasets, e.g., text classification, text generation, image
captioning, and speech recognition. For instance, in the single-label text
classification schema (TEXT), each example consists of three attributes (id,
text, label), where id denotes a unique row identifier, text denotes the input
text, and label denotes a discriminative target variable. We elaborate on the
attributes of each schema in Appendix B.

Language | langid.py | FastText | CLD3
---|---|---|---
Top-1 | Top-3 | Top-1 | Top-3 | Top-1
Eng | 98.33 | 99.33 | 94.05 | 99.03 | 99.69
Ind | 72.11 | 90.39 | 82.42 | 89.92 | 60.27
Sun | — | — | 34.28 | 75.21 | 50.53
Jav | 48.97 | 79.07 | 28.08 | 69.43 | 46.88

Table 1: Language identification accuracy based on different languages. For
Sundanese and Javanese, several datasets consist of informal Indonesian
utterances including Ind–Sun and Ind–Jav code-mixed sentences. Figure 3:
Summary of tasks, schemas, modalities, and languages55footnotemark: 5 in
NusaCrowd. $\sim$75% of the datasets are textual language data in Indonesian,
with the other two modalities being vision-language and speech. Textual
language data covers 19 Indonesian languages (Indonesian and 18 other
languages spoken in the region), the speech data covers 8 languages
(Indonesian and 7 local languages), while vision-language data only covers
Indonesian.

To assess the quality of the datasets in NusaCrowd, we perform manual curation
for each datasheet submission based on two criteria: language correctness, and
the annotation process. We provide the results as metadata for each dataset.
We check the correctness of the reported language using off-the-shelf language
identification (LID) tools. We perform LID in 4 languages: English,
Indonesian, Sundanese, and Javanese, We measure the LID accuracy compared to
the reported languages in the metadata on all tasks containing text modality
in NusaCrowd. Since many datasets consist of a large number of samples,
language correctness checking is done both automatically and manually.

We conduct automatic language identification for 4 languages, i.e., English,
Indonesian, Sundanese, and Javanese 666We only perform language identification
as these are the only languages supported by most of the existing off-the-
shelf language identification tools. using 3 off-the-shelf language
identification tools, i.e., langid.py Lui and Baldwin (2012), FastText LID
Ooms (2022), and Google CLD3 Ooms (2022). For other languages, since there is
no language identification library available, the curation is done manually
through sampling. Based on the automatic language identification result in
Table 1, the correctness of languages is quite high, indicated by the top-3
accuracy of each language identification tools 777we don’t consider Top-1 for
Sundanese and Javanese since the languages are low-resource and often
mispredicted. Additionally, the accuracy of Indonesian is not as high as
English, we conjecture that this is caused by there are many English terms
from tasks that are collected from online platforms.

For assessing the annotation process for each dataset, we manually check the
dataset annotation process from relevant publications and/or other
descriptions and classify them into five categories, i.e., human-generated,
crawling with human annotation, machine-generated with human curation,
machine-generated or crawling without human curation, and unknown. The
statistics of the dataset annotation assessment are shown in Appendix K. In
general, $\sim$90% of all the datasets listed in NusaCrowd are human-curated,
showing that most of the datasets in NusaCrowd are high-quality and well-
suited for building and evaluating Indonesian NLP models. Moreover, almost
half of the datasets are collected through crawling and are annotated manually
by humans, usually for NLP tasks such as sentiment analysis, emotion
recognition, hate speech detection, named entity recognition, and machine
translation. The crawling often comes from sources such as social media, news
platforms, online reviews, etc.

###  3.4 Datasets in NusaCrowd

NusaCrowd includes 137 datasheets and 118 dataloaders, including access to 14
previously non-public datasets, and a variety of tasks and languages. We list
all of the previously private datasets in Appendix I. NusaCrowd covers 36 task
types, including: machine translation, summarization, sentiment analysis,
part-of-speech (POS) tagging, and question answering, which are standardized
into 13 different schemas. The datasets in NusaCrowd stem from three
modalities—image, text, and speech—with the majority of the data coming from
the text modality. In terms of languages, NusaCrowd covers 19 Indonesian
languages, i.e., Indonesian and 18 regional languages, in addition to some
non-Indonesian languages such as Japanese, English, Spanish, and Russian,
which come into the mix as machine translation language pairs. A summary of
the datasets is shown in Footnote 5. A list of language codes with the
complete language name and family is provided in Appendix A. We present
comprehensive details of the datasets in Appendix K, and a comparison of
NusaCrowd with other initiatives in Appendix J.

#### Modalities

NusaCrowd comprises datasets from three different modalities, i.e., image,
text, and speech, all of which are related to language tasks. Most datasets
contain text data for natural language understanding (e.g., sentiment
analysis, named entity recognition, and parsing) and natural language
generation tasks (e.g., machine translation, paraphrasing, and abstractive
summarization). These account for 29 out of 36 task types in NusaCrowd. In
addition, NusaCrowd covers three vision tasks: vision-language pre-training,
image captioning, and text-to-image generation. For speech, NusaCrowd covers
four tasks: automatic speech recognition (ASR), text-to-speech synthesis
(TTS), speech-to-text translation (S2T), and speech-to-speech translation
(S2S).

77footnotetext: Based on ISO639-3 language codes:
https://iso639-3.sil.org/code_tables/639/data.

Figure 4: (left) The datasets used in NusaNLU and (right) zero-shot
generalization in NusaNLU. Box plots show summary statistics of accuracy
scores. For XGLM and BLOOMZ, each point denotes average per-dataset
performance using three different prompts. (ind) and (eng) denote the prompt
language used for prompting, i.e., Indonesian and English, respectively.

#### Languages

NusaCrowd covers Indonesian and 18 regional languages. Most languages covered
in NusaCrowd belong to the Austronesian language family, 14 of which are part
of Malayo-Polynesian family (including Indonesian), 2 of which are creole
languages, i.e., Tok Pisin (tpi) and Tetun Dili (tdt).888The two languages are
not spoken in Indonesia, but instead used in neighboring countries: Papua New
Guinea and Timor Leste. The other two languages — Hakka/Khek (hak) and Min Nan
(nan) with Teochew dialect — are Sinitic and belong to the Sino-Tibetan
language family. Detailed descriptions of each language are provided in
Appendix A.

##  4 NusaCrowd Benchmarks

To showcase the benefit of NusaCrowd, we develop three different benchmarks
from subsets of the datasets. Specifically, we develop benchmarks for
Indonesian and other local languages including a zero-shot NLU benchmark
(NusaNLU), a zero-shot NLG benchmark (NusaNLG), and a multilingual ASR
benchmark (NusaASR).

###  4.1 NusaNLU

Existing benchmarks Wilie et al. (2020); Koto et al. (2020b) in Indonesian NLU
only cover one language, i.e., the national language, Indonesian. Moreover,
these benchmarks only focus on comparing traditional machine learning
approaches with the fine-tuning approaches of pre-trained LMs. Following
recent work in other high-resource languages that explore zero-shot
generalization of large LMs Scao et al. (2022); Lin et al. (2022); Muennighoff
et al. (2022); Fries et al. (2022), we develop NusaNLU, the first zero-shot
NLU benchmark in Indonesian and regional languages to benchmark zero-shot
techniques over 26 datasets using both Indonesian monolingual and multilingual
LMs. NusaNLU covers 12 languages across various tasks, including 3 emotion
classification tasks Saputri et al. (2018); Yulianti et al. (2021); Riccosan
et al. (2022), 18 sentiment analysis tasks Winata et al. (2023); Nurlaila et
al. (2017); Hidayatullah et al. (2020); Wongso et al. (2021); Koto et al.
(2020b); Purwarianti and Crisdayanti (2019), one review score rating
task999https://huggingface.co/datasets/jakartaresearch/google-play-review, one
hate speech detection task Ibrohim and Budi (2019), one abusive language
detection task Putri et al. (2021), one next tweet prediction task Koto et al.
(2020b), and one natural language inference (NLI) task Mahendra et al. (2021).
A visual overview of the datasets in NusaNLU is provided in Figure 4.

#### Models

We evaluate three state-of-the-art multilingual language models: XLM-R Conneau
et al. (2020), XGLM Lin et al. (2022), and BLOOMZ Muennighoff et al. (2022).
We generally evaluate in a zero-shot cross-lingual transfer setting Hu et al.
(2020). For XLM-R, we employ intermediate-task training on NLI by predicting
the entailment relation between the input text and the label Phang et al.
(2020). We explore both XLM-R fine-tuned on XNLI Conneau et al. (2018) and
Indonesian IndoNLI Mahendra et al. (2021). For XGLM and BLOOMZ, we employ
zero-shot prompt-based learning with prompts in English and Indonesian. For
each language and task, we employ three different prompts and take the average
score for the evaluation of each task. More details about fine-tuning
hyperparameters and the prompt used in the NLU experiments are shown in
Appendix C.

#### Results

Figure 4 shows the zero-shot NLU results of all the models. Overall, the
prompting performance of BLOOMZ outperforms other models. Prompting with
BLOOMZ outperforms XGLM by a huge margin, providing evidence of the benefit of
instruction tuning for prompting. Interestingly, zero-shot cross-task transfer
using XLM-R trained on XNLI (XLM-R XNLI) outperforms prompting using XGLM and
performs on par with prompting using BLOOMZ, despite the huge difference in
their model sizes. This suggests that large LMs are not always needed to
perform zero-shot NLU tasks and better efficiency can be achieved through
cross-task transfer using much smaller models while achieving similar
performance.

Comparing the performance of cross-task fine-tuning across monolingual and
multilingual NLI, XLM-R XNLI (122k training instances) outperforms XLM-R
IndoNLI (11k training instances) by a large margin, suggesting that using
large-scale multilingual data is more effective than using smaller-scale data
from closely-related or even the same language to fine-tune a multilingual
model in a zero-shot cross-task setting. Comparing the language of the
prompts, both BLOOMZ and XGLM with English prompts perform better than the
corresponding models with Indonesian prompts. Our findings align with prior
work Muennighoff et al. (2022); Lin et al. (2022); Shi et al. (2022), which
shows that, in most cases, the corresponding models perform better in English
than on human-translated prompts, despite the language distance between the
prompt template and the corresponding text data.

Comparing the performance across different languages, as shown in Figure 5, we
can conclude that the performance of all models is generally better for
Indonesian and English compared to regional Indonesian languages, suggesting
that existing multilingual models are unable to generalize well on these
languages, and better language representations are vital to close the gap. A
full breakdown of per-task performance is provided in Appendix F.

Figure 5: Average zero-shot performance per language across all models on the
NusaX subset. All models achieve higher scores for Indonesian (ind) and
English (eng).

Figure 6: (left) The datasets used in NusaNLG and (right) Zero-shot
generalization to machine translation and summarization tasks in NusaNLG. Box
plots show summary statistics of the evaluation performance. Points are per-
dataset scores from the average of performances over 3 different prompts.
(ind) and (eng) denote the prompt language used for prompting, i.e.,
Indonesian and English, respectively.

###  4.2 NusaNLG

Recent work on Indonesian NLG benchmarks Cahyawijaya et al. (2021b); Guntara
et al. (2020) has employed transformer-based models, both decoder-only (e.g.,
IndoGPT) and encoder–decoder (e.g., IndoBART) architectures. To further
broaden NLG research in Indonesian and other regional languages, we develop an
NLG benchmark, NusaNLG, which covers NLG tasks in 12 languages including
English, Indonesian, and 10 local languages. NusaNLG incorporates a total of
36 datasets across various tasks covering 33 machine translation tasks Guntara
et al. (2020); Cahyawijaya et al. (2021b) and 3 summarization tasks Kurniawan
and Louvan (2018); Koto et al. (2020a) (Figure 6). We use SacreBLEU for
machine translation evaluation, and ROUGE-L for summarization evaluation.

#### Models

Following recent work in prompting, we explore the possibility of zero-shot
generalization of various large LMs on generation tasks through prompting on
two NLG tasks, i.e., machine translation and summarization. To explore the
effect of different prompt languages on the zero-shot generalization
performance, we evaluate prompts in English and Indonesian. We employ two
large LMs: XGLM Lin et al. (2022), and BLOOMZ Muennighoff et al. (2022). For
each task and prompt language, we provide three different prompts and average
the result. More details about the hyperparameters and the prompt used in the
NLG experiments are shown in Appendix D.

#### Results

The zero-shot NLG results of all models are shown in Figure 6. Outputs
obtained by prompting BLOOMZ outperform those obtained from XGLM for both
English and Indonesian prompts. The performance is better on average when
prompting BLOOMZ with English prompts than when using Indonesian prompts,
which aligns with the results of BLOOMZ on XNLI Conneau et al. (2018), where
BLOOMZ with English prompts performs better than the human-translated prompts
Muennighoff et al. (2022).

Language | Ind prompt | Eng prompt
---|---|---
eng $\rightarrow$ ind | 5.11 | 6.04
ind $\rightarrow$ eng | 4.65 | 7.90
local $\rightarrow$ ind | 2.11 | 2.72
ind $\rightarrow$ local | 1.66 | 2.96

Table 2: Average SacreBLEU performance of BLOOMZ for different language pairs.
Local denotes all Indonesian local languages in NusaCrowd.

Prompting using XGLM yields better quality outputs using Indonesian language
prompts than using English prompts. A similar result is reported in XGLM
evaluation for Spanish XNLI and Chinese XCOPA Ponti et al. (2020), which shows
that prompting with the human-translated prompt to the target language
produces a better score than the English one. For the BLOOMZ models, the
result for English is better since we use the BLOOMZ checkpoint fine-tuned
only on English prompts. Additionally, we found that the zero-shot translation
quality across all models and prompt languages is poor, especially for local
languages, as shown in Table 2. This is even more severe when local languages
are involved, yielding $\sim$2% SacreBLEU. This finding suggests that existing
large multilingual LMs still fail to learn representations for these local
languages. A full breakdown of per-task results over NusaNLG is provided in
Appendix G.

Model | ace | ban | btk | bug | ind | jav | min | sun
---|---|---|---|---|---|---|---|---
Single-task Training
wav2vec 2.0-pt | 100.00 | 71.99 | 64.77 | 100.00 | 12.51 | 85.78 | 100.00 | 83.01
wav2vec 2.0-ft |  49.31 |  28.74 |  40.92 | 90.09 |  2.13 |  32.11 |  24.29 |  26.62
Monolingual Multi-task Training
wav2vec 2.0-pt (ind) | 95.14 |  $>$100 |  $>$100 |  96.70 | 4.20 |  $>$100 |  46.19 |  $>$100
wav2vec 2.0-pt (jav) |  $>$100 | 67.02 | 81.24 |  $>$100 | 88.87 | 46.97 | 68.10 | 69.89
wav2vec 2.0-pt (sun) | 92.36 | 82.37 | 74.67 |  $>$100 | 91.22 | 93.43 | 98.57 | 40.42
wav2vec 2.0-ft (ind) | 91.67 |  $>$100 |  $>$100 |  $>$100 | 1.87 |  $\geq$100 | 70.48 |  $>$100
wav2vec 2.0-ft (jav) | 90.28 |  52.63 |  59.79 |  $>$100 | 78.87 |  27.23 | 52.86 | 54.31
wav2vec 2.0-ft (sun) |  89.58 | 76.52 | 61.34 |  $>$100 | 89.59 | 88.50 | 79.05 |  25.11
Multilingual Multi-task Training
wav2vec 2.0-pt | 40.85 | 16.73 | 18.98 | 41.59 | 8.05 | 18.57 | 16.94 | 13.93
wav2vec 2.0-ft | 31.94 | 21.05 | 35.99 | 53.30 |  1.90 | 27.55 | 18.10 | 20.79

Table 3: Speech recognition results in terms of average word error rate (WER)
per language over NusaASR (lower is better). For monolingual multi-task
training, the language in brackets denotes the language used for training.
Bold denotes the best performance across all groups. Underline denotes the
best performance within the group. In monolingual multi-task training,
Highlight denotes that the model is trained in the corresponding language.

###  4.3 NusaASR

In addition to zero-shot benchmarks for textual language data, we showcase the
benefit of NusaCrowd by extending the NLP benchmark in Indonesian languages to
speech. We develop the first multilingual ASR benchmark for Indonesian and
other local languages covering 17 ASR datasets in eight languages:
5$\times$Indonesian (ind), 3$\times$Sundanese (sun), 3$\times$Javanese (jav),
2$\times$Balinese (ban), 1$\times$Acehnese (ace), 1$\times$Batak (btk),
1$\times$Buginese (bug), and 1$\times$Minangkabau (min).

#### Models

We employ pre-trained wav2vec 2.0 Baevski et al. (2020) models in our
experiment. We explore three training settings: single-task monolingual
training, where we fine-tune and evaluate the model on the corresponding ASR
dataset; multi-task monolingual training, where we fine-tune the model using
multiple ASR datasets on a single language (we evaluate three languages with
the largest resources, i.e., Indonesian, Javanese, and Sundanese); and joint
multi-task multilingual training, where we fine-tune the model using all 17
ASR datasets listed on NusaASR. We experiment with two wav2vec 2.0LARGE
($\sim$300M parameters) checkpoints, i.e., an unsupervised pre-trained XLS-R
wav2vec 2.0 (wav2vec
2.0-pt)101010https://huggingface.co/facebook/wav2vec2-large-xlsr-53 and an
Indonesian, Javanese, and Sundanese ASR fine-tuned XLS-R wav2vec 2.0 (wav2vec
2.0-ft).111111https://huggingface.co/indonesian-nlp/wav2vec2-indonesian-
javanese-sundanese In addition to wav2vec 2.0, we also employ WhisperSMALL
Radford et al. (2022)121212https://huggingface.co/openai/whisper-small
($\sim$250M parameters). Full details of the experiment setup are provided in
Appendix E.

#### Results

Table 3 shows the per-language task-averaged performances of wav2vec 2.0
models over NusaASR. The complete per-task results of NusaASR along with the
performance of WhisperSMALL are provided in Appendix H. Based on the results,
single-task training on wav2vec 2.0-pt performs poorly due to the limited
training data to adapt from unsupervised contrastive pre-training to the ASR
task, while the ASR fine-tuned wav2vec 2.0-ft model yields decent results in
most languages, except for Buginese (bug) with 90.09% WER. This suggests
limited transferability from Indonesian, Sundanese, and Javanese to Buginese,
consistent with the analysis from NusaX Winata et al. (2023) regarding the low
overlap between Buginese and other local languages included in NusaCrowd. For
monolingual multi-task training, all models perform well only in the languages
that they were trained on. This shows that there is a large difference between
vocabulary and speech features from one language to another.

For all models evaluated over NusaASR (wav2vec 2.0-pt, wav2vec 2.0-ft, and
Whisper), the best performance is achieved through multilingual multi-task
training, yielding as low as $\sim$20% average WER across all languages,
suggesting transferability of speech features from one language to the others
Fung et al. (1998); PLU et al. (2000); Sakti et al. (2012); Nakayama et al.
(2019). Unlike prior work Winata et al. (2023), where Acehnese (ace) yields
similar performance to other languages in sentiment analysis, the same
behavior is not reflected in ASR. This suggests that there is a distinction
between the speech of Acehnese (ace) and other regional languages, despite
vocabulary overlap and shared language structure.

##  5 Discussion

#### Multilinguality for Low-Resource Languages

Despite the higher pre-training cost relative to monolingual LMs Cahyawijaya
et al. (2021b), multilingual LMs are more versatile and transferable. Recent
low-resource monolingual language LMs are on the scale of a hundred million
parameters, while the size of multilingual LMs, within a period of three
years, has increased by around 1,000$\times$ from $\sim$100M to $\geq$100B
parameters Devlin et al. (2019); Xue et al. (2021); Tang et al. (2021);
Muennighoff et al. (2022); Scao et al. (2022). This benefit comes from the
data scale of multilingual LMs, which is orders of magnitude larger than
monolingual LMs. Additionally, multilingual LMs benefit from positive transfer
between related languages, which is especially beneficial for low-resource
languages. Moving forward, we expect that multilingual LMs will play a
significant role in the exploration of low-resource languages.

#### Viability of Large Models for Indonesian

Computational resources are limited among Indonesian research institutions and
in industry, even among the top Indonesian universities Indonesia (2020);
Nityasya et al. (2020). Focusing solely on large LMs will limit accessibility,
and adoption will likely be low. Therefore, although larger LMs empirically
offer better quality, we instead suggest investing more effort in efficiency.
This includes smaller sizes LMs and modularized LMs Pfeiffer et al. (2020);
Ansell et al. (2021); Pfeiffer et al. (2022). Furthermore, more work on
efficiency through factorization Winata et al. (2020); Cahyawijaya et al.
(2021a), pruning Frankle and Carbin (2019); Dai et al. (2021), quantization
Shen et al. (2020); Aji and Heafield (2020), or distillation Zhang et al.
(2020); Bai et al. (2021); Dai et al. (2022) are also likely to be beneficial.

##  6 Conclusion

We have introduced NusaCrowd, a combined resource for Indonesian and regional
languages, covering 137 datasets, 118 of which have a standardized loader.
NusaCrowd covers Indonesian and 18 regional languages, encompassing 3
different data modalities. Manual and automatic curation processes were
conducted to verify the quality of the collected datasets. The effectiveness
of NusaCrowd is shown in three use cases: zero-shot NLU (NusaNLU), zero-shot
NLG (NusaNLG), and multilingual ASR (NusaASR) benchmarks. Our experiments
provide evidence regarding the efficiency of cross-tasks method over prompting
for zero-shot NLU, the limited capabilities of existing large LMs for handling
NLG tasks in local languages, and the potential of joint multilingual multi-
task learning for Indonesian ASR. We hope NusaCrowd will benefit the research
community as a data hub for Indonesian and regional languages by facilitating
easy access to datasets, as well as accelerating research and development.

##  7 Limitations

#### Dataset Utilization

We have collected 137 datasets, yet we have only conducted experiments over a
minority of these ($\sim$40 datasets), leaving the remaining datasets
unexplored. Since the datasets are already curated, future work should further
explore these datasets in additional experiments. In this work, we do not
experiment on image-text datasets for two reasons: (1) all of the image-text
datasets are translated from English versions; and (2) there is no large LM
available for zero-shot image-to-text generation.

#### Experiments

We did not attempt few-shot or fully-supervised learning experiments in
NusaCrowd since prior work has explored these approaches on some of the
datasets Wilie et al. (2020); Koto et al. (2020b); Cahyawijaya et al. (2021b);
Winata et al. (2023). We specifically conduct our experiments on zero-shot
methods to explore the generalization of zero-shot cross-lingual and zero-shot
prompting approaches to extremely low-resource languages.

#### Task Diversity

The tasks represented in NusaCrowd are skewed towards MT, sentiment, abusive
text classification, and ASR. Many other tasks remain unexplored for
Indonesian and regional languages. Furthermore, most ASR work come from the
same authors or research groups. While these topics are prevalent among
Indonesian researchers, it is also important to expand to other tasks.

#### Domain Diversity

The datasets in NusaCrowd are primarily from the domains of social media,
news, and other general domain sources. Despite having a huge potential,
narrow-domain datasets, such as clinical, biomedical, legal, financial, and
educational datasets remain underrepresented for Indonesian and regional
languages. Exploration of domain-specific data and use cases for Indonesian
and regional languages is critical.

#### Language Diversity

There are 700+ languages in Indonesia. However, we have only focused on a
small fraction of these languages. In addition, there are also other regional
languages similar to the two Sinitic languages in NusaCrowd, i.e., Hakka
(Khek) and Min Nan (Teochew). More focus on under-represented languages is an
interesting future direction.

#### Multimodality

The datasets in NusaCrowd are mainly in the text modality. Exploration of
speech, image, and other modalities for Indonesian and regional languages is
still limited, and there are potentially exciting opportunities to capture
locally-relevant Indonesian culture in such modalities.

#### Utilization of Datasets

There are 137 datasets contained in NusaCrowd. While we showcased three
different use cases for the datasets (i.e., zero-shot NLU, zero-shot NLG, and
multilingual ASR benchmarks), there is much greater potential to use the
datasets in NusaCrowd. Potential areas of focus include experimenting with
various approaches and analyses over multiple datasets, such as multi-task
learning, continual learning, or few-shot learning.

##  8 Ethical Statement

Our work highlights the importance of democratizing access to Natural Language
Processing (NLP) technology for underrepresented and extremely low-resource
languages with a focus on the Austronesian language family specifically in
Indonesian languages. Within our study, we are well aware of the ethical
responsibility associated with language research and the potential impact that
comes with it. Our study prioritizes diversity, inclusivity, and fairness.
Within this work, the contribution of each collaborator is calculated
following a fair and transparent scoring guideline that empowers the core
principles of NusaCrowd. We have obtained informed consent from all dataset
authors to provide publicly open-access corpora and benchmarks. Throughout our
research process, we have made conscious efforts to engage with the language
communities, involve local experts, and respect their linguistic and cultural
nuances. We encourage further collaboration and engagement with
underrepresented language communities to ensure that their voices are heard
and their needs are addressed in future NLP development. We remain committed
to the principles of ethical research, diversity, inclusivity, and fairness,
striving to promote social good through our work in the field of language
technology.

## References

  * Abidin and Ahmad (2021) Z Abidin and I Ahmad. 2021.  Effect of mono corpus quantity on statistical machine translation Indonesian–Lampung dialect of nyo.  In _Journal of Physics: Conference Series_ , volume 1751. IOP Publishing.
  * Adelani et al. (2021) David Ifeoluwa Adelani, Jade Abbott, Graham Neubig, Daniel D’souza, Julia Kreutzer, Constantine Lignos, Chester Palen-Michel, Happy Buzaaba, Shruti Rijhwani, Sebastian Ruder, Stephen Mayhew, Israel Abebe Azime, Shamsuddeen H. Muhammad, Chris Chinenye Emezue, Joyce Nakatumba-Nabende, Perez Ogayo, Aremu Anuoluwapo, Catherine Gitau, Derguene Mbaye, Jesujoba Alabi, Seid Muhie Yimam, Tajuddeen Rabiu Gwadabe, Ignatius Ezeani, Rubungo Andre Niyongabo, Jonathan Mukiibi, Verrah Otiende, Iroro Orife, Davis David, Samba Ngom, Tosin Adewumi, Paul Rayson, Mofetoluwa Adeyemi, Gerald Muriuki, Emmanuel Anebi, Chiamaka Chukwuneke, Nkiruka Odu, Eric Peter Wairagala, Samuel Oyerinde, Clemencia Siro, Tobius Saul Bateesa, Temilola Oloyede, Yvonne Wambui, Victor Akinode, Deborah Nabagereka, Maurice Katusiime, Ayodele Awokoya, Mouhamadane MBOUP, Dibora Gebreyohannes, Henok Tilaye, Kelechi Nwaike, Degaga Wolde, Abdoulaye Faye, Blessing Sibanda, Orevaoghene Ahia, Bonaventure F. P. Dossou, Kelechi Ogueji, Thierno Ibrahima DIOP, Abdoulaye Diallo, Adewale Akinfaderin, Tendai Marengereke, and Salomey Osei. 2021.  MasakhaNER: Named entity recognition for African languages.  _Transactions of the Association for Computational Linguistics_ , 9:1116–1131.
  * Aji et al. (2021) Alham Fikri Aji, Radityo Eko Prasojo Tirana Noor Fatyanosa, Philip Arthur, Suci Fitriany, Salma Qonitah, Nadhifa Zulfa, Tomi Santoso, and Mahendra Data. 2021\.  Paracotta: Synthetic multilingual paraphrase corpora from the most diverse translation sample pair.  In _Proceedings of the 35th Pacific Asia Conference on Language, Information and Computation_ , pages 666–675.
  * Aji and Heafield (2020) Alham Fikri Aji and Kenneth Heafield. 2020.  Compressing neural machine translation models with 4-bit precision.  In _Proceedings of the Fourth Workshop on Neural Generation and Translation_ , pages 35–42, Online. Association for Computational Linguistics.
  * Aji et al. (2022) Alham Fikri Aji, Genta Indra Winata, Fajri Koto, Samuel Cahyawijaya, Ade Romadhony, Rahmad Mahendra, Kemal Kurniawan, David Moeljadi, Radityo Eko Prasojo, Timothy Baldwin, Jey Han Lau, and Sebastian Ruder. 2022.  One country, 700+ languages: NLP challenges for underrepresented languages and dialects in Indonesia.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 7226–7249, Dublin, Ireland. Association for Computational Linguistics.
  * Alfina et al. (2020) Ika Alfina, Indra Budi, and Heru Suhartanto. 2020.  Tree rotations for dependency trees: Converting the head-directionality of noun phrases.  _Journal of Computer Science_ , 16(11):1585–1597.
  * Alfina et al. (2019) Ika Alfina, Arawinda Dinakaramani, Mohamad Ivan Fanany, and Heru Suhartanto. 2019\.  A gold standard dependency treebank for Indonesian.  In _Proceedings of the 33rd Pacific Asia Conference on Language, Information and Computation_ , pages 1–9. Waseda Institute for the Study of Language and Information.
  * Alfina et al. (2017a) Ika Alfina, Rio Mulia, Mohamad Ivan Fanany, and Yudo Ekanata. 2017a.  Hate speech detection in the Indonesian language: A dataset and preliminary study.  In _2017 International Conference on Advanced Computer Science and Information Systems (ICACSIS)_ , pages 233–238.
  * Alfina et al. (2017b) Ika Alfina, Septiviana Savitri, and Mohamad Ivan Fanany. 2017b.  Modified DBpedia entities expansion for tagging automatically NER dataset.  In _2017 International Conference on Advanced Computer Science and Information Systems (ICACSIS)_ , pages 216–221. IEEE.
  * Altaher et al. (2022) Yousef Altaher, Ali Fadel, Mazen Alotaibi, Mazen Alyazidi, Mishari Al-Mutairi, Mutlaq Aldhbuiub, Abdulrahman Mosaibah, Abdelrahman Rezk, Abdulrazzaq Alhendi, Mazen Abo Shal, Emad A. Alghamdi, Maged S. Alshaibani, Jezia Zakraoui, Wafaa Mohammed, Kamel Gaanoun, Khalid N. Elmadani, Mustafa Ghaleb, Nouamane Tazi, Raed Alharbi, Maraim Masoud, and Zaid Alyafeai. 2022.  Masader Plus: A new interface for exploring+ 500 Arabic NLP datasets.  _arXiv preprint arXiv:2208.00932_.
  * Alyafeai et al. (2022) Zaid Alyafeai, Maraim Masoud, Mustafa Ghaleb, and Maged S. Al-shaibani. 2022.  Masader: Metadata sourcing for Arabic text and speech data resources.  In _Proceedings of the Thirteenth Language Resources and Evaluation Conference_ , pages 6340–6351, Marseille, France. European Language Resources Association.
  * Anastasopoulos et al. (2020) Antonios Anastasopoulos, Alessandro Cattelan, Zi-Yi Dou, Marcello Federico, Christian Federman, Dmitriy Genzel, Francisco Guzmán, Junjie Hu, Macduff Hughes, Philipp Koehn, Rosie Lazar, Will Lewis, Graham Neubig, Mengmeng Niu, Alp Öktem, Eric Paquin, Grace Tang, and Sylwia Tur. 2020.  TICO-19: the Translation Initiative for COVID-19.  In _Proceedings of the 1st Workshop on NLP for COVID-19 (Part 2) at EMNLP 2020_.
  * Anderbeck (2008) Karl Ronald Anderbeck. 2008.  _Malay dialects of the Batanghari river basin (Jambi, Sumatra)_.  SIL International.
  * Ansell et al. (2021) Alan Ansell, Edoardo Maria Ponti, Jonas Pfeiffer, Sebastian Ruder, Goran Glavaš, Ivan Vulić, and Anna Korhonen. 2021.  MAD-G: Multilingual adapter generation for efficient cross-lingual transfer.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 4762–4781, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Ardila et al. (2020) Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber. 2020\.  Common voice: A massively-multilingual speech corpus.  In _Proceedings of the 12th Language Resources and Evaluation Conference_ , pages 4218–4222.
  * Ardiyanti Suryani et al. (2022a) Arie Ardiyanti Suryani, Dwi Hendratmo Widyantoro, Ayu Purwarianti, and Yayat Sudaryat. 2022a.  Postagged sundanese monolingual corpus.  Telkom University Dataverse.
  * Ardiyanti Suryani et al. (2022b) Arie Ardiyanti Suryani, Dwi Hendratmo Widyantoro, Ayu Purwarianti, and Yayat Sudaryat. 2022b.  Sundanese-indonesian parallel corpus.
  * Arka (2003) I Wayan Arka. 2003.  _Balinese morphosyntax: a lexical-functional approach_.  Pacific Linguistics.
  * Artari et al. (2021) Valentina Kania Prameswara Artari, Rahmad Mahendra, Meganingrum Arista Jiwanggi, Adityo Anggraito, and Indra Budi. 2021.  A multi-pass sieve coreference resolution for Indonesian.  In _Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)_ , pages 79–85, Held Online. INCOMA Ltd.
  * Arwidarasti et al. (2019) Jessica Naraiswari Arwidarasti, Ika Alfina, and Adila Alfa Krisnadhi. 2019.  Converting an Indonesian constituency treebank to the Penn Treebank format.  In _23rd International Conference on Asian Language Processing, IALP 2019_ , pages 331–336. Institute of Electrical and Electronics Engineers Inc.
  * Aulia and Budi (2019) Nofa Aulia and Indra Budi. 2019.  Hate speech detection on Indonesian long text documents using machine learning approach.  In _Proceedings of the 2019 5th International Conference on Computing and Artificial Intelligence_ , ICCAI ’19, page 164–169, New York, NY, USA. Association for Computing Machinery.
  * Azhar et al. (2019) A. N. Azhar, M. L. Khodra, and A. P. Sutiono. 2019.  Multi-label aspect categorization with convolutional neural networks and extreme gradient boosting.  In _2019 International Conference on Electrical Engineering and Informatics (ICEEI)_ , pages 35–40.
  * Baevski et al. (2020) Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020.  wav2vec 2.0: A framework for self-supervised learning of speech representations.  _Advances in Neural Information Processing Systems_ , 33:12449–12460.
  * Bai et al. (2021) Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jin Jin, Xin Jiang, Qun Liu, Michael Lyu, and Irwin King. 2021.  BinaryBERT: Pushing the limit of BERT quantization.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 4334–4348, Online. Association for Computational Linguistics.
  * Barik et al. (2019) Anab Maulana Barik, Rahmad Mahendra, and Mirna Adriani. 2019.  Normalization of Indonesian-English code-mixed Twitter data.  In _Proceedings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019)_ , pages 417–424, Hong Kong, China. Association for Computational Linguistics.
  * Blust (2013) Robert Blust. 2013.  _The Austronesian Languages_.  The Australian National University.
  * Cahyawijaya et al. (2021a) Samuel Cahyawijaya, Genta Indra Winata, Holy Lovenia, Bryan Wilie, Wenliang Dai, Etsuko Ishii, and Pascale Fung. 2021a.  Greenformer: Factorization toolkit for efficient deep neural networks.  arXiv 2109.06762.
  * Cahyawijaya et al. (2021b) Samuel Cahyawijaya, Genta Indra Winata, Bryan Wilie, Karissa Vincentio, Xiaohong Li, Adhiguna Kuncoro, Sebastian Ruder, Zhi Yuan Lim, Syafri Bahar, Masayu Khodra, Ayu Purwarianti, and Pascale Fung. 2021b.  IndoNLG: Benchmark and resources for evaluating Indonesian natural language generation.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 8875–8898, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Changpinyo et al. (2021) Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. 2021.  Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts.  In _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 3557–3567.
  * Chocron and Pareti (2018) Paula Chocron and Paolo Pareti. 2018.  Vocabulary alignment for collaborative agents: a study with real-world multilingual how-to instructions.  In _Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18_ , pages 159–165. International Joint Conferences on Artificial Intelligence Organization.
  * Cohn and Ravindranath (2014) Abigail C Cohn and Maya Ravindranath. 2014.  Local languages in indonesia: Language maintenance or language shift.  _Linguistik Indonesia_ , 32(2):131–148.
  * Conneau et al. (2020) Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Édouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020.  Unsupervised cross-lingual representation learning at scale.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8440–8451.
  * Conneau et al. (2018) Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018.  Xnli: Evaluating cross-lingual sentence representations.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics.
  * Crouch (2009) Sophie Elizabeth Crouch. 2009.  _Voice and verb morphology in Minangkabau, a language of West Sumatra, Indonesia_.  Ph.D. thesis, The University of Western Australia.
  * Dai et al. (2021) Wenliang Dai, Samuel Cahyawijaya, Zihan Liu, and Pascale Fung. 2021.  Multimodal end-to-end sparse model for emotion recognition.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 5305–5316, Online. Association for Computational Linguistics.
  * Dai et al. (2022) Wenliang Dai, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu, and Pascale Fung. 2022.  Enabling multimodal generation on CLIP via vision-language knowledge distillation.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 2383–2395, Dublin, Ireland. Association for Computational Linguistics.
  * Darwis et al. (2019) Robby Darwis, Herry Sujaini, and Rudy Dwi Nyoto. 2019.  Peningkatan mesin penerjemah statistik dengan menambah kuantitas korpus monolingual (studi kasus: Bahasa indonesia-sunda).  _JUSTIN (Jurnal Sistem dan Teknologi Informasi)_ , 7(1):27–32.
  * Davies (2010) William D Davies. 2010.  _A grammar of Madurese_.  Mouton De Gruyter.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Dhole et al. (2021) Kaustubh D. Dhole, Varun Gangal, Sebastian Gehrmann, Aadesh Gupta, Zhenhao Li, Saad Mahamood, Abinaya Mahendiran, Simon Mille, Ashish Shrivastava, Samson Tan, Tongshuang Wu, Jascha Sohl-Dickstein, Jinho D. Choi, Eduard Hovy, Ondrej Dusek, Sebastian Ruder, Sajant Anand, Nagender Aneja, Rabin Banjade, Lisa Barthe, Hanna Behnke, Ian Berlot-Attwell, Connor Boyle, Caroline Brun, Marco Antonio Sobrevilla Cabezudo, Samuel Cahyawijaya, Emile Chapuis, Wanxiang Che, Mukund Choudhary, Christian Clauss, Pierre Colombo, Filip Cornell, Gautier Dagan, Mayukh Das, Tanay Dixit, Thomas Dopierre, Paul-Alexis Dray, Suchitra Dubey, Tatiana Ekeinhor, Marco Di Giovanni, Tanya Goyal, Rishabh Gupta, Rishabh Gupta, Louanes Hamla, Sang Han, Fabrice Harel-Canada, Antoine Honore, Ishan Jindal, Przemyslaw K. Joniak, Denis Kleyko, Venelin Kovatchev, Kalpesh Krishna, Ashutosh Kumar, Stefan Langer, Seungjae Ryan Lee, Corey James Levinson, Hualou Liang, Kaizhao Liang, Zhexiong Liu, Andrey Lukyanenko, Vukosi Marivate, Gerard de Melo, Simon Meoni, Maxime Meyer, Afnan Mir, Nafise Sadat Moosavi, Niklas Muennighoff, Timothy Sum Hon Mun, Kenton Murray, Marcin Namysl, Maria Obedkova, Priti Oli, Nivranshu Pasricha, Jan Pfister, Richard Plant, Vinay Prabhu, Vasile Pais, Libo Qin, Shahab Raji, Pawan Kumar Rajpoot, Vikas Raunak, Roy Rinberg, Nicolas Roberts, Juan Diego Rodriguez, Claude Roux, Vasconcellos P. H. S., Ananya B. Sai, Robin M. Schmidt, Thomas Scialom, Tshephisho Sefara, Saqib N. Shamsi, Xudong Shen, Haoyue Shi, Yiwen Shi, Anna Shvets, Nick Siegel, Damien Sileo, Jamie Simon, Chandan Singh, Roman Sitelew, Priyank Soni, Taylor Sorensen, William Soto, Aman Srivastava, KV Aditya Srivatsa, Tony Sun, Mukund Varma T, A Tabassum, Fiona Anting Tan, Ryan Teehan, Mo Tiwari, Marie Tolkiehn, Athena Wang, Zijian Wang, Gloria Wang, Zijie J. Wang, Fuxuan Wei, Bryan Wilie, Genta Indra Winata, Xinyi Wu, Witold Wydmański, Tianbao Xie, Usama Yaseen, Michael A. Yee, Jing Zhang, and Yue Zhang. 2021.  NL-augmenter: A framework for task-sensitive natural language augmentation.  _arXiv preprint arXiv:2112.02721_.
  * Dinakaramani et al. (2014) Arawinda Dinakaramani, Fam Rashel, Andry Luthfi, and Ruli Manurung. 2014.  Designing an Indonesian part of speech tagset and manually tagged Indonesian corpus.  In _2014 International Conference on Asian Language Processing (IALP)_ , pages 66–69.
  * Doddapaneni et al. (2022) Sumanth Doddapaneni, Rahul Aralikatte, Gowtham Ramesh, Shreya Goyal, Mitesh M Khapra, Anoop Kunchukuttan, and Pratyush Kumar. 2022.  Indicxtreme: A multi-task benchmark for evaluating indic languages.  _arXiv preprint arXiv:2212.05409_.
  * Durie (1985) Mark Durie. 1985.  _A Grammar of Acehnese on the Basis of a Dialect of North Aceh_.  Verhandelingen van het Koninklijk Instituut voor Taal-, Land- en Volkenkunde, Dordrecht-Holland.
  * Durie (1988) Mark Durie. 1988.  Preferred argument structure in an active language: Arguments against the category ‘intransitive subject’.  _Lingua_ , 74(1):1–25.
  * Eberhard et al. (2021) David M. Eberhard, Gary F. Simons, and Charles D. Fennig. 2021.  _Ethnologue: Languages of the World. Twenty-fourth edition._ Dallas, Texas: SIL International.
  * Etsa et al. (2018) Muhammad Dwi Etsa, Herry Sujaini, and Novi Safriadi. 2018.  Pengaruh metode dictionary lookup pada cleaning korpus terhadap akurasi mesin penerjemah statistik indonesia–melayu pontianak.  _Jurnal Edukasi dan Penelitian Informatika (JEPIN)_ , 4(1):49.
  * Fachri (2014) Muhammad Fachri. 2014.  Named entity recognition for Indonesian text using hidden Markov model.  _Universitas Gadjah Mada_.
  * Faruqui and Kumar (2015) Manaal Faruqui and Shankar Kumar. 2015.  Multilingual open relation extraction using cross-lingual projection.  In _Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1351–1356.
  * Fauzi and Puspitorini (2018) Andri Imam Fauzi and Dwi Puspitorini. 2018.  Dialect and identity: A case study of Javanese use in WhatsApp and Line.  In _IOP Conference Series: Earth and Environmental Science_ , volume 175, page 012111. IOP Publishing.
  * Fernando et al. (2019) Jordhy Fernando, Masayu Leylia Khodra, and Ali Akbar Septiandri. 2019.  Aspect and opinion terms extraction using double embeddings and attention mechanism for Indonesian hotel reviews.  In _2019 International Conference of Advanced Informatics: Concepts, Theory and Applications (ICAICTA)_ , pages 1–6. IEEE.
  * Frankle and Carbin (2019) Jonathan Frankle and Michael Carbin. 2019.  The lottery ticket hypothesis: Finding sparse, trainable neural networks. In _ICLR_. OpenReview.net.
  * Fries et al. (2022) Jason Alan Fries, Leon Weber, Natasha Seelam, Gabriel Altay, Debajyoti Datta, Samuele Garda, Myungsun Kang, Ruisi Su, Wojciech Kusa, Samuel Cahyawijaya, Fabio Barth, Simon Ott, Matthias Samwald, Stephen Bach, Stella Biderman, Mario Sänger, Bo Wang, Alison Callahan, Daniel León Periñán, Théo Gigant, Patrick Haller, Jenny Chim, Jose David Posada, John Michael Giorgi, Karthik Rangasai Sivaraman, Marc Pàmies, Marianna Nezhurina, Robert Martin, Michael Cullan, Moritz Freidank, Nathan Dahlberg, Shubhanshu Mishra, Shamik Bose, Nicholas Michio Broad, Yanis Labrak, Shlok S Deshmukh, Sid Kiblawi, Ayush Singh, Minh Chien Vu, Trishala Neeraj, Jonas Golde, Albert Villanova del Moral, and Benjamin Beilharz. 2022.  Bigbio: A framework for data-centric biomedical natural language processing.  In _Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track_.
  * Fung et al. (1998) Pascale Fung, Chi Shun Cheung, Kwok Leung Lam, Wai Kat Liu, and Yuen Yee Lo. 1998\.  SALSA version 1.0: a speech-based web browser for hong kong english.  In _5th International Conference on Spoken Language Processing (ICSLP 1998)_. ISCA.
  * Gehrmann et al. (2021) Sebastian Gehrmann, Tosin Adewumi, Karmanya Aggarwal, Pawan Sasanka Ammanamanchi, Anuoluwapo Aremu, Antoine Bosselut, Khyathi Raghavi Chandu, Miruna-Adriana Clinciu, Dipanjan Das, Kaustubh Dhole, Wanyu Du, Esin Durmus, Ondřej Dušek, Chris Chinenye Emezue, Varun Gangal, Cristina Garbacea, Tatsunori Hashimoto, Yufang Hou, Yacine Jernite, Harsh Jhamtani, Yangfeng Ji, Shailza Jolly, Mihir Kale, Dhruv Kumar, Faisal Ladhak, Aman Madaan, Mounica Maddela, Khyati Mahajan, Saad Mahamood, Bodhisattwa Prasad Majumder, Pedro Henrique Martins, Angelina McMillan-Major, Simon Mille, Emiel van Miltenburg, Moin Nadeem, Shashi Narayan, Vitaly Nikolaev, Andre Niyongabo Rubungo, Salomey Osei, Ankur Parikh, Laura Perez-Beltrachini, Niranjan Ramesh Rao, Vikas Raunak, Juan Diego Rodriguez, Sashank Santhanam, João Sedoc, Thibault Sellam, Samira Shaikh, Anastasia Shimorina, Marco Antonio Sobrevilla Cabezudo, Hendrik Strobelt, Nishant Subramani, Wei Xu, Diyi Yang, Akhila Yerukola, and Jiawei Zhou. 2021.  The GEM benchmark: Natural language generation, its evaluation and metrics.  In _Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)_ , pages 96–120, Online. Association for Computational Linguistics.
  * Gehrmann et al. (2022) Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran, Alex Wang, Alexandros Papangelis, Aman Madaan, Angelina McMillan-Major, Anna Shvets, Ashish Upadhyay, Bingsheng Yao, Bryan Wilie, Chandra Bhagavatula, Chaobin You, Craig Thomson, Cristina Garbacea, Dakuo Wang, Daniel Deutsch, Deyi Xiong, Di Jin, Dimitra Gkatzia, Dragomir Radev, Elizabeth Clark, Esin Durmus, Faisal Ladhak, Filip Ginter, Genta Indra Winata, Hendrik Strobelt, Hiroaki Hayashi, Jekaterina Novikova, Jenna Kanerva, Jenny Chim, Jiawei Zhou, Jordan Clive, Joshua Maynez, João Sedoc, Juraj Juraska, Kaustubh D. Dhole, Khyathi Raghavi Chandu, Laura Perez-Beltrachini, Leonardo F. R. Ribeiro, Lewis Tunstall, Li Zhang, Mahima Pushkarna, Mathias Creutz, Michael White, Mihir Sanjay Kale, Moussa Kamal Eddine, Nico Daheim, Nishant Subramani, Ondrej Dusek, Paul Pu Liang, Pawan Sasanka Ammanamanchi, Qi Zhu, Ratish Puduppully, Reno Kriz, Rifat Shahriyar, Ronald Cardenas, Saad Mahamood, Salomey Osei, Samuel Cahyawijaya, Sanja Stajner, Sébastien Montella, Shailza Jolly, Simon Mille, Tahmid Hasan, Tianhao Shen, Tosin P. AMahidewumi, Vikas Raunak, Vipul Raheja, Vitaly Nikolaev, Vivian Tsai, Yacine Jernite, Ying Xu, Yisi Sang, Yixin Liu, and Yufang Hou. 2022.  Gemv2: Multilingual nlg benchmarking in a single line of code.  _CoRR_ , abs/2206.11249.
  * Grimes (2000) Barbara F Grimes. 2000.  _Ethnologue_.  SIL International, Dallas, TX.
  * Gultom and Wibowo (2017) Yohanes Gultom and Wahyu Catur Wibowo. 2017.  Automatic open domain information extraction from Indonesian text.  In _2017 International Workshop on Big Data and Information Security (IWBIS)_ , pages 23–30. IEEE.
  * Gunawan et al. (2021) Wahyu Gunawan, Herry Sujaini, and Tursina Tursina. 2021.  Analisis perbandingan nilai akurasi mekanisme attention bahdanau dan luong pada neural machine translation bahasa indonesia ke bahasa melayu ketapang dengan arsitektur recurrent neural network.  _Jurnal Edukasi dan Penelitian Informatika (JEPIN)_ , 7(3):488.
  * Guntara et al. (2020) Tri Wahyu Guntara, Alham Fikri Aji, and Radityo Eko Prasojo. 2020.  Benchmarking multidomain English-Indonesian machine translation.  In _Proceedings of the 13th Workshop on Building and Using Comparable Corpora_ , pages 35–43, Marseille, France. European Language Resources Association.
  * Gupta and Srikumar (2021) Ashim Gupta and Vivek Srikumar. 2021.  X-fact: A new benchmark dataset for multilingual fact checking.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pages 675–682.
  * Haidir and Purwarianti (2020) Muh Haidir and Ayu Purwarianti. 2020.  Short answer grading using contextual word embedding and linear regression.  _Jurnal Linguistik Komputasional_ , 3(2):54–61.
  * Haryono (2012) Akhmad Haryono. 2012.  _Perubahan dan perkembangan bahasa: Tinjauan historis dan sosiolinguistik_.  Ph.D. thesis, Udayana University.
  * Hasan et al. (2021) Tahmid Hasan, Abhik Bhattacharjee, Md Saiful Islam, Kazi Mubasshir, Yuan-Fang Li, Yong-Bin Kang, M Sohel Rahman, and Rifat Shahriyar. 2021.  Xl-sum: Large-scale multilingual abstractive summarization for 44 languages.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 4693–4703.
  * Hasbiansyah et al. (2016) Muhammad Hasbiansyah, Herry Sujaini, and Novi Safriadi. 2016.  Tuning for quality untuk uji akurasi mesin penerjemah statistik (mps) bahasa indonesia-bahasa dayak kanayatn.  _JUSTIN (Jurnal Sistem dan Teknologi Informasi)_ , 4(1):209–213.
  * Hidayatullah et al. (2020) Ahmad Fathan Hidayatullah, Siwi Cahyaningtyas, and Rheza Daffa Pamungkas. 2020.  Attention-based cnn-bilstm for dialect identification on javanese text.  _Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control_ , pages 317–324.
  * Hoesen and Purwarianti (2018) Devin Hoesen and Ayu Purwarianti. 2018.  Investigating bi-LSTM and CRF with POS tag embedding for Indonesian named entity tagger.  In _2018 International Conference on Asian Language Processing (IALP)_ , pages 35–38. IEEE.
  * Hu et al. (2020) Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020.  XTREME: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation.  In _Proceedings of the 37th International Conference on Machine Learning_ , volume 119 of _Proceedings of Machine Learning Research_ , pages 4411–4421. PMLR.
  * Ibrohim and Budi (2018) Muhammad Okky Ibrohim and Indra Budi. 2018.  A dataset and preliminaries study for abusive language detection in Indonesian social media.  _Procedia Computer Science_ , 135:222–229.  The 3rd International Conference on Computer Science and Computational Intelligence (ICCSCI 2018) : Empowering Smart Technology in Digital Era for a Better Life.
  * Ibrohim and Budi (2019) Muhammad Okky Ibrohim and Indra Budi. 2019.  Multi-label hate speech and abusive language detection in Indonesian Twitter.  In _Proceedings of the Third Workshop on Abusive Language Online_ , pages 46–57, Florence, Italy. Association for Computational Linguistics.
  * Ilmania et al. (2018) Arfinda Ilmania, Abdurrahman, Samuel Cahyawijaya, and Ayu Purwarianti. 2018.  Aspect detection and sentiment classification using deep neural network for indonesian aspect-based sentiment analysis.  In _2018 International Conference on Asian Language Processing (IALP)_ , pages 62–67.
  * Ilmy and Khodra (2020) Adylan Roaffa Ilmy and Masayu Leylia Khodra. 2020.  Parsing Indonesian sentence into Abstract Meaning Representation using machine learning approach.  _2020 7th International Conference on Advance Informatics: Concepts, Theory and Applications (ICAICTA)_ , pages 1–6.
  * Indonesia (2020) Kercedasan Artifisal Indonesia. 2020.  _National Strategy for Artificial Intelligence 2020-2045 (2020) (Indonesian)_.  Kercedasan Artifisal Indonesia.
  * Indrayana (2016) Danny Indrayana. 2016.  Meningkatkan akurasi pada mesin penerjemah bahasa indonesia ke bahasa melayu pontianak dengan part of speech.  _JUSTIN (Jurnal Sistem dan Teknologi Informasi)_ , 4(3):476–480.
  * Jahang and Meirina (2021) Benediktus Sridin Sulu Jahang and Zita Meirina. 2021.  1,3 juta anak di ntt belum bisa berbahasa indonesia.  Last accessed on 05/10/2021.
  * Jannati et al. (2018) Rini Jannati, Rahmad Mahendra, Cakra Wishnu Wardhana, and Mirna Adriani. 2018.  Stance classification towards political figures on blog writing.  In _2018 International Conference on Asian Language Processing (IALP)_ , pages 96–101. IEEE.
  * Jia et al. (2022) Ye Jia, Michelle Tadmor Ramanovich, Quan Wang, and Heiga Zen. 2022.  CVSS corpus and massively multilingual speech-to-speech translation.  In _Proceedings of Language Resources and Evaluation Conference (LREC)_ , pages 6691–6703.
  * Joshi et al. (2020) Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020.  The state and fate of linguistic diversity and inclusion in the NLP world.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 6282–6293, Online. Association for Computational Linguistics.
  * Kakwani et al. (2020) Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, Gokul N.C., Avik Bhattacharyya, Mitesh M. Khapra, and Pratyush Kumar. 2020.  IndicNLPSuite: Monolingual corpora, evaluation benchmarks and pre-trained multilingual language models for Indian languages.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 4948–4961, Online. Association for Computational Linguistics.
  * Karo et al. (2022) Ichwanul Muslim Karo Karo, Mohd Farhan Md Fudzee, Shahreen Kasim, and Azizul Azhar Ramli. 2022.  Sentiment analysis in karonese tweet using machine learning.  _Indonesian Journal of Electrical Engineering and Informatics (IJEEI)_ , 10(1):219–231.
  * Kiasati Desrul and Romadhony (2019) Dhamir Raniah Kiasati Desrul and Ade Romadhony. 2019.  Abusive language detection on Indonesian online news comments.  In _2019 International Seminar on Research of Information Technology and Intelligent Systems (ISRITI)_ , pages 320–325.
  * Kjartansson et al. (2018) Oddur Kjartansson, Supheakmungkol Sarin, Knot Pipatsrisawat, Martin Jansche, and Linne Ha. 2018.  Crowd-sourced speech corpora for javanese, sundanese, sinhala, nepali, and bangladeshi bengali.  In _Proceedings of the 6th International Workshop on Spoken Language Technologies for Under-Resourced Languages_ , pages 52–55.
  * Koto and Koto (2020) Fajri Koto and Ikhwan Koto. 2020.  Towards computational linguistics in minangkabau language: Studies on sentiment analysis and machine translation.  In _Proceedings of the 34th Pacific Asia Conference on Language, Information and Computation_ , pages 138–148.
  * Koto et al. (2020a) Fajri Koto, Jey Han Lau, and Timothy Baldwin. 2020a.  Liputan6: A large-scale Indonesian dataset for text summarization.  In _Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing_ , pages 598–608, Suzhou, China. Association for Computational Linguistics.
  * Koto et al. (2021) Fajri Koto, Jey Han Lau, and Timothy Baldwin. 2021.  IndoBERTweet: A pretrained language model for Indonesian Twitter with effective domain-specific vocabulary initialization.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 10660–10668.
  * Koto et al. (2020b) Fajri Koto, Afshin Rahimi, Jey Han Lau, and Timothy Baldwin. 2020b.  IndoLEM and IndoBERT: A benchmark dataset and pre-trained language model for Indonesian NLP.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 757–770.
  * Koto and Rahmaningtyas (2017) Fajri Koto and Gemala Y Rahmaningtyas. 2017.  Inset lexicon: Evaluation of a word list for Indonesian sentiment analysis in microblogs.  In _2017 International Conference on Asian Language Processing (IALP)_ , pages 391–394. IEEE.
  * Kumar et al. (2022) Aman Kumar, Himani Shrotriya, Prachi Sahu, Amogh Mishra, Raj Dabre, Ratish Puduppully, Anoop Kunchukuttan, Mitesh M. Khapra, and Pratyush Kumar. 2022.  IndicNLG benchmark: Multilingual datasets for diverse NLG tasks in Indic languages.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 5363–5394, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Kurniawan (2013) Eri Kurniawan. 2013.  _Sundanese complementation_.  Ph.D. thesis, The University of Iowa.
  * Kurniawan (2019) Kemal Kurniawan. 2019.  KaWAT: A word analogy task dataset for Indonesian.  _arXiv preprint arXiv:10.48550_.
  * Kurniawan and Louvan (2018) Kemal Kurniawan and Samuel Louvan. 2018.  Indosum: A new benchmark dataset for Indonesian text summarization.  In _2018 International Conference on Asian Language Processing (IALP)_ , pages 215–220.
  * Ladhak et al. (2020) Faisal Ladhak, Esin Durmus, Claire Cardie, and Kathleen Mckeown. 2020.  Wikilingua: A new benchmark dataset for cross-lingual abstractive summarization.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 4034–4048.
  * Larasati (2012) Septina Dian Larasati. 2012.  IDENTIC corpus: Morphologically enriched Indonesian-English parallel corpus.  In _Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12)_ , pages 902–906, Istanbul, Turkey. European Language Resources Association (ELRA).
  * Lestari (2006) Desi Puji Lestari. 2006.  A large vocabulary continuous speech recognition system for Indonesian language.  In _Proc. 15th Indonesian Scientific Conference in Japan (ISA-Japan), Hiroshima, Japan, 2006_ , pages 17–22.
  * Lewis (2009) M. Paul Lewis, editor. 2009.  _Ethnologue: Languages of the World_ , sixteenth edition.  SIL International, Dallas, TX, USA.
  * Liang et al. (2020) Yaobo Liang, Nan Duan, Yeyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, Xiaodong Fan, Ruofei Zhang, Rahul Agrawal, Edward Cui, Sining Wei, Taroon Bharti, Ying Qiao, Jiun-Hung Chen, Winnie Wu, Shuguang Liu, Fan Yang, Daniel Campos, Rangan Majumder, and Ming Zhou. 2020.  XGLUE: A new benchmark datasetfor cross-lingual pre-training, understanding and generation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6008–6018, Online. Association for Computational Linguistics.
  * Lin et al. (2022) Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyanov, and Xian Li. 2022.  Few-shot learning with multilingual language models.  In _Proceedings of EMNLP 2022_.
  * Lin et al. (2021) Zhaojiang Lin, Zihan Liu, Genta Indra Winata, Samuel Cahyawijaya, Andrea Madotto, Yejin Bang, Etsuko Ishii, and Pascale Fung. 2021.  Xpersona: Evaluating multilingual personalized chatbot.  In _Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI_ , pages 102–112.
  * Liu et al. (2021a) Fangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti, Siva Reddy, Nigel Collier, and Desmond Elliott. 2021a.  Visually grounded reasoning across languages and cultures.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 10467–10485.
  * Liu et al. (2021b) Qianchu Liu, Edoardo Maria Ponti, Diana McCarthy, Ivan Vulić, and Anna Korhonen. 2021b.  AM2iCo: Evaluating word meaning in context across low-resource languages with adversarial examples.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 7151–7162, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Lui and Baldwin (2012) Marco Lui and Timothy Baldwin. 2012.  langid. py: An off-the-shelf language identification tool.  In _Proceedings of the ACL 2012 system demonstrations_ , pages 25–30.
  * Mager et al. (2021) Manuel Mager, Arturo Oncevay, Abteen Ebrahimi, John Ortega, Annette Rios, Angela Fan, Ximena Gutierrez-Vasques, Luis Chiruzzo, Gustavo Giménez-Lugo, Ricardo Ramos, Ivan Vladimir Meza Ruiz, Rolando Coto-Solano, Alexis Palmer, Elisabeth Mager-Hois, Vishrav Chaudhary, Graham Neubig, Ngoc Thang Vu, and Katharina Kann. 2021.  Findings of the AmericasNLP 2021 shared task on open machine translation for indigenous languages of the Americas.  In _Proceedings of the First Workshop on Natural Language Processing for Indigenous Languages of the Americas_ , pages 202–217.
  * Mahendra et al. (2021) Rahmad Mahendra, Alham Fikri Aji, Samuel Louvan, Fahrurrozi Rahman, and Clara Vania. 2021.  IndoNLI: A natural language inference dataset for Indonesian.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 10511–10527, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Mahendra et al. (2018) Rahmad Mahendra, Heninggar Septiantri, Haryo Akbarianto Wibowo, Ruli Manurung, and Mirna Adriani. 2018.  Cross-lingual and supervised learning approach for Indonesian word sense disambiguation task.  In _Proceedings of the 9th Global Wordnet Conference_ , pages 245–250.
  * Mahfuzh et al. (2019) Miftahul Mahfuzh, Sidik Soleman, and Ayu Purwarianti. 2019.  Improving joint layer RNN based keyphrase extraction by using syntactical features.  In _2019 International Conference of Advanced Informatics: Concepts, Theory and Applications (ICAICTA)_ , pages 1–6. IEEE.
  * Majewska et al. (2023) Olga Majewska, Evgeniia Razumovskaia, Edoardo M. Ponti, Ivan Vulić, and Anna Korhonen. 2023.  Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation.  _Transactions of the Association for Computational Linguistics_ , 11:139–156.
  * McDonald et al. (2013) Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith Hall, Slav Petrov, Hao Zhang, Oscar Täckström, Claudia Bedini, Nuria Bertomeu Castelló, and Jungmee Lee. 2013.  Universal dependency annotation for multilingual parsing.  In _Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 92–97.
  * McMillan-Major et al. (2022) Angelina McMillan-Major, Zaid Alyafeai, Stella Biderman, Kimbo Chen, Francesco De Toni, Gérard Dupont, Hady Elsahar, Chris Emezue, Alham Fikri Aji, Suzana Ilić, Nurulaqilla Khamis, Colin Leong, Maraim Masoud, Aitor Soroa, Pedro Ortiz Suarez, Zeerak Talat, Daniel van Strien, and Yacine Jernite. 2022\.  Documenting geographically and contextually diverse data sources: The BigScience catalogue of language data and resources.  _arXiv preprint arXiv:2201.10066_.
  * Moeljadi (2012) David Moeljadi. 2012.  Usage of Indonesian possessive verbal predicates: A statistical analysis based on questionnaire and storytelling surveys.  In _APLL-5 conference. SOAS, University of London_.
  * Moeljadi (2017) David Moeljadi. 2017.  Building Jati: A treebank for Indonesian.  In _Proceedings of The 4th Atma Jaya Conference on Corpus Studies (ConCorps 4)_ , pages 1–9.
  * Moeljadi and Aminullah (2020) David Moeljadi and Zakariya Pamuji Aminullah. 2020.  Building the old Javanese Wordnet.  In _Proceedings of the Twelfth Language Resources and Evaluation Conference_ , pages 2940–2946, Marseille, France. European Language Resources Association.
  * Moeljadi et al. (2019) David Moeljadi, Aditya Kurniawan, and Debaditya Goswami. 2019.  Building cendana: a treebank for informal indonesian.  In _Proceedings of the 33rd Pacific Asia Conference on Language, Information and Computation_ , pages 156–164. Waseda Institute for the Study of Language and Information.
  * Muennighoff et al. (2022) Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, and Colin Raffel. 2022\.  Crosslingual generalization through multitask finetuning.  _arXiv preprint arXiv:2211.01786_.
  * Muis and Purwarianti (2020) Ferdiant Joshua Muis and Ayu Purwarianti. 2020.  Sequence-to-sequence learning for Indonesian automatic question generator.  In _2020 7th International Conference on Advance Informatics: Concepts, Theory and Applications (ICAICTA)_ , pages 1–6. IEEE.
  * Nakayama et al. (2019) Sahoko Nakayama, Andros Tjandra, Sakriani Sakti, and Satoshi Nakamura. 2019.  Zero-shot code-switching asr and tts with multilingual machine speech chain.  In _2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)_ , pages 964–971.
  * Nasution and Ayuningtyas (2020) Vivi Adryani Nasution and Niza Ayuningtyas. 2020.  The language choice of chinese community in medan: A sociolinguistics study.  _JOALL (Journal of Applied Linguistics And Literature)_ , 5(1):11–25.
  * Ningtyas et al. (2018) Della Widya Ningtyas, Herry Sujaini, and Novi Safriadi. 2018.  Penggunaan pivot language pada mesin penerjemah statistik bahasa inggris ke bahasa melayu sambas.  _Jurnal Edukasi dan Penelitian Informatika (JEPIN)_ , 4(2):173.
  * Nityasya et al. (2020) Made Nindyatama Nityasya, Haryo Akbarianto Wibowo, Radityo Eko Prasojo, and Alham Fikri Aji. 2020.  Costs to consider in adopting nlp for your business.  _arXiv preprint arXiv:2012.08958_.
  * NLLB Team et al. (2022) NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. 2022.  No language left behind: Scaling human-centered machine translation.  _arXiv preprint arXiv:2207.04672_.
  * Nomoto (2022) Hiroki Nomoto. 2022.  Kyokushoushugi ni motoduku heiretsu tsuriibanku no kouchiku [building a parallel treebank based on minimalism].  In _Proceedings of the Twenty-Eighth Annual Meeting of the Association for Natural Language Processing_ , pages 103–107.
  * Nomoto et al. (2018) Hiroki Nomoto, Kenji Okano, David Moeljadi, and Hideo Sawada. 2018.  TUFS Asian Language Parallel Corpus (TALPCO).  In _Proceedings of the Twenty-Fourth Annual Meeting of the Association for Natural Language Processing_.
  * Novitasari et al. (2020) Sashi Novitasari, Andros Tjandra, Sakriani Sakti, and Satoshi Nakamura. 2020.  Cross-lingual machine speech chain for Javanese, Sundanese, Balinese, and Bataks speech recognition and synthesis.  In _Proc. Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL)_ , pages 131–138, Marseille, France.
  * Nuranti et al. (2022) Eka Qadri Nuranti, Evi Yulianti, and Husna Sarirah Husin. 2022.  Predicting the category and the length of punishment in Indonesian courts based on previous court decision documents.  _Computers_ , 11(6):88.
  * Nurjanah (2018) Fajrin Nurjanah. 2018.  Pengembangan kemampuan berbahasa indonesia siswa sekolah dasar desa terpencil melalui metode karyawisata berbasis potensi lokal.  _FKIP e-PROCEEDING_ , pages 167–176.
  * Nurlaila et al. (2017) Afifah Nurlaila, Wiranto Wiranto, and Ristu Sapton. 2017.  Classification of Customers Emotion using Naive Bayes Classifier (Case Study: Natasha Skin Care).  In _ITSMART: Jurnal Teknologi dan Informasi_ , volume 6.
  * Ooms (2022) Jeroen Ooms. 2022.  _cld3: Google’s Compact Language Detector 3_.  Https://docs.ropensci.org/cld3/, https://github.com/ropensci/cld3 (devel) https://github.com/google/cld3 (upstream).
  * Pan et al. (2017) Xiaoman Pan, Boliang Zhang, Jonathan May, Joel Nothman, Kevin Knight, and Heng Ji. 2017.  Cross-lingual name tagging and linking for 282 languages.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1946–1958.
  * Park et al. (2019) Seunghyun Park, Seung Shin, Bado Lee, Junyeop Lee, Jaeheung Surh, Minjoon Seo, and Hwalsuk Lee. 2019.  {CORD}: A consolidated receipt dataset for post-{ocr} parsing.  In _Workshop on Document Intelligence at NeurIPS 2019_.
  * Park et al. (2021) Sungjoon Park, Jihyung Moon, Sungdong Kim, Won Ik Cho, Ji Yoon Han, Jangwon Park, Chisung Song, Junseong Kim, Youngsook Song, Taehwan Oh, Joohong Lee, Juhyun Oh, Sungwon Lyu, Younghoon Jeong, Inkwon Lee, Sangwoo Seo, Dongjun Lee, Hyunwoo Kim, Myeonghwa Lee, Seongbo Jang, Seungwon Do, Sunkyoung Kim, Kyungtae Lim, Jongwon Lee, Kyumin Park, Jamin Shin, Seonghyun Kim, Lucy Park, Alice Oh, Jung-Woo Ha, and Kyunghyun Cho. 2021.  KLUE: Korean language understanding evaluation.  In _Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)_.
  * Peng (2011) Anne E Peng. 2011.  Head-final and head-initial relative clauses in jambi teochew.  In _Online Proceedings of GLOW in Asia Workshop for Young Scholars_ , volume 262, page 276.
  * Pfeiffer et al. (2022) Jonas Pfeiffer, Naman Goyal, Xi Lin, Xian Li, James Cross, Sebastian Riedel, and Mikel Artetxe. 2022.  Lifting the curse of multilinguality by pre-training modular transformers.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3479–3495, Seattle, United States. Association for Computational Linguistics.
  * Pfeiffer et al. (2020) Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder. 2020.  MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 7654–7673, Online. Association for Computational Linguistics.
  * Phang et al. (2020) Jason Phang, Iacer Calixto, Phu Mon Htut, Yada Pruksachatkun, Haokun Liu, Clara Vania, Katharina Kann, and Samuel R. Bowman. 2020.  English intermediate-task training improves zero-shot cross-lingual transfer too.  In _Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing_ , pages 557–575, Suzhou, China. Association for Computational Linguistics.
  * Pimentel et al. (2021) Tiago Pimentel, Maria Ryskina, Sabrina J. Mielke, Shijie Wu, Eleanor Chodroff, Brian Leonard, Garrett Nicolai, Yustinus Ghanggo Ate, Salam Khalifa, Nizar Habash, Charbel El-Khaissi, Omer Goldman, Michael Gasser, William Lane, Matt Coler, Arturo Oncevay, Jaime Rafael Montoya Samame, Gema Celeste Silva Villegas, Adam Ek, Jean-Philippe Bernardy, Andrey Shcherbakov, Aziyana Bayyr-ool, Karina Sheifer, Sofya Ganieva, Matvey Plugaryov, Elena Klyachko, Ali Salehi, Andrew Krizhanovsky, Natalia Krizhanovsky, Clara Vania, Sardana Ivanova, Aelita Salchak, Christopher Straughn, Zoey Liu, Jonathan North Washington, Duygu Ataman, Witold Kieraś, Marcin Woliński, Totok Suhardijanto, Niklas Stoehr, Zahroh Nuriah, Shyam Ratan, Francis M. Tyers, Edoardo M. Ponti, Grant Aiton, Richard J. Hatcher, Emily Prud’hommeaux, Ritesh Kumar, Mans Hulden, Botond Barta, Dorina Lakatos, Gábor Szolnok, Judit Ács, Mohit Raj, David Yarowsky, Ryan Cotterell, Ben Ambridge, and Ekaterina Vylomova. 2021.  SIGMORPHON 2021 shared task on morphological reinflection: Generalization across languages.  In _Proceedings of the 18th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology_.
  * Pisceldo et al. (2008) Femphy Pisceldo, Rahmad Mahendra, Ruli Manurung, and I Wayan Arka. 2008.  A two-level morphological analyser for the Indonesian language.  In _Proceedings of the Australasian Language Technology Association Workshop 2008_ , pages 142–150, Hobart, Australia.
  * PLU et al. (2000) Amélie PLU, MA Chi Yuen, and Pascale Fung. 2000.  Salsa version 3.0: A single recognizer-based multilingual speech-based web browser.  In _Content-Based Multimedia Information Access - Volume 1_ , RIAO ’00, page 426–430, Paris, FRA. LE CENTRE DE HAUTES ETUDES INTERNATIONALES D’INFORMATIQUE DOCUMENTAIRE.
  * Ponti et al. (2020) Edoardo M. Ponti, Goran Glavaš, Olga Majewska, Qianchu Liu, Ivan Vulić, and Anna Korhonen. 2020.  XCOPA: A multilingual dataset for causal commonsense reasoning.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_.
  * Pratiwi et al. (2017) Inggrid Yanuar Risca Pratiwi, Rosa Andrie Asmara, and Faisal Rahutomo. 2017.  Study of hoax news detection using naïve Bayes classifier in Indonesian language.  In _2017 11th International Conference on Information & Communication Technology and System (ICTS)_, pages 73–78. IEEE.
  * Purwarianti and Crisdayanti (2019) Ayu Purwarianti and Ida Ayu Putu Ari Crisdayanti. 2019.  Improving bi-LSTM performance for Indonesian sentiment analysis using paragraph vector.  In _2019 International Conference of Advanced Informatics: Concepts, Theory and Applications (ICAICTA)_. IEEE.
  * Purwarianti et al. (2007) Ayu Purwarianti, Masatoshi Tsuchiya, and Seiichi Nakagawa. 2007.  A machine learning approach for Indonesian question answering system.  In _Artificial Intelligence and Applications_ , pages 573–578.
  * Putra et al. (2020) Oddy Virgantara Putra, Fathin Muhammad Wasmanson, Triana Harmini, and Shoffin Nahwa Utama. 2020.  Sundanese Twitter dataset for emotion classification.  In _2020 International Conference on Computer Engineering, Network, and Intelligent Multimedia (CENIM)_ , pages 391–395. IEEE.
  * Putri and Oh (2022) Rifki Afina Putri and Alice Oh. 2022.  IDK-MRC: Unanswerable questions for Indonesian machine reading comprehension.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 6918–6933, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Putri et al. (2021) Shofianina Dwi Ananda Putri, Muhammad Okky Ibrohim, and Indra Budi. 2021.  Abusive language and hate speech detection for javanese and sundanese languages in tweets: Dataset and preliminary study.  In _2021 11th International Workshop on Computer Science and Engineering, WCSE 2021_ , pages 461–465. International Workshop on Computer Science and Engineering (WCSE).
  * Qi et al. (2018) Ye Qi, Devendra Sachan, Matthieu Felix, Sarguna Padmanabhan, and Graham Neubig. 2018\.  When and why are pre-trained word embeddings useful for neural machine translation?  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pages 529–535.
  * Radford et al. (2022) Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022.  Robust speech recognition via large-scale weak supervision.
  * Riccosan et al. (2022) Riccosan, Karen Etania Saputra, Galih Dea Pratama, and Andry Chowanda. 2022.  Emotion dataset from Indonesian public opinion.  _Data in Brief_ , 43:108465.
  * Riza and Hakim (2009) Hammam Riza and Chairil Hakim. 2009.  Resource report: building parallel text corpora for multi-domain translation system.  In _Proceedings of the 7th Workshop on Asian Language Resources (ALR7)_ , pages 92–95.
  * Ruder (2022) Sebastian Ruder. 2022.  The State of Multilingual AI.  http://ruder.io/state-of-multilingual-ai/.
  * Sakti et al. (2004) Sakriani Sakti, Arry Akhmad Arman, Satoshi Nakamura, and Paulus Hutagaol. 2004.  Indonesian speech recognition for hearing and speaking impaired people.  In _Proc. International Conference on Spoken Language Processing (INTERSPEECH - ICSLP)_ , pages 1037–1040, Jeju Island, Korea.
  * Sakti et al. (2008a) Sakriani Sakti, Eka Kelana, Hammam Riza, Shinsuke Sakai, Konstantin Markov, and Satoshi Nakamura. 2008a.  Development of Indonesian large vocabulary continuous speech recognition system within A-STAR project.  In _Proceedings of the Workshop on Technologies and Corpora for Asia-Pacific Speech Translation (TCAST)_.
  * Sakti et al. (2008b) Sakriani Sakti, Ranniery Maia, Shinsuke Sakai, Tohru Shimizu, and Satoshi Nakamura. 2008b.  Development of HMM-based Indonesian speech synthesis.  In _Proc. Oriental COCOSDA_ , volume 1.
  * Sakti and Nakamura (2013) Sakriani Sakti and Satoshi Nakamura. 2013.  Towards language preservation: Design and collection of graphemically balanced and parallel speech corpora of Indonesian ethnic languages.  In _2013 International Conference Oriental COCOSDA held jointly with 2013 Conference on Asian Spoken Language Research and Evaluation (O-COCOSDA/CASLRE)_ , pages 1–5. IEEE.
  * Sakti and Nakamura (2014) Sakriani Sakti and Satoshi Nakamura. 2014.  Recent progress in developing grapheme-based speech recognition for Indonesian ethnic languages: Javanese, Sundanese, Balinese and Bataks.  In _Proc. 4th Workshop on Spoken Language Technologies for Under-Resourced Languages (SLTU 2014)_ , pages 46–52, St. Petersburg, Russia.
  * Sakti et al. (2012) Sakriani Sakti, Michael Paul, Andrew Finch, Xinhui Hu, Jinfu Ni, Noriyuki Kimura, Shigeki Matsuda, Chiori Hori, Yutaka Ashikari, Hisashi Kawai, Hideki Kashioka, Eiichiro Sumita, and Satoshi Nakamura. 2012.  Distributed speech translation technologies for multiparty multilingual communication.  _ACM Trans. Speech Lang. Process._ , 9(2).
  * Sakti et al. (2013) Sakriani Sakti, Michael Paul, Andrew Finch, Shinsuke Sakai, Thang Tat Vu, Noriyuki Kimura, Chiori Hori, Eiichiro Sumita, Satoshi Nakamura, Jun Park, Chai Wutiwiwatchai, Bo Xu, Hammam Riza, Karunesh Arora, Chi Mai Luong, and Haizhou Li. 2013.  A-STAR: Toward translating asian spoken languages.  _Computer Speech & Language_, 27(2):509–527.
  * Sakti et al. (2010) Sakriani Sakti, Shinsuke Sakai, Ryosuke Isotani, Hisashi Kawai, and Satoshi Nakamura. 2010.  Quality and intelligibility assessment of indonesian hmm-based speech synthesis system.  In _Proc. MALINDO_ , pages 51–57, Jakarta, Indonesia.
  * Saleh et al. (2018) Muhammad Saleh, Syukur Kholil, and Ahmad Tamrin Sikumbang. 2018.  Chinese ethnic communication pattern in the environment of indigenous people in lhokseumawe, indonesia.  _Budapest International Research and Critics Institute-Journal (BIRCI-Journal) Vol I (4)_ , pages 114–123.
  * Salsabila et al. (2018) Nikmatun Aliyah Salsabila, Yosef Ardhito Winatmoko, Ali Akbar Septiandri, and Ade Jamal. 2018.  Colloquial indonesian lexicon.  In _2018 International Conference on Asian Language Processing (IALP)_ , pages 226–229. IEEE.
  * Sani et al. (2012) Auliya Sani, Sakriani Sakti, Graham Neubig, Tomoki Toda, Adi Mulyanto, and Satoshi Nakamura. 2012.  Towards language preservation: Preliminary collection and vowel analysis of Indonesian ethnic speech data.  In _Proc. Oriental COCOSDA_ , pages 118–122, Macau, China.
  * Saputri et al. (2018) Mei Silviana Saputri, Rahmad Mahendra, and Mirna Adriani. 2018.  Emotion classification on Indonesian Twitter dataset.  In _2018 International Conference on Asian Language Processing (IALP)_. IEEE.
  * Scao et al. (2022) Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Benoît Sagot, Niklas Muennighoff, Albert Villanova del Moral, Olatunji Ruwase, Rachel Bawden, Stas Bekman, Angelina McMillan-Major, Iz Beltagy, Huu Nguyen, Lucile Saulnier, Samson Tan, Pedro Ortiz Suarez, Victor Sanh, Hugo Laurençon, Yacine Jernite, Julien Launay, Margaret Mitchell, Colin Raffel, Aaron Gokaslan, Adi Simhi, Aitor Soroa, Alham Fikri Aji, Amit Alfassy, Anna Rogers, Ariel Kreisberg Nitzav, Canwen Xu, Chenghao Mou, Chris Emezue, Christopher Klamm, Colin Leong, Daniel van Strien, David Ifeoluwa Adelani, Dragomir Radev, Eduardo González Ponferrada, Efrat Levkovizh, Ethan Kim, Eyal Bar Natan, Francesco De Toni, Gérard Dupont, Germán Kruszewski, Giada Pistilli, Hady Elsahar, Hamza Benyamina, Hieu Tran, Ian Yu, Idris Abdulmumin, Isaac Johnson, Itziar Gonzalez-Dios, Javier de la Rosa, Jenny Chim, Jesse Dodge, Jian Zhu, Jonathan Chang, Jörg Frohberg, Joseph Tobing, Joydeep Bhattacharjee, Khalid Almubarak, Kimbo Chen, Kyle Lo, Leandro Von Werra, Leon Weber, Long Phan, Loubna Ben allal, Ludovic Tanguy, Manan Dey, Manuel Romero Muñoz, Maraim Masoud, María Grandury, Mario Šaško, Max Huang, Maximin Coavoux, Mayank Singh, Mike Tian-Jian Jiang, Minh Chien Vu, Mohammad A. Jauhar, Mustafa Ghaleb, Nishant Subramani, Nora Kassner, Nurulaqilla Khamis, Olivier Nguyen, Omar Espejel, Ona de Gibert, Paulo Villegas, Peter Henderson, Pierre Colombo, Priscilla Amuok, Quentin Lhoest, Rheza Harliman, Rishi Bommasani, Roberto Luis López, Rui Ribeiro, Salomey Osei, Sampo Pyysalo, Sebastian Nagel, Shamik Bose, Shamsuddeen Hassan Muhammad, Shanya Sharma, Shayne Longpre, Somaieh Nikpoor, Stanislav Silberberg, Suhas Pai, Sydney Zink, Tiago Timponi Torrent, Timo Schick, Tristan Thrush, Valentin Danchev, Vassilina Nikoulina, Veronika Laippala, Violette Lepercq, Vrinda Prabhu, Zaid Alyafeai, Zeerak Talat, Arun Raja, Benjamin Heinzerling, Chenglei Si, Elizabeth Salesky, Sabrina J. Mielke, Wilson Y. Lee, Abheesht Sharma, Andrea Santilli, Antoine Chaffin, Arnaud Stiegler, Debajyoti Datta, Eliza Szczechla, Gunjan Chhablani, Han Wang, Harshit Pandey, Hendrik Strobelt, Jason Alan Fries, Jos Rozen, Leo Gao, Lintang Sutawika, M Saiful Bari, Maged S. Al-shaibani, Matteo Manica, Nihal Nayak, Ryan Teehan, Samuel Albanie, Sheng Shen, Srulik Ben-David, Stephen H. Bach, Taewoon Kim, Tali Bers, Thibault Fevry, Trishala Neeraj, Urmish Thakker, Vikas Raunak, Xiangru Tang, Zheng-Xin Yong, Zhiqing Sun, Shaked Brody, Yallow Uri, Hadar Tojarieh, Adam Roberts, Hyung Won Chung, Jaesung Tae, Jason Phang, Ofir Press, Conglong Li, Deepak Narayanan, Hatim Bourfoune, Jared Casper, Jeff Rasley, Max Ryabinin, Mayank Mishra, Minjia Zhang, Mohammad Shoeybi, Myriam Peyrounette, Nicolas Patry, Nouamane Tazi, Omar Sanseviero, Patrick von Platen, Pierre Cornette, Pierre François Lavallée, Rémi Lacroix, Samyam Rajbhandari, Sanchit Gandhi, Shaden Smith, Stéphane Requena, Suraj Patil, Tim Dettmers, Ahmed Baruwa, Amanpreet Singh, Anastasia Cheveleva, Anne-Laure Ligozat, Arjun Subramonian, Aurélie Névéol, Charles Lovering, Dan Garrette, Deepak Tunuguntla, Ehud Reiter, Ekaterina Taktasheva, Ekaterina Voloshina, Eli Bogdanov, Genta Indra Winata, Hailey Schoelkopf, Jan-Christoph Kalo, Jekaterina Novikova, Jessica Zosa Forde, Jordan Clive, Jungo Kasai, Ken Kawamura, Liam Hazan, Marine Carpuat, Miruna Clinciu, Najoung Kim, Newton Cheng, Oleg Serikov, Omer Antverg, Oskar van der Wal, Rui Zhang, Ruochen Zhang, Sebastian Gehrmann, Shani Pais, Tatiana Shavrina, Thomas Scialom, Tian Yun, Tomasz Limisiewicz, Verena Rieser, Vitaly Protasov, Vladislav Mikhailov, Yada Pruksachatkun, Yonatan Belinkov, Zachary Bamberger, Zdeněk Kasner, Alice Rueda, Amanda Pestana, Amir Feizpour, Ammar Khan, Amy Faranak, Ana Santos, Anthony Hevia, Antigona Unldreaj, Arash Aghagol, Arezoo Abdollahi, Aycha Tammour, Azadeh HajiHosseini, Bahareh Behroozi, Benjamin Ajibade, Bharat Saxena, Carlos Muñoz Ferrandis, Danish Contractor, David Lansky, Davis David, Douwe Kiela, Duong A. Nguyen, Edward Tan, Emi Baylor, Ezinwanne Ozoani, Fatima Mirza, Frankline Ononiwu, Habib Rezanejad, Hessie Jones, Indrani Bhattacharya, Irene Solaiman, Irina Sedenko, Isar Nejadgholi, Jesse Passmore, Josh Seltzer, Julio Bonis Sanz, Karen Fort, Livia Dutra, Mairon Samagaio, Maraim Elbadri, Margot Mieskes, Marissa Gerchick, Martha Akinlolu, Michael McKenna, Mike Qiu, Muhammed Ghauri, Mykola Burynok, Nafis Abrar, Nazneen Rajani, Nour Elkott, Nour Fahmy, Olanrewaju Samuel, Ran An, Rasmus Kromann, Ryan Hao, Samira Alizadeh, Sarmad Shubber, Silas Wang, Sourav Roy, Sylvain Viguier, Thanh Le, Tobi Oyebade, Trieu Le, Yoyo Yang, Zach Nguyen, Abhinav Ramesh Kashyap, Alfredo Palasciano, Alison Callahan, Anima Shukla, Antonio Miranda-Escalada, Ayush Singh, Benjamin Beilharz, Bo Wang, Caio Brito, Chenxi Zhou, Chirag Jain, Chuxin Xu, Clémentine Fourrier, Daniel León Periñán, Daniel Molano, Dian Yu, Enrique Manjavacas, Fabio Barth, Florian Fuhrimann, Gabriel Altay, Giyaseddin Bayrak, Gully Burns, Helena U. Vrabec, Imane Bello, Ishani Dash, Jihyun Kang, John Giorgi, Jonas Golde, Jose David Posada, Karthik Rangasai Sivaraman, Lokesh Bulchandani, Lu Liu, Luisa Shinzato, Madeleine Hahn de Bykhovetz, Maiko Takeuchi, Marc Pàmies, Maria A Castillo, Marianna Nezhurina, Mario Sänger, Matthias Samwald, Michael Cullan, Michael Weinberg, Michiel De Wolf, Mina Mihaljcic, Minna Liu, Moritz Freidank, Myungsun Kang, Natasha Seelam, Nathan Dahlberg, Nicholas Michio Broad, Nikolaus Muellner, Pascale Fung, Patrick Haller, Ramya Chandrasekhar, Renata Eisenberg, Robert Martin, Rodrigo Canalli, Rosaline Su, Ruisi Su, Samuel Cahyawijaya, Samuele Garda, Shlok S Deshmukh, Shubhanshu Mishra, Sid Kiblawi, Simon Ott, Sinee Sang-aroonsiri, Srishti Kumar, Stefan Schweter, Sushil Bharati, Tanmay Laud, Théo Gigant, Tomoya Kainuma, Wojciech Kusa, Yanis Labrak, Yash Shailesh Bajaj, Yash Venkatraman, Yifan Xu, Yingxin Xu, Yu Xu, Zhe Tan, Zhongli Xie, Zifan Ye, Mathilde Bras, Younes Belkada, and Thomas Wolf. 2022.  Bloom: A 176b-parameter open-access multilingual language model.
  * Schuhmann et al. (2021) Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. 2021.  Laion-400m: Open dataset of clip-filtered 400 million image-text pairs.
  * Seelawi et al. (2021) Haitham Seelawi, Ibraheem Tuffaha, Mahmoud Gzawi, Wael Farhan, Bashar Talafha, Riham Badawi, Zyad Sober, Oday Al-Dweik, Abed Alhakim Freihat, and Hussein Al-Natsheh. 2021.  ALUE: Arabic language understanding evaluation.  In _Proceedings of the Sixth Arabic Natural Language Processing Workshop_ , pages 173–184, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  * Septiandri and Winatmoko (2020) Ali Akbar Septiandri and Yosef Ardhito Winatmoko. 2020.  Ukara 1.0 challenge track 1: automatic short-answer scoring in Bahasa Indonesia.  _arXiv preprint arXiv:2002.12540_.
  * Setya and Mahendra (2023) Ken Nabila Setya and Rahmad Mahendra. 2023.  Semi-supervised textual entailment on indonesian wikipedia data.  In _Computational Linguistics and Intelligent Text Processing_ , pages 416–427, Cham. Springer Nature Switzerland.
  * Shen et al. (2020) Sheng Shen, Zhen Dong, Jiayu Ye, Linjian Ma, Zhewei Yao, Amir Gholami, Michael W. Mahoney, and Kurt Keutzer. 2020.  Q-BERT: Hessian based ultra low precision quantization of BERT.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 34(05):8815–8821.
  * Shi et al. (2022) Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. 2022.  Language models are multilingual chain-of-thought reasoners.  _arXiv preprint arXiv:2210.03057_.
  * Siallagan and Alfina (2023) Emmanuella Anggi Siallagan and Ika Alfina. 2023.  Poetry Generation for Indonesian Pantun : Comparison Between SeqGAN and GPT-2.  _Jurnal Ilmu Komputer dan Informasi (Journal of Computer Science and Information)_ , 16(1):59–67.
  * Singh et al. (2022) Amanpreet Singh, Ronghang Hu, Vedanuj Goswami, Guillaume Couairon, Wojciech Galuba, Marcus Rohrbach, and Douwe Kiela. 2022.  FLAVA: A foundational language and vision alignment model.  In _CVPR_.
  * Sinurat (2019) Ray Andrew Obaja Sinurat. 2019.  Pembangkitan deskripsi gambar dalam bahasa indonesia dengan pendekatan semantic compositional networks.  Master’s thesis, Teknik Informatika, Institut Teknologi Bandung.
  * Siregar et al. (2014) Masitowarni Siregar, Syamsul Bahri, and Dedi Sanjaya. 2014.  Code switching and code mixing in Indonesia: Study in sociolinguistics.  _English Language and Literature Studies_ , 4(1):77–92.
  * Sneddon (2003) James Neil Sneddon. 2003.  _The Indonesian language: Its history and role in modern society_.  UNSW Press, Sydney.
  * Sodimana et al. (2018) Keshan Sodimana, Pasindu-De Silva, Supheakmungkol Sarin, Oddur Kjartansson, Martin Jansche, Knot Pipatsrisawat, and Linne Ha. 2018.  A step-by-step process for building tts voices using open source data and frameworks for bangla, javanese, khmer, nepali, sinhala, and sundanese.  In _Proc. The 6th Intl. Workshop on Spoken Language Technologies for Under-Resourced Languages_ , pages 52–55.
  * Soeparno (2015) Soeparno. 2015.  Kerancuan fono-ortografis dan orto-fonologis bahasa Indonesia ragam lisan dan tulis.  _Diksi_ , 12(2).
  * Sommerlot (2020) Carly J Sommerlot. 2020.  _On the Syntax of West Kalimantan: Asymmetries and A’-Movement in Malayic and Land Dayak Languages_.  Ph.D. thesis, The University of Texas at Arlington.
  * Srivastava et al. (2022) Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Ambrose Slone, Ameet Rahane, Anantharaman S. Iyer, Anders Andreassen, Andrea Madotto, Andrea Santilli, Andreas Stuhlmüller, Andrew Dai, Andrew La, Andrew Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabharwal, Austin Herrick, Avia Efrat, Aykut Erdem, Ayla Karakaş, B. Ryan Roberts, Bao Sheng Loe, Barret Zoph, Bartłomiej Bojanowski, Batuhan Özyurt, Behnam Hedayatnia, Behnam Neyshabur, Benjamin Inden, Benno Stein, Berk Ekmekci, Bill Yuchen Lin, Blake Howald, Cameron Diao, Cameron Dour, Catherine Stinson, Cedrick Argueta, César Ferri Ramírez, Chandan Singh, Charles Rathkopf, Chenlin Meng, Chitta Baral, Chiyu Wu, Chris Callison-Burch, Chris Waites, Christian Voigt, Christopher D. Manning, Christopher Potts, Cindy Ramirez, Clara E. Rivera, Clemencia Siro, Colin Raffel, Courtney Ashcraft, Cristina Garbacea, Damien Sileo, Dan Garrette, Dan Hendrycks, Dan Kilman, Dan Roth, Daniel Freeman, Daniel Khashabi, Daniel Levy, Daniel Moseguí González, Danielle Perszyk, Danny Hernandez, Danqi Chen, Daphne Ippolito, Dar Gilboa, David Dohan, David Drakard, David Jurgens, Debajyoti Datta, Deep Ganguli, Denis Emelin, Denis Kleyko, Deniz Yuret, Derek Chen, Derek Tam, Dieuwke Hupkes, Diganta Misra, Dilyar Buzan, Dimitri Coelho Mollo, Diyi Yang, Dong-Ho Lee, Ekaterina Shutova, Ekin Dogus Cubuk, Elad Segal, Eleanor Hagerman, Elizabeth Barnes, Elizabeth Donoway, Ellie Pavlick, Emanuele Rodola, Emma Lam, Eric Chu, Eric Tang, Erkut Erdem, Ernie Chang, Ethan A. Chi, Ethan Dyer, Ethan Jerzak, Ethan Kim, Eunice Engefu Manyasi, Evgenii Zheltonozhskii, Fanyue Xia, Fatemeh Siar, Fernando Martínez-Plumed, Francesca Happé, Francois Chollet, Frieda Rong, Gaurav Mishra, Genta Indra Winata, Gerard de Melo, Germán Kruszewski, Giambattista Parascandolo, Giorgio Mariani, Gloria Wang, Gonzalo Jaimovitch-López, Gregor Betz, Guy Gur-Ari, Hana Galijasevic, Hannah Kim, Hannah Rashkin, Hannaneh Hajishirzi, Harsh Mehta, Hayden Bogar, Henry Shevlin, Hinrich Schütze, Hiromu Yakura, Hongming Zhang, Hugh Mee Wong, Ian Ng, Isaac Noble, Jaap Jumelet, Jack Geissinger, Jackson Kernion, Jacob Hilton, Jaehoon Lee, Jaime Fernández Fisac, James B. Simon, James Koppel, James Zheng, James Zou, Jan Kocoń, Jana Thompson, Jared Kaplan, Jarema Radom, Jascha Sohl-Dickstein, Jason Phang, Jason Wei, Jason Yosinski, Jekaterina Novikova, Jelle Bosscher, Jennifer Marsh, Jeremy Kim, Jeroen Taal, Jesse Engel, Jesujoba Alabi, Jiacheng Xu, Jiaming Song, Jillian Tang, Joan Waweru, John Burden, John Miller, John U. Balis, Jonathan Berant, Jörg Frohberg, Jos Rozen, Jose Hernandez-Orallo, Joseph Boudeman, Joseph Jones, Joshua B. Tenenbaum, Joshua S. Rule, Joyce Chua, Kamil Kanclerz, Karen Livescu, Karl Krauth, Karthik Gopalakrishnan, Katerina Ignatyeva, Katja Markert, Kaustubh D. Dhole, Kevin Gimpel, Kevin Omondi, Kory Mathewson, Kristen Chiafullo, Ksenia Shkaruta, Kumar Shridhar, Kyle McDonell, Kyle Richardson, Laria Reynolds, Leo Gao, Li Zhang, Liam Dugan, Lianhui Qin, Lidia Contreras-Ochando, Louis-Philippe Morency, Luca Moschella, Lucas Lam, Lucy Noble, Ludwig Schmidt, Luheng He, Luis Oliveros Colón, Luke Metz, Lütfi Kerem Şenel, Maarten Bosma, Maarten Sap, Maartje ter Hoeve, Maheen Farooqi, Manaal Faruqui, Mantas Mazeika, Marco Baturan, Marco Marelli, Marco Maru, Maria Jose Ramírez Quintana, Marie Tolkiehn, Mario Giulianelli, Martha Lewis, Martin Potthast, Matthew L. Leavitt, Matthias Hagen, Mátyás Schubert, Medina Orduna Baitemirova, Melody Arnaud, Melvin McElrath, Michael A. Yee, Michael Cohen, Michael Gu, Michael Ivanitskiy, Michael Starritt, Michael Strube, Michał Swędrowski, Michele Bevilacqua, Michihiro Yasunaga, Mihir Kale, Mike Cain, Mimee Xu, Mirac Suzgun, Mo Tiwari, Mohit Bansal, Moin Aminnaseri, Mor Geva, Mozhdeh Gheini, Mukund Varma T, Nanyun Peng, Nathan Chi, Nayeon Lee, Neta Gur-Ari Krakover, Nicholas Cameron, Nicholas Roberts, Nick Doiron, Nikita Nangia, Niklas Deckers, Niklas Muennighoff, Nitish Shirish Keskar, Niveditha S. Iyer, Noah Constant, Noah Fiedel, Nuan Wen, Oliver Zhang, Omar Agha, Omar Elbaghdadi, Omer Levy, Owain Evans, Pablo Antonio Moreno Casares, Parth Doshi, Pascale Fung, Paul Pu Liang, Paul Vicol, Pegah Alipoormolabashi, Peiyuan Liao, Percy Liang, Peter Chang, Peter Eckersley, Phu Mon Htut, Pinyu Hwang, Piotr Miłkowski, Piyush Patil, Pouya Pezeshkpour, Priti Oli, Qiaozhu Mei, Qing Lyu, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker, Ramón Risco Delgado, Raphaël Millière, Rhythm Garg, Richard Barnes, Rif A. Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand, Roman Novak, Roman Sitelew, Ronan LeBras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Ruslan Salakhutdinov, Ryan Chi, Ryan Lee, Ryan Stovall, Ryan Teehan, Rylan Yang, Sahib Singh, Saif M. Mohammad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Samuel R. Bowman, Samuel S. Schoenholz, Sanghyun Han, Sanjeev Kwatra, Sarah A. Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebastian Schuster, Sepideh Sadeghi, Shadi Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asaadi, Shixiang Shane Gu, Shubh Pachchigar, Shubham Toshniwal, Shyamolima Upadhyay, Debnath, Siamak Shakeri, Simon Thormeyer, Simone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo-Hwan Lee, Spencer Torene, Sriharsha Hatwar, Stanislas Dehaene, Stefan Divic, Stefano Ermon, Stella Biderman, Stephanie Lin, Stephen Prasad, Steven T. Piantadosi, Stuart M. Shieber, Summer Misherghi, Svetlana Kiritchenko, Swaroop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq Ali, Tatsu Hashimoto, Te-Lin Wu, Théo Desbordes, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo Schick, Timofei Kornev, Timothy Telleen-Lawton, Titus Tunduny, Tobias Gerstenberg, Trenton Chang, Trishala Neeraj, Tushar Khot, Tyler Shultz, Uri Shaham, Vedant Misra, Vera Demberg, Victoria Nyamai, Vikas Raunak, Vinay Ramasesh, Vinay Uday Prabhu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders, William Zhang, Wout Vossen, Xiang Ren, Xiaoyu Tong, Xinran Zhao, Xinyi Wu, Xudong Shen, Yadollah Yaghoobzadeh, Yair Lakretz, Yangqiu Song, Yasaman Bahri, Yejin Choi, Yichi Yang, Yiding Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yufang Hou, Yuntao Bai, Zachary Seid, Zhuoye Zhao, Zijian Wang, Zijie J. Wang, Zirui Wang, and Ziyi Wu. 2022.  Beyond the imitation game: Quantifying and extrapolating the capabilities of language models.  _arXiv preprint arXiv:2206.04615_.
  * Stenberg (2015) Josh Stenberg. 2015.  Multilingualism and the west kalimantan hakka.  In _Multilingualism in the Chinese diaspora worldwide_ , pages 123–140. Routledge.
  * Suherik and Purwarianti (2017) Gilang Julian Suherik and Ayu Purwarianti. 2017.  Experiments on coreference resolution for Indonesian language with lexical and shallow syntactic features.  In _2017 5th International Conference on Information and Communication Technology (ICoIC7)_.
  * Sujaini (2019) Herry Sujaini. 2019.  Penggunaan bahasa indonesia sebagai pivot language pada mesin penerjemah madura-sunda dengan metode transfer dan triangulation.  _Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi)_ , 3(2):170–175.
  * Sujaini (2020) Herry Sujaini. 2020.  Improving the role of language model in statistical machine translation (Indonesian-Javanese).  _International Journal of Electrical and Computer Engineering (IJECE)_ , 10(2):2102–2109.
  * Tang et al. (2021) Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2021.  Multilingual translation from denoising pre-training.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 3450–3466, Online. Association for Computational Linguistics.
  * Tho et al. (2021) C Tho, Y Heryadi, L Lukas, and A Wibowo. 2021.  Code-mixed sentiment analysis of indonesian language and javanese language using lexicon based approach.  _Journal of Physics: Conference Series_ , 1869(1):012084.
  * Trisedya and Inastra (2014) Bayu Distiawan Trisedya and Dyah Inastra. 2014.  Creating Indonesian-Javanese parallel corpora using Wikipedia articles.  In _2014 International Conference on Advanced Computer Science and Information System_ , pages 239–245.
  * Uchibori and Shibata (1988) Motomitsu Uchibori and Norio Shibata. 1988.  Ngaju-Dayak Language.  _The Sanseido Encyclopedia of Linguistics: Languages of The World_ , 1:1156–1160.
  * Valk and Alumäe (2021) Jörgen Valk and Tanel Alumäe. 2021.  Voxlingua107: a dataset for spoken language recognition.  In _2021 IEEE Spoken Language Technology Workshop (SLT)_ , pages 652–658. IEEE.
  * van der Goot et al. (2021a) Rob van der Goot, Alan Ramponi, Arkaitz Zubiaga, Barbara Plank, Benjamin Muller, Iñaki San Vicente Roncal, Nikola Ljubešic, Özlem Çetinoğlu, Rahmad Mahendra, Talha Çolakoglu, Timothy Baldwin, Tommaso Caselli, and Wladimir Sidorenko. 2021a.  MultiLexNorm: A shared task on multilingual lexical normalization.  In _Seventh Workshop on Noisy User-generated Text (W-NUT 2021)_ , pages 493–509. Association for Computational Linguistics.
  * van der Goot et al. (2021b) Rob van der Goot, Ibrahim Sharaf, Aizhan Imankulova, Ahmet Üstün, Marija Stepanovic, Alan Ramponi, Siti Oryza Khairunnisa, Mamoru Komachi, and Barbara Plank. 2021b.  From masked language modeling to translation: Non-english auxiliary tasks improve zero-shot spoken language understanding.  In _NAACL-HLT_.
  * Veniranda (2015) Yohana Veniranda. 2015.  _Perfective aspect and negation in Pontianak Teochew_.  Ph.D. thesis, University of Delaware.
  * Wahyuni et al. (2019) Mirda Wahyuni, Herry Sujaini, and Hafiz Muhardi. 2019.  Pengaruh kuantitas korpus monolingual terhadap akurasi mesin penerjemah statistik.  _Jurnal Sistem dan Teknologi Informasi (JUSTIN)_ , 7:20.
  * Wang et al. (2018) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018.  GLUE: A multi-task benchmark and analysis platform for natural language understanding.  In _Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_ , pages 353–355, Brussels, Belgium. Association for Computational Linguistics.
  * Wang et al. (2021) Changhan Wang, Anne Wu, Jiatao Gu, and Juan Pino. 2021.  CoVoST 2 and Massively Multilingual Speech Translation.  In _Proc. Interspeech 2021_ , pages 2247–2251.
  * Weda (2016) Sukardi Weda. 2016.  Syntactic variation of buginese, a language in austronesian great family.  _Kongres Internasional Masyarakat Linguistik Indonesia (KIMLI) 2016_ , pages 838–841.
  * Wibowo et al. (2021) Haryo Akbarianto Wibowo, Made Nindyatama Nityasya, Afra Feyza Akyürek, Suci Fitriany, Alham Fikri Aji, Radityo Eko Prasojo, and Derry Tanti Wijaya. 2021.  IndoCollex: A testbed for morphological transformation of Indonesian colloquial words.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 3170–3183, Online. Association for Computational Linguistics.
  * Wibowo et al. (2020) Haryo Akbarianto Wibowo, Tatag Aziz Prawiro, Muhammad Ihsan, Alham Fikri Aji, Radityo Eko Prasojo, Rahmad Mahendra, and Suci Fitriany. 2020.  Semi-supervised low-resource style transfer of Indonesian informal to formal language with iterative forward-translation.  In _2020 International Conference on Asian Language Processing (IALP)_ , pages 310–315. IEEE.
  * Wilie et al. (2020) Bryan Wilie, Karissa Vincentio, Genta Indra Winata, Samuel Cahyawijaya, Xiaohong Li, Zhi Yuan Lim, Sidik Soleman, Rahmad Mahendra, Pascale Fung, Syafri Bahar, and Ayu Purwarianti. 2020.  IndoNLU: Benchmark and resources for evaluating Indonesian natural language understanding.  In _Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing_ , pages 843–857.
  * William and Sari (2020) Andika William and Yunita Sari. 2020.  CLICK-ID: A novel dataset for Indonesian clickbait headlines.  _Data in Brief_ , 32:106231.
  * Winata et al. (2023) Genta Indra Winata, Alham Fikri Aji, Samuel Cahyawijaya, Rahmad Mahendra, Fajri Koto, Ade Romadhony, Kemal Kurniawan, David Moeljadi, Radityo Eko Prasojo, Pascale Fung, Timothy Baldwin, Jey Han Lau, Rico Sennrich, and Sebastian Ruder. 2023.  NusaX: Multilingual parallel sentiment dataset for 10 Indonesian local languages.  In _Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics_ , pages 815–834, Dubrovnik, Croatia. Association for Computational Linguistics.
  * Winata et al. (2020) Genta Indra Winata, Samuel Cahyawijaya, Zhaojiang Lin, Zihan Liu, and Pascale Fung. 2020.  Lightweight and efficient end-to-end speech recognition using low-rank transformer.  In _ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 6144–6148.
  * Wirawan (2022) Cahya Wirawan. 2022.  indonesian-nlp/librivox-indonesia.
  * Wongso et al. (2022) Wilson Wongso, Henry Lucky, and Derwin Suhartono. 2022.  Pre-trained transformer-based language models for sundanese.  _Journal of Big Data_ , 9(1):1–17.
  * Wongso et al. (2021) Wilson Wongso, David Samuel Setiawan, and Derwin Suhartono. 2021.  Causal and masked language modeling of javanese language using transformer-based architectures.  In _2021 International Conference on Advanced Computer Science and Information Systems (ICACSIS)_ , pages 1–7.
  * Woollams (2005) Geoffrey Woollams. 2005.  Karo batak.  _The Austronesian Languages of Asia and Madagascar_ , pages 534–561.
  * Xu et al. (2020) Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Kyle Richardson, and Zhenzhong Lan. 2020.  CLUE: A Chinese language understanding evaluation benchmark.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 4762–4772, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  * Xue et al. (2021) Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021.  mT5: A massively multilingual pre-trained text-to-text transformer.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 483–498, Online. Association for Computational Linguistics.
  * Yulianti et al. (2021) Evi Yulianti, Ajmal Kurnia, Mirna Adriani, and Yoppy Setyo Duto. 2021.  Normalisation of Indonesian-English code-mixed text and its effect on emotion classification.  _Int. J. Adv. Comput. Sci. Appl._ , 12(11).
  * Zhang et al. (2020) Wei Zhang, Lu Hou, Yichun Yin, Lifeng Shang, Xiao Chen, Xin Jiang, and Qun Liu. 2020\.  TernaryBERT: Distillation-aware ultra-low bit BERT.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 509–521, Online. Association for Computational Linguistics.

Appendix

##  Appendix A Languages in NusaCrowd

Table A provides the language code, name, and family for all 19 languages
listed in NusaCrowd. The language family information is collected from
Ethnologue Eberhard et al. (2021). We follow the ISO 639-3
standard131313https://iso639-3.sil.org/ for language coding in NusaCrowd. The
language tree of all languages in NusaCrowd is shown in Figure A.

Lang Code | Lang Name | Family
---|---|---
ace | Acehnese | MP
abl | Lampung Nyo | MP
ban | Balinese | MP
bbc | Batak Toba | MP
bjn | Banjar | MP
btk | Batak | MP
btx | Batak Karo | MP
bug | Buginese | MP
hak | Hakka/Khek | ST
ind | Indonesian | MP
jav | Javanese | MP
mad | Madura | MP
min | Minangkabau | MP
nan | Min Nan (Teochew) | ST
nij | Ngaju | MP
sun | Sundanese | MP
tpi | Tok Pisin | CR
tdt | Tetun Dili | CR
xdy | Malayic Dayak | MP

Table A: Language codes and its complete names for all 19 languages listed in
NusaCrowd. MP denotes Malayo-Polynesian, CR denotes Creole, and ST denotes
Sino-Tibetan language family.

Figure A: Language family tree for all the languages covered in NusaCrowd.
Most languages are Austronesian with two Creole languages and two other
languages are under Sino-Tibetan language family.

Acehnese (ace) is a language spoken mainly in the Aceh province. Although it
is the de facto language of provincial identity of Aceh, language use is
shifting to Indonesian in urban areas. Acehnese has features typical of the
Mon-Khmer languages of mainland Southeast Asia, a result of its former status
as part of the early Chamic dialect continuum on the coast of Vietnam. It has
at least ten contrasting vowels and as many distinct diphthongs, as well as
voiceless aspirated stops and murmured voiced stops Blust (2013). In addition
to the large number of diphthongs, it has a high percentage of monosyllabic
root morphemes. Prefixes and infixes play an active role while suffixes are
absent Durie (1985). It is of the ‘active’ or so-called ‘Split-S’ type: some
intransitive verbs take arguments, which have the properties of ‘transitive
subjects’ while others take arguments with the properties of ‘transitive
objects’ Durie (1988).

Lampung Nyo (abl) is a language spoken in three enclaves east between Kanan
and Seputih rivers in Lampung province. It is one of the three languages under
the subgroup Lampung. The other two languages are Komering and Lampung Api. It
has four dialects: Abung, Tulangbawang, Sukadana, and Melinting, with 77% of
lexical similarity among dialects. It was written in Kaganga script but it is
written mainly in Latin script Eberhard et al. (2021).

Balinese (ban) is a language spoken mainly in the Bali province and in the
West Nusa Tenggara province. It has three main dialects: Highland Balinese,
Lowland Balinese, and Nusa Penida. It is mainly written in the Latin script
since the early 20th century although it has its own Balinese script. The word
order in Balinese is SVO. It is non-tonal and has 17 consonant and 6 vowel
phonemes. Stress is on the penultimate syllable. It has three sociolinguistic
registers. Regarding patterns of verb affixation, Balinese is an ‘active’ or
‘split-S’ language: verbs with Undergoer-like subject arguments are marked in
one way (with a ‘zero prefix’), while verbs with Actor-like subject
arguments—intransitive or transitive—are marked in another (either with the
nasal prefix ‘N-’, or with ‘ma-’) Arka (2003).

Toba Batak (bbc) is a language spoken in the North Sumatra province. Similarly
to Acehnese, it is slowly being replaced by Indonesian in urban and migrant
areas. It used to be written in the Batak script but is mainly written in
Latin script now. The Batak languages are predicate-initial, and have verb
systems reminiscent of Philippine languages, although they differ from them in
many details Blust (2013).

Banjarese (bjn) is a language spoken in Kalimantan (Central, East, South, and
West Kalimantan provinces). It became a language of wider communication
through trade in the market, in business, and in media. It is dominant in the
South Kalimantan Province and also growing rapidly in the Central and Eastern
Kalimantan provinces. It has two main dialects: Kuala and Hulu dialects.
Although it is a Malayic language, it has many Javanese loanwords, probably
acquired during the Majapahit period from the late thirteenth century until
the fifteenth century Blust (2013). It has 73% of lexical similarity with
Indonesian141414i.e., 73% of its words also occur in Indonesian. and it is
written in Arabic and Latin scripts.

Batak languages (btk) are a subgroup of the languages of Northwest Sumatra-
Barrier Islands spoken by the Batak people in the North Sumatra province and
surrounding areas. Batak languages can be divided into three groups: Northern,
Simalungan, and Southern. The Northern group consists of three languages:
Batak Alas-Kluet (btz), Batak Dairi (btd), and Batak Karo (btx). The
Simalungan group has one language only, i.e., Batak Simalungun (bts). The
Southern group consists of three languages: Batak Angkola (akb), Batak
Mandailing (btm), and Batak Toba (bbc) Eberhard et al. (2021). The Batak
languages were written using the Batak script, but the Latin script is now
used for most writing.

Batak Karo (btx) is a language spoken in Aceh province and North Sumatra
province. The language status is threatened. The lexical similarity is 81%
with Batak Dairi (btd), 80% with Batak Simalungun (bts), and 76% with Batak
Alas-Kluet (btz) Woollams (2005). It has 17 consonants and 7 vowels. The
stress is on the penultimate syllable. Similar to Indonesian, it has
inclusive/exclusive pronouns. The basic word order is SVO with prepositions.
It is a head initial language, except for the order of quantifiers. It has two
voices: actor-voice and undergoer-voice. It is written in Batak script and
also Latin script.

Buginese (bug) is a language spoken mainly in the South Sulawesi, Southeast
Sulawesi, Central Sulawesi, and West Sulawesi provinces. The word order is
SVO. Verb affixes are used to mark persons. It is non-tonal and has 19
consonant and 6 vowel phonemes. Stress is on the penultimate syllable. It was
written in the Buginese script in the past (derived from Brahmi script) but is
mainly written in Latin script now Eberhard et al. (2021). In Buginese, the
pronoun ‘I’ has three forms: the independent form ‘iyya’, the ergative form
‘-ka’, and the absolutive form/clitic ‘u-’. Buginese employs sentence
patterns, pronouns, and certain terms to express politeness Weda (2016).

Hakka (hak) is a language spoken in Southeastern China, mainly in Guangdong
province, also in Fujian, Guangxi, Hainan, Hunan, south Jiangxi, and Sichuan
provinces. It is also spoken by Chinese descendants in some parts in
Indonesia, such as in Singkawang in West Kalimantan province Stenberg (2015),
in Medan in North Sumatra province Nasution and Ayuningtyas (2020), and in
Lhokseumawe in Aceh province Saleh et al. (2018). It is a tonal language and
the basic word order is SVO. It is written in Han script and also Latin
script.

Indonesian (ind) is the national language of Indonesia in 1945 Constitution,
Article 36. Its lexical similarity to Standard Malay is over 80%. The word
order is SVO. It is non-tonal and has 19 consonants, 6 vowels, and 3
diphthongs. The stress is on the penultimate syllable. It has a rich
affixation system, including a variety of prefixes, suffixes, circumfixes, and
reduplication. Most of the affixes in Indonesian are derivational Pisceldo et
al. (2008). It is developed from literary ‘Classical Malay’ of the Riau-Johor
sultanate Sneddon (2003) and has regional variants. It is written in Latin
script.

Javanese (jav) is a language spoken mainly in Java island. It is the de facto
language of provincial identity in central and eastern Java. The word order is
SVO. It has 21 consonants and 8 vowels. It used to be written in Javanese
script but since 20th century is mostly written in Latin script. Javanese
differs from most other languages of western Indonesia in contrasting dental
and retroflex stops, and in the feature of breathy voice or murmur as a
phonetic property of its voiced obstruents. Javanese also differs from most
languages of the Philippines and western Indonesia in allowing a number of
word-initial consonant clusters. It has an elaborate system of speech levels
Blust (2013).

Madurese (mad) is a language spoken in the East Java province, mainly on
Madura Island, south and west of Surabaya city, Bawean, Kangean, and Sapudi
islands. It has vowel harmony, gemination, rich affixation, three types of
reduplication, and SVO basic word order Davies (2010).

Minangkabau (min) is a language spoken mainly in West Sumatra and other
provinces on Sumatra Island such as Bengkulu and Riau. Although it is
classified as Malay, it is not intelligible with Indonesian. The word order is
SVO written in Latin script. Standard Minangkabau voice can be characterised
as an Indonesian-type system whereas colloquial Minangkabau voice is more
effectively characterised as a Sundic-type system Crouch (2009).

Min Nan (nan) is a language spoken in Southeastern China. One of its dialects
is Chaozhou-Shantou (Chao-Shan dialect) or Teochew dialect. It is spoken by
Chinese descendants in some parts of Indonesia such as in Jambi Peng (2011)
and in Pontianak in West Kalimantan province Veniranda (2015). While Teochew
is historically Chinese, its contact with languages in Indonesia has resulted
in some changes uncharacteristic of Chinese languages. For example, regarding
word order, Teochew spoken in Jambi exhibits both head-final and head-initial
relative clauses even though head-initial relative clauses are generally
ungrammatical in Chinese languages. In addition to the head-initial word
order, Jambi Teochew has also borrowed the Malay relativizer yang Peng (2011).
It is a tonal language with tone sandhi. The word order is SVO Eberhard et al.
(2021).

Ngaju (nij) is a language spoken in the Central Kalimantan province. It is
widely used as a language of wider communication for trade in much of
Kalimantan, from the Barito to the Sampit river. It is used in many domains
(church, school, village-level government, market, etc.). It has various
affixes and reduplication, similar to Indonesian. The active voice is marked
by prefix ‘maN-’ and the passive voice is marked by prefix ‘iN-’. The word