In contrast to the older writing system of the 19th century, modern Hawaiian orthography employs characters for long vowels and glottal stops. These extra characters account for about one-third of the phonemes in Hawaiian, so including them makes a big difference to reading comprehension and pronunciation. However, transliterating between older and newer texts is a laborious task when performed manually. We introduce two related methods to help solve this transliteration problem automatically, given that there were not enough data to train an end-to-end deep learning model. One approach is implemented, end-to-end, using finite state transducers (FSTs). The other is a hybrid deep learning approach which approximately composes an FST with a recurrent neural network (RNN). We find that the hybrid approach outperforms the end-to-end FST by partitioning the original problem into one part that can be modelled by hand, using an FST, and into another part, which is easily solved by an RNN trained on the available data. From 1834 to 1948, more than 125,000 newspaper pages were published in the Hawaiian language (Nogelmeier, 2010 ). Yet by 1981, many expected this once flourishing language to die BID0 . Hawaiian has since defied expectations and experienced the beginnings of a remarkable recovery (Warner, 2001; Wilson and Kamanā, 2001) . However much of the literary inheritance that is contained in the newspapers has become difficult for modern Hawaiians to read, since the newspapers were written in an orthography that failed to represent about one-third of the language's phonemes. This orthography, which we will refer to as the missionary orthography, excluded Hawaiian phonemes that did not have equivalents in American English (see Schütz, 1994) , namely long vowels /i: e: a: o: u:/ and glottal stop /P/. By contrast, the modern Hawaiian orthography, an innovation of Pukui and Elbert's Hawaiian dictionary (Pukui and Elbert, 1957) , presents a nearly perfect, one-to-one mapping between graphemes and phonemes. The process of manual transliteration from missionary to modern Hawaiian orthography is extremely labor intensive. Yet the cultural benefits are so great that hundreds of pages of newspaper-serials have already been transliterated by hand, such as Nogelmeier's new edition of the epic tale of Hi'iakaikapoliopele, the volcano goddess's sister BID6 . Critically important as such efforts are to the continued revitalization of this endangered language, they are still only a small sample of the material that could be made available to a modern Hawaiian audience.In this paper, we propose to automate, or semi-automate, the transliteration of old Hawaiian texts into the modern orthography. Following a brief review of related work (Section 2), we begin by describing a dataset of modern Hawaiian (Section 3). In Section 4, we present two methods for recovering missing graphemes (and hence phonemes) from the missionary orthography. The first composes a series of weighted FSTs; the second approximately composes an FST with a recurrent neural network language model (RNNLM) using a beam search procedure. Both approaches require only modern Hawaiian texts for training, which are much more plentiful than parallel corpora. Section 5 reports the results of our transliteration experiments using a simulated parallel corpus, as well as two 19th century newspaper articles for which we also have modern Hawaiian transcriptions. Being based on FSTs, both approaches are modular and extensible. We observe useful and promising results for both of our methods, with the best results obtained by the hybrid FST-RNNLM. These results showcase the strength of combining established hand-engineering methods with deep learning in a smaller data regime, with practical applications for an endangered language. With this paper we introduced a new transliteration problem to the field, that of mapping between old and new Hawaiian orthographies-where the modern Hawaiian orthography represents linguistic information that is missing from older missionary-era texts. One difficulty of this problem is that there is a limited amount of Hawaiian data, making data-hungry solutions like end-to-end deep learning unlikely to work. To solve the transliteration problem, we therefore proposed two models: the first was implemented end-to-end using weighted FSTs; the second was a hybrid deep learning approach that combined an FST and an RNNLM. Both models gave promising results, but the hybrid approach performed best. It allowed us to use a more powerful recurrent neural network-based language model, despite our dataset's small size. Factoring a problem like ours into one part that can be modelled exactly using expert domain knowledge and into another part that can be learned directly from data using deep learning is not novel; however it is a promising research direction for data-efficient modelling. To our knowledge, this paper is the first to describe a procedure to compose an FST with an RNN by approximately performing beam search over the FST.While the role of the RNNLM part of the hybrid approach may be obvious, the FST component plays an important role too. For example, the hand-designed FST can be replaced without needing to re-train the RNNLM. We tried to showcase this modularity by constructing two FSTs which we referred to as C and C wb , where only the latter allowed the insertion of spaces. Future work could extend the FST to model orthographic changes suggested by an error analysis of the current model's predictions.Input: Weheia ka Malapua Alii a Kanuia na Uluwehi no ia Wao. Prediction: Wehe 'ia ka Māla pua Ali'i a Kanu 'ia na Uluwehi nō ia Wao. Ground-truth: Wehe 'ia ka Māla Pua Ali'i a Kanu 'ia nā Uluwehi no ia Wao. (Pukui et al., 2006) . Correct predictions are green and bold. Characters omitted by the model as compared to the ground-truth are denoted by blue italics; erroneous insertions or substitutions are denoted by red underline. To make white spaces explicit, we represent them with the symbol ' '. More sample predictions can be found in Appendix A.An example of the current model's predictions (i.e. missionary input, predicted modern text, modern ground-truth) is given in TAB4 . In this example, we see the model correctly predicting some word boundaries, glottal stops and long vowels; however, we note that the model could not predict uppercase Pua (correct), because the input text contained lowercase pua (incorrect), and no (p : P) transitions were included in C or C wb . Similar observations (see Appendix A) motivate new mappings for consonant substitutions like (r : l) and (s : k) that occur in loanword adaptations (e.g. rose ⇒ loke). The error analysis also motivates mappings to delete spaces ( : ) and to handle contractions, like na'lii ⇒ nā ali'i. We could further incorporate linguistic knowledge of Hawaiian into the FST, which tells us, for example, about expected sequences of vowels (Parker Jones, 2010) . Additional improvements to the hybrid model might be obtained by increasing the amount of modern Hawaiian text used to train the RNNLM. One way to do this would be to accelerate the rate at which missionary-era Hawaiian texts are modernized. To this end, we hope that the present models will be used within the Hawaiian community to semi-automate, and thereby accelerate, the modernization of old Hawaiian texts.Tomas Mikolov, Martin Karafiát, Lukás Burget, JanČernocký, and Sanjeev Khudanpur. 2010.
A novel, hybrid deep learning approach provides the best solution to a limited-data problem (which is important to the conservation of the Hawaiian language)