models constructed from a vocabulary that includes all the words in the training corpus. However, when recurrent neural network (RNN) language models are used, their ability to utilize long contexts gives a larger gain to subword-based modeling. Our best results are from RNN language models that are based on statistical morphs. We show that the suitable size for a subword vocabulary depends on the language. Using time delay neural network (TDNN) acoustic models, we were able to achieve new state of the art in Finnish and Estonian conversational speech recognition, 27.1 % word error rate in the Finnish task and 21.9 % in the Estonian task.
author:
- 'Seppo Enarvi, Peter Smit, Sami Virpioja, and Mikko Kurimo,  [^1][^2][^3]'
bibliography:
- 'IEEEabrv.bib'
- 'references.bib'
title: Automatic Speech Recognition with Very Large Conversational Finnish and Estonian Vocabularies
---

[Enarvi : ASR with Very Large Conversational Finnish and Estonian Vocabularies]{}

language modeling, word classes, subword units, artificial neural networks, automatic speech recognition

Introduction
============

and Estonian are agglutinative languages, meaning that words are formed by concatenating smaller linguistic units, and a great deal of grammatical information is conveyed by inflection. Modeling these inflected words correctly is important for automatic speech recognition, to produce 