run the experiments on a high-end GPU. Our experiments show that class and subword models are more attractive than word models for several reasons. They are efficient computationally and in terms of memory consumption, and they can achieve better performance than word models. Usually subword vocabularies include all the individual letters, meaning that any word that uses the same letters can be constructed. Class models are restricted to a certain vocabulary, but the efficiency is not limited by the vocabulary size, so very large vocabularies can be used.

To summarize, this is the first time Finnish and Estonian subword models have outperformed word models in conversational speech recognition, even without limiting the word vocabulary size. We compare word clustering techniques and show that class-based models outperform full-vocabulary word models in these tasks. We also present the first comparison of word, class, and subword NNLMs trained using different softmax approximations, applied to speech recognition. Finally, we test a novel method for weighting NNLM training corpora.

Class-Based Language Models
===========================

Finnish and Estonian are highly agglutinative languages, so the number of different word forms that appear in training corpora is huge. The pronunciation variation in colloquial Finnish is also written down, making 