the order in which the words are iterated may affect the result. The initialization may also affect whether the optimization will get stuck in a local optimum, and how fast it will converge. We use the exchange[^6] tool, which by default initializes the classes by sorting the words by frequency and assigning word $w_i$ to class $i \mod N_C$, where $i$ is the sorted index. We compare this to initialization from other clustering methods.

Clustering Based on Distributed Representation of Words
-------------------------------------------------------

Neural networks that process words need to represent them using real-valued vectors. The networks learn the word embeddings automatically. These *distributed representations* are interesting on their own, because the network tends to learn similar representation for semantically similar words [@Mikolov:2013:NAACL]. An interesting alternative to statistical clustering of words is to cluster words based on their vector representations using traditional clustering methods.

Distributed word representations can be created quickly using shallow networks, such as the Continuous Bag-of-Words (CBOW) model [@Mikolov:2013:ICLR]. We use word2vec[^7] to cluster words by creating word embeddings using the CBOW model and clustering them using k-means.

A Rule-Based Method for Clustering Finnish Words
------------------------------------------------

Much of the vocabulary in conversational Finnish text is due to colloquial 