RL and NLP, presenting a broad spectrum of challenges for learning algorithms [@cote2018textworld][^1]. In addition to language understanding, successful play generally requires long-term memory, planning, exploration [@yuan2018counting], affordance extraction [@fulda2017can], and common sense. Text games also highlight major open challenges for RL: the action space (text) is combinatorial and compositional, while game states are partially observable since text is often ambiguous or under-specific. Also, TBG often introduce stochastic dynamics, which is currently missing in standard benchmarks [@machado2017revisiting]. For example, in Zork, there is a random probability of a troll killing the player. A thief can appear (also randomly) in each room.

**Representations for TBG:** To learn control policies from high-dimensional complex data such as text, good word representations are necessary. Kim ([-@kim2014convolutional]) designed a shallow word-level CNN and demonstrated state-of-the-art results on a variety of classification tasks by using word embeddings. For classification tasks with millions of labeled data, random embeddings were shown to outperform state-of-the-art techniques [@zahavy2016picture]. On smaller data sets, using *word2vec* [@mikolov2013distributed] yields good performance [@kim2014convolutional].

Previous work on TBG used pre-trained embeddings directly for control [@kostka2017text; @fulda2017can]. Other works combined pre-trained embeddings with neural networks. For example, He et al. ([-@DBLP:journals/corr/HeCHGLDO15]) proposed to use Bag Of 