attachment version for a total of 249600 sentences. An example of four sentences generated for English given the two nouns *building* and *system* is below.

. \[en\_full\] . Everybody ignored the system of the buildings that was . Everybody ignored the systems of the building that was . Everybody ignored the system of the buildings that were . Everybody ignored the systems of the building that were

Not all combinations are semantically coherent; however, @gulordavaetal18 suggest that syntactic operations (e.g., subject-verb agreement) are still possible for RNN LMs with “completely meaningless” sentences [@gulordavaetal18 p. 2].

RNN LM Details {#lm_details}
--------------

We analyzed long short-term memory networks [LSTMs; @hochreiterschmidhuber97] throughout the present paper. For English, we used the English Wikipedia training data provided by @gulordavaetal18.[^5] For Spanish, we constructed a comparable training corpus from Spanish Wikipedia following the process used by @gulordavaetal18. A recent dump of Spanish Wikipedia was downloaded, raw text was extracted using WikiExtractor,[^6] and tokenization was done using TreeTagger. A 100-million word subset of the data was extracted, shuffled by sentence, and split into training (80%) and validation (10%) sets.[^7] For LM training, we included the 50K most frequent words in the vocabulary, replacing the other tokens with 