What makes a language easy to deep-learn? Deep neural networks and
  humans similarly benefit from compositional structure
Deep neural networks drive the success of natural language processing. A
fundamental property of language is its compositional structure, allowing
humans to systematically produce forms for new meanings. For humans, languages
with more compositional and transparent structures are typically easier to
learn than those with opaque and irregular structures. However, this
learnability advantage has not yet been shown for deep neural networks,
limiting their use as models for human language learning. Here, we directly
test how neural networks compare to humans in learning and generalizing
different languages that vary in their degree of compositional structure. We
evaluate the memorization and generalization capabilities of a large language
model and recurrent neural networks, and show that both deep neural networks
exhibit a learnability advantage for more structured linguistic input: neural
networks exposed to more compositional languages show more systematic
generalization, greater agreement between different agents, and greater
similarity to human learners.
Lukas Galke, Yoav Ram, Limor Raviv
# What makes a language easy to deep-learn?

###### Abstract

Neural networks drive the success of natural language processing. A fundamental property of natural languages is their compositional structure, allowing us to describe new meanings systematically. However, neural networks notoriously struggle with systematic generalization and do not necessarily benefit from a compositional structure in emergent communication simulations. Here, we test how neural networks compare to humans in learning and generalizing a new language. We do this by closely replicating an artificial language learning study (conducted originally with human participants) and evaluating the memorization and generalization capabilities of deep neural networks with respect to the degree of structure in the input language. Our results show striking similarities between humans and deep neural networks: More structured linguistic input leads to more systematic generalization and better convergence between humans and neural network agents and between different neural agents. We then replicate this structure bias found in humans and our recurrent neural networks with a Transformer-based large language model (GPT-3), showing a similar benefit for structured linguistic input regarding generalization systematicity and memorization errors. These findings show that the underlying structure of languages is crucial for systematic generalization. Due to the correlation between community size and linguistic structure in natural languages, our findings underscore the challenge of automated processing of low-resource languages. Nevertheless, the similarity between humans and machines opens new avenues for language evolution research.

Natural language processing Machine learning Language acquisition Language evolution

## 1 Introduction

Fueled by deep learning [1, 2], natural language processing is becoming increasingly influential [3, 4, 5, 6, 7, 8]. In particular, large pretrained language models based on deep neural networks [5, 6, 9, 10], such as GPT-3 [7], have shown to be incredibly successful, even when generalizing to new tasks without being trained for them [7, 11]. Although these neural networks are not biologically plausible [12], an increasing body of work reports striking similarities to humans regarding their properties in natural language processing, including similarity in neural activity patterns [13] and similar biases in reasoning [14]. However, the role of linguistic structure in the input language is largely unknown, i. e., to what extent artificial neural networks generalize to new meanings in a systematic way and whether they make similar mistakes during memorization. Here, we investigate how the degree of systematic structure in a language affects the memorization and generalization capabilities of deep neural networks, small and large, and directly compare their performance to human learners.
For humans, languages with more regular, compositional, and transparent structures are typically easier to learn compared to languages with opaque and irregular structures [15, 16, 17, 18]. This is because more structured languages allow learners to derive a set of generative rules rather than memorizing individual forms [19, 20]. While all languages are fully learnable in the long run, cross-linguistic variation in systematic structure, i. e., the degree to which grammatical rules are informative, productive, and clearly marked [21, 22, 23, 24, 25, 26, 27], can have important consequences for learning: some languages appear to be easier to learn than others, and human learners exposed to languages that are more morphologically opaque show slower learning trajectories and worse overall proficiency [28, 29, 30, 31, 32, 33, 34].

This pattern from real languages was recently confirmed in a large-scale experimental study with human participants, which used an artificial language learning paradigm to test the acquisition of a broad yet tightly controlled range of comparable languages with different degrees of systematic structures [32]. Results showed that languages with more systematic structures were indeed learned faster, better, and more consistently, and also promoted better generalizations and more robust convergence on new, unfamiliar forms. These findings have far-reaching implications not only for research on language acquisition but also for broader theories on language evolution in our species. However, whether this structure advantage carries over to computational learning systems in natural language processing is poorly understood.

The central point we aim to establish here is: How similar are deep neural networks and humans in language learning? What makes a language easy to learn for deep learning models? In other words, do neural network architectures that have been successful in natural language processing tasks exhibit the same structure bias as humans do and benefit from more structured linguistic input when learning a new language? Answering these questions is crucial for two reasons. First, it reflects on a broader and long-standing debate on the similarity between humans and neural networks with respect to core cognitive functions such as learning, categorization, and abstraction across a wide range of research areas, e. g., language understanding [35, 36], language emergence [37], language acquisition [38, 39], reasoning [14], neural activity patterns [13], shape versus texture in computer vision [40, 41], catastrophic forgetting [42] in continual learning [43], i. e., forgetting all previous knowledge when trained for a new task, difficulties when generalizing out of the training distribution [44], systematic generalization in natural language processing [45, 46, 47], and the abilities of large language models that emerge at scale, such as tackling new tasks without further training [7, 11].

Second, answering these questions is crucial for simulating language evolution with neural networks [48, 49, 50, 51, 52, 53, 54, 55, 56]. In these simulations, deep neural networks start without any prior knowledge of the language, with no predefined vocabulary, and need to develop a language to solve a communication game. Although these simulations have great potential for advancing our understanding of how languages emerge, we can only expect insights gained with deep networks to inform studies of language evolution in our species if the resulting languages had the same properties as natural languages [57]. However, the properties of the emergent languages show crucial mismatches with human languages [58, 59, 60]. Specifically, it has been found that deep neural networks show no correlation between the degree of structure in the emergent language and the generalization capabilities of the networks [58, 60]. This finding stands in stark contrast to the compositional structure of human languages [61] and raises the question if systematic and compositional structure is helpful at all for deep neural networks--in the same way as it is for humans.

Notably, popular models of language evolution directly attribute the emergence of systematic grammatical structures in human languages to the learnability and expressivity pressures operating during cross-generational transmission and communication [62, 63]. In particular, more systematic and predictable grammars are favored in the cultural evolution process exactly because they are learned better by novices and are advantageous for generalizations, seeing as they allow learners to overcome the poverty of stimulus and produce infinite utterances after exposure to just a finite set [19, 20, 64, 65, 66, 67, 68, 69, 70, 62, 64, 67, 69, 80, 62, 68, 64, 69, 90, 67, 68, 69, 70].

Here, we investigate this precise relationship between linguistic structure and generalization performance in language learning with deep neural networks and compare that to human language learning. We are interested in five aspects: First, how well can they memorize form-meaning pairs of the input languages? Second, how systematically do they generalize to new meanings? Third, to what extent do different neural network agents generalize in the same way? Fourth, to what extent do neural networks behave in the same way as humans (i.e., how similar are the machine-generated labels and the corresponding human-generated labels)? Lastly, do these patterns also hold for large language models like GPT-3?

To answer these questions, we replicate a large-scale preregistered study on language learnability carried out with adult human participants [32]. That study tested the learnability of artificial languages created by participants in a previous group communication experiment [71]. The input languages vary in the degree of systematic structure they contain, i. e., ranging from completely unstructured languages with no grammatical rules, to languages with fully systematic and compositional mappings between meanings and labels. For example, one of the weakly structured languages uses moof for shape 1 moving upwards and wait for the same shape 1 moving towards the top left of the screen. In contrast, one of the highly structured languages uses tup-oo for shape 1 moving upwards (see Figure 1(_Left_)) and tup-oi for the same shape moving to the top left. In this highly structured case, labels contain a sub-part for describing the shape
and a sub-part for describing the direction. In contrast, the two words in the unstructured language do not share a single character despite describing a scene with the same shape. In the learnability study, 100 human participants were randomly assigned to ten different input language conditions and then engaged in repeated learning blocks consisting of passive exposure, guessing trials, and production trials. After training, participants were tested on their knowledge of the input language in a memorization test (measuring their reproduction accuracy on learned items) and a generalization test (measuring their ability to produce labels for new, unseen items).

We directly replicate this experimental setup with neural networks. Specifically, we use the same ten input languages and train 100 differently-initialized neural network models for each language, i. e., 10 per human participant. We carefully follow the humans experiment training protocols to mimic the learning objectives, procedures, and measures as closely as possible by training neural networks on the same stimuli presented to humans and in the same order, using the same learning tasks, and providing the same feedback during learning blocks. To simulate the previous study's exposure, guessing, and production blocks as closely as possible, we have designed a custom neural network architecture that combines generative and contrastive components (Figure 1). The architecture is inspired by image-captioning approaches [72], the emergent communication literature [51], and a recent review paper [57] which suggested having shared model parameters between generation and processing of a label. Our model consists of two components: a generative component that facilitates the production of a descriptive sequence of symbols (here, a label) for a scene and a contrastive component that shapes the latent space and enables the models to tackle guessing tasks (i.e., given a label, pick the correct scene from a set of distractors). Each component has its sequential module, the well-known long short-term memory [73], to carry out the generation and processing of a sequence, respectively. The two components share the symbol embedding that maps each symbol to a continuous vector and an encoder module that transforms an input scene into a latent representation. Production tasks are modeled by a generative objective: the model generates a label, character by character. This generated label is compared to the target input language by character-level cross-entropy. Guessing tasks are modeled by a contrastive objective [74], which aligns the representations of input scenes and corresponding labels and facilitates selecting the correct scene from a set of distractors. We then evaluate the neural networks' learning and generalization performance using the same metrics used for humans and compare the labels produced by networks to the ones produced by humans in terms of edit distance.

To preview our results, we found that neural networks display the same structure bias as humans: Neural networks trained on more structured languages generalized more systematically to new scenes, and their produced labels were more similar to those produced by humans--for both meanings in the training set and new meanings. Moreover, different neural network models are more likely to converge to the same generalization when there is more structure in the input language. In addition, we show that GPT-3, a large language model based on the Transformer architecture [4], also displays the same structure bias as humans and as our custom architecture based on recurrent neural networks.

## 2 Results

The language learning capabilities of neural networks were evaluated with a memorization test and a generalization test following each training round. The memorization test included the production of labels for the 23 scenes that were part of the training data. In contrast, during generalization, networks needed to produce labels for 13 new scenes

Figure 1: Overview of exemplary input data _(Left)_, the experimental procedure _(Center)_, and model architecture _(Right)_.
Figure 2: **Neural Networks’ memorization and generalization performance**. More structured languages lead to better and faster reproduction of the input language (a), to better generalization on unknown scenes (c), better agreement with human participants during memorization (b) and generalization (d), and higher convergence between networks (e). **(A)** Production similarity between labels generated by neural agents and labels of the input language. **(B)** Production similarity between labels generated by neural agents and labels generated by human participants. **(C)** Generalization score of labels generated by neural agents for new scenes that were not part of the training data. **(D)** Production similarity between labels generated by neural agents and labels generated by human participants for unseen scenes. **(E)** Convergence score measures the similarity between labels generated for unseen scenes by different neural agents. Stars mark the round at which neural agents first exceed the final performance of human participants. Input languages are grouped into 5 bins. Each line is the average of 200 neural agents with a different random initialization. A star marks the epoch at which the NN agents exceed human performance. Results are cut off for visualization at epoch 60.
that were not a part of the training data. For each of the 100 agents trained on each of the ten input languages, we calculated the following measures after each of the 100 rounds: the similarity between networks' productions and the input language; the similarity between networks' productions and the human learners' productions during memorization and generalization; a generalization score capturing the degree of systematic generalization [32]; and a convergence score capturing the agreement between different agents.

The results are shown in Figure 2. In the following, we present the results for the learning trajectory organized along the two types of tests: memorization and generalization. Subsequently, we compare the generalization systematicity of the large language model GPT-3 with the final generalization systematicity of our neural network agents and that of humans. Lastly, we conduct an error analysis of the imperfect reproductions in the final memorization test to investigate to what extent the mistakes are affected by the structure.

### Memorization

How well did neural agents memorize the input languages? And how similar were their generated labels to those produced by human learners? This is measured by production similarity [32], which captures the similarity between the original label and the produced label by calculating the average normalized edit distance between two labels for the same scene. We use this measure in two ways: once to compare the generated labels to the true label of the input language and once to compare the machine-generated label to the human-generated label for the same scene.

Similarity to Input Languages during MemorizationWith sufficient training rounds, all languages can be learned by all neural agents, reaching a production similarity of at least 0.8 (out of 1) by round 60 (Figure 2A). Structured languages are learned significantly better (LME 1; \(\beta=0.045\), \(\mathrm{SD}=0.001\), \(z=62.865\), \(p<0.001\)), i. e. they show a higher similarity with the input language. However, this advantage tends to diminish over training rounds (LME 1; \(\beta=-0.005\), \(\mathrm{SD}<0.001\), \(z=-54.978\), \(p<0.001\)).

Similarity to Humans during MemorizationWe measured the similarity between the human memorization data (i. e. their productions at test time after completing the training rounds) and the memorization test data of the neural network agents after each training round (Figure 2B). Structured languages led to a significantly greater similarity with human learners (LME 2; \(\beta=0.097\), \(\mathrm{SD}=0.001\), \(z=81.429\), \(p<0.001\)). This effect became even stronger over rounds (LME 2; \(\beta=0.022\), \(\mathrm{SD}=0.000\), \(z=208.708\), \(p<0.001\)).

### Generalization

We evaluate the productions of neural agents when they generalize, i. e., produce labels for new scenes that were not part of the training data. We test the productions regarding three aspects: the degree of systematicity, the similarity to humans, and the generalization convergence between different agents.

Generalization SystematicitySystematicity is the degree to which similar scenes are described by similar meanings, as measured by the generalization score [32]. In the generalization test, there is no true label in the input language. The generalization score thus correlates the pairwise label difference and the pairwise semantic difference between the labels generated for new scenes and the labels generated by the same agent for known scenes, reflecting the degree to which new labels conform to the labels of the input language. More structured languages consistently led to significantly higher generalization scores (Figure 2C) (LME 3; \(\beta=0.088\), \(\mathrm{SD}=0.001\), \(z=148.901\), \(p<0.001\)), and this effect became stronger with time (\(\beta=0.046\), \(\mathrm{SD}<0.001\), \(z=703.483\), \(p<0.001\)).

Similarity to Humans during GeneralizationWe measure the similarity between the productions of neural agents and humans for new scenes (Figure 2D), i. e. during generalization. More structure in the input language led to a significantly higher similarity between humans and neural agents (LME 5; \(\beta=0.132\), \(\mathrm{SD}=0.002\), \(z=70.280\), \(p<0.001\)), which became stronger over rounds (\(\beta=0.046\), \(\mathrm{SD}<0.001\), \(z=344.287\), \(p<0.001\)). Table 1 shows examples of generated labels.

Convergence between Networks during GeneralizationMore structured languages lead to better agreement between networks (LME 4; \(\beta=0.043\), \(\mathrm{SD}=0.001\), \(z=49.027\), \(p<0.001\)), such that, for more structured languages, different neural agents learning the same input language produced more similar labels for new scenes (Figure 2E). This effect became stronger over rounds (\(\beta=0.009\), \(\mathrm{SD}<0.001\), \(z=121.740\), \(p<0.001\)).
### Comparison with a Large Language Model (GPT-3)

We tested how well a large language model, the GPT-3 model text-davinci-003, performs on our input language. We compared its produced labels to the final productions of humans and our neural network agents. Strikingly, the results reveal that higher structure leads to more systematic generalization in all cases (Figure 3). Table 1 shows examples of the final productions of humans, our recurrent neural networks, and large language models in the generalization test.

Memorization Error AnalysisWe seek to understand better whether the memorization errors of humans and neural networks are similarly affected by the structure. Therefore, we analyze the cases where the learning systems fail to

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Struct.** & **Shape** & **Angle** & **Human** & **RNN** & **GPT-3** \\ \hline low & 2 & 360 & kokoke & seefe & tik-tik \\  & 4 & 45 & woti & kite & hihi \\  & 3 & 150 & priu & mimi & hihi \\ \hline mid-low & 3 & 225 & wangsuus & wangsoe & wangsuus \\  & 4 & 225 & gntsoe & gntuu & gntsi \\  & 1 & 135 & sketsi & gesh & geshts \\ \hline mid & 3 & 60 & powi & powu-u-u & powee \\  & 4 & 330 & fuottoa & fuotio & fuottu-u-u \\  & 1 & 30 & fewo-o-o-o & fewen & fewee \\ \hline mid-high & 1 & 30 & fas-a & fas-a & fas-a \\  & 3 & 360 & muif-i & muif-a & muif-i \\  & 1 & 225 & fas-huif & fas-huif & fas-huif \\ \hline high & 4 & 60 & smut-tkk & smut-tk & smut-ttk \\  & 2 & 360 & nif-k & nif-kks & nif-k \\  & 1 & 315 & wef-ks & wef-kks & wef-kks \\ \hline \hline \end{tabular}
\end{table}
Table 1: Generalization examples from neural and human learners, showing labels generated for unseen scenes. The column GPT-3 corresponds to completions generated by the GPT-3 model text-davinci-003 via in-context learning, where the training data is provided in context. The examples cover the differently structured input languages from low to high.

Figure 3: Comparison of the final generalization score achieved by 10 humans _(Left)_, 100 recurrent neural networks _(Center)_, and GPT-3 _(Right)_ for each of the input languages. The x-axis shows the structure score of the input languages. Error regions show 95% confidence intervals estimated via bootstrapping.
memorize the correct label perfectly. We compare the final memorization test results of humans, our custom neural networks, and GPT-3. The results show the same pattern for all three learning systems (Figure 4): When there is more structure in the input language, the non-perfectly memorized productions are still more similar to the correct labels.

## 3 Discussion

Main FindingsOverall, our results show that neural networks benefit from more structured linguistic input in the same way humans do, and that neural networks' performance becomes increasingly more human-like when trained on more structured languages. This structure bias can be found in networks' learning trajectories and even more so in the networks' generalization behavior, mimicking previous findings with humans. Although all languages can eventually be learned, we show that more structured languages are learned better and with greater similarity to human productions. Networks and humans produce nearly identical labels when trained on high-structured languages but not when trained on low-structured languages. Moreover, networks that learn more structured languages are significantly better at systematic generalization to new, unseen items, and crucially, their generalizations are significantly more consistent and more human-like. This means that highly systematic grammars allow for better generalization and facilitate greater alignment between different neural agents as well as between neural agents and humans. We replicate these results with small recurrent neural networks and with transformer-based large language models, showing that, together with humans, all three learning systems show the same bias in systematic generalization and memorization errors. Thus, our findings add to the increasing evidence of similarity in language learning between humans and machines [13, 75, 76, 77, 78], which strengthen the idea that language models are useful for studying human cognitive mechanisms. For example, our findings support using language models' estimated probabilities as surprisal scores in psycholinguistics studies [79, 80, 81].

ImplicationsFirst and foremost, our findings support a usage-based view of language learning: languages' underlying grammatical structure can be learned directly from the available linguistic input [82, 83], without the need to assume innate knowledge or a universal grammar [84].

Our findings have further implications for natural language processing, where systematic generalization is of high interest [45, 46, 47]. Specifically, we show that seeding models with well-structured inputs can improve the generalization capabilities of the learning system. Even though our study is based on artificial languages, our findings directly pertain to the natural language processing of real-world languages. To confirm this prediction, we reanalyzed data from Wu et al. [85], who used the Wug Test [86] to test language models' ability to predict different forms of unfamiliar words in a wide range of natural languages. Indeed, we find that the Wug Test accuracy negatively correlates with the degree of irregularity of the language (Spearman's \(\rho=-0.96\), \(p<10^{-15}\); Kendall's \(\tau=-0.86,p<10^{-14}\)).

Figure 4: Memorization error analysis for human participants _(Left)_, recurrent neural networks _(Center)_, and GPT-3 _(Right)_. The error rates are 33.30% for humans, 13.87% for NNs, and 7.39% for GPT-3. The x-axis shows the structure score of the input language. Error regions show 95% confidence intervals estimated via bootstrapping.
This strong negative correlation suggests that natural languages with fewer irregularities, i. e. more structured natural languages, are indeed easier to learn.

Moreover, our results have direct implications for the natural language processing of low-resource languages, i. e. languages for which there is only very little training data available. Low-resource languages are typically spoken by smaller communities, and cross-linguistic and experimental work shows that such languages typically have less structured languages with more irregularities [87, 88, 89, 90]. Since our study predicts that more structured natural languages would be easier to learn than less structured ones, this would result in a double whammry for natural language processing of small communities' languages: Not only is fewer data available for languages spoken by smaller communities but also these languages tend to be harder to learn for machine learning models due to their tendency of having a lower degree of structure.

Finally, our results are of high relevance to the field of emergent communication, which simulates the evolution of language with multi-agent reinforcement learning [48, 49, 50, 51, 52, 53, 54, 55, 56]. As argued in the introduction, certain linguistic phenomena of natural language appear to be hard to replicate with multi-agent reinforcement learning [57, 58, 59, 60], which raises the question whether compositionality is helpful for neural networks at all. We hypothesize that these mismatches are caused by the lack of cognitive constraints [57] eradicating the learnability pressure that underlies human language evolution [62]. Previous work has already shown that, by introducing an artificial learnability pressure, the advantage of structured languages can be restored [50, 52, 54]. This leaves us with the _hypothesis_ that structured languages are easier to learn for deep networks. Our results confirm this hypothesis for recurrent neural networks and GPT-3 as a transformer-based large language model. We find that more structured languages lead to more systematic generalization, more agreement between different networks, and more similarity to humans. Therefore, our findings support the importance of a learnability pressure for compositional languages to emerge. By replicating a result previously found in humans [32] with neural network agents, we take the first steps to bring emergent communication closer to the field of language evolution, supporting the computational modeling of language evolution with neural networks.

Limitations and Future WorkOne potential limitation of our work is that we had to make certain decisions about the neural network architecture design, optimization procedure, and hyperparameters. All these decisions may impact the results. However, we have varied all relevant hyperparameter settings and found that the results are robust to specific settings (see Supporting Information). Another potential caveat is that we used a simplified input representation for the neural networks: while humans saw videos of shapes moving in a direction, our neural agents received an attribute vector that disentangles key features. Since pixel input would be more challenging (as neural networks would first need to learn disentangled representations themselves [91]), here we focused on the ability of systematic generalization in language learning rather than the ability to learn disentangled representations. The latter is a long-standing endeavor in deep learning research [92, 93, 94], and future work can incorporate an appropriate vision module as the encoder for input scenes. Lastly, we have intentionally limited this study to purely supervised learning in order to shed new light on the relationship between language structure and the generalization performance of neural network models. In future work, we intend to extend the present results by analyzing how such neural network models engage in a collaborative communication game in which they are made to invent a new communication protocol.

## 4 Conclusion

Our findings shed light on the relationship between language and language-learning systems, showing that linguistic structure is crucial for language learnability and systematic generalization. More structured languages are easier to learn, regardless of the learning system: a human participant, a small recurrent neural network model, or a large language model. Thus, generalization capabilities are heavily influenced by language structure, with both biological and artificial learning systems benefiting from more structured input by facilitating more systematic and transparent generalizations.

## 5 Materials and Methods

### Materials

Input LanguagesThe input languages come from a previous communication study in which groups of interacting participants took turns producing and guessing labels for different dynamic scenes, creating new artificial languages over time [90]. The final languages created by these groups then served as input languages for a follow-up study on language learnability with humans [32]. For our experiments, we used the same ten input languages. These input languages are considered the ground truth.
Each of the ten input languages contains a set of 23 label-scene mappings. Each scene is composed of one of four different shapes moving in different directions between 0 and 360 degrees. Crucially, the ten input languages have different degrees of structure, ranging from languages with no structure to languages with consistent, systematic grammar. Each language has a structure score represented by topographic similarity [95], quantifying the degree to which similar labels describe similar meanings. The topographic similarity is measured as the correlation between all labels' pairwise length-normalized edit distances and their corresponding pairwise semantic differences. The semantic difference between two scenes is calculated as the sum of the difference in shape and the difference in angles [32]. The difference in shape is zero if the two scenes contain the same shape, and one otherwise. The difference in angles is calculated as the absolute difference divided by 180. The topographic similarity of a language is then calculated as the pairwise correlation between all semantic differences and all normalized edit distances.

Human Learning DataAside from the input languages, we have data from 100 participants who learned these input languages. These participants are different from the participants who created the languages. A hundred participants, ten per input language, engaged in repeated learning blocks consisting of passive exposure (in which the target label-meaning mappings were presented on the screen one by one), guessing trials (in which participants needed to pick the right scene from a set of possible distractors), and production trials (in which participants needed to generate a descriptive label for a target scene based on what they had learned). During training, humans received feedback on their performance.

Input RepresentationThe scenes were shown to human participants as short videos [32]. For the neural agents, we use a simplified input representation for the scenes: a one-hot encoding of the shape concatenated with a sine and a cosine transformation of the angle. The sine-cosine transformation promotes a similar treatment of angles that are close to each other, while each unique angle can be distinguished. For example, shape 2 (between 1 and 4) moving at a 90-degree angle is converted to a vector \((0,1,0,0,1,0)\), shape 3 with 45 degrees is converted to \((0,0,1,0,0.71,0.71)\), and shape 4 with 135 degrees is converted to \((0,0,0,1,0.71,-0.71)\). We refer to the resulting 6-dimensional vector representation of the input as a scene \(\mathbf{x}\).

### Methods

Neural Network Architecture

Generative ComponentThe input scene \(\mathbf{x}\) is encoded to a latent representation \(\mathbf{h}\) by Encoder, a feedforward network (i.e., multilayer perceptron), such that \(\mathbf{h}=\textsc{Encoder}(\mathbf{x})\). This latent representation \(\mathbf{h}\) is then used as the initial state of a recurrent generative network Writer. The Writer sequentially produces a label, a sequence of characters, as the output. This Writer consists of three modules: an input embedding for previously produced characters, a long short-term memory cell [73], and an output layer that produces the next character.

Contrastive ComponentFor the exposure and guessing tasks, we have another recurrent module Reader that reads a label \(\mathbf{m}\) sequentially (i. e., character by character) while updating its state. A feedforward layer then transforms the final state into a latent representation \(\mathbf{z}\), such that \(\mathbf{z}=\textsc{Reader}(\mathbf{m})\). These reading components are used for contrastive learning, i. e. they are trained so that the hidden representation of the label \(\mathbf{z}\) matches the representation of the corresponding scene \(\mathbf{h}\), which is used as the initial hidden state of the generative Writer module.

Shared ParametersTo ensure that the contrastive training procedure affects the generative component, we couple the two components: First, the embedding (i. e., the mapping between the agent's alphabet and the first latent representation) parameters are shared between the input layer of Reader, the input layer of Writer, and the output layer of Writer. Second, the same encoder module is used in both the generative and the contrastive components (see Figure 1).

HyperparametersThe output dimension of Encoder, the hidden state sizes of Reader and Writer, and the embedding size are all set to 50.

Training Procedure

We train the model for multiple training rounds: as in human experiments [32], each training round consists of three blocks: exposure, guessing, and production block, described in detail in the following. As typical in neural network training, we train the network with backpropagation and stochastic gradient descent, where the gradient is estimated based on a small number of examples (minibatches) [96, 2]. The batch size is set to 5, except for the guessing block, in which we use a batch size of 1.
Exposure BlockIn the exposure block, human participants were exposed to scenes with the corresponding target labels. Therefore, we train the deep learning models using a loss function with two terms: a generative and a contrastive loss term. The generative loss, \(\mathcal{L}_{\mathrm{gen}}\), is a tokenwise cross-entropy with the ground-truth label of the original language. The contrastive loss, \(\mathcal{L}_{\mathrm{con}}\), promotes similar latent representations of scenes and labels that correspond to each other and contrasts representations that do not. Specifically, we use the NTXent loss [74]. We use other scenes in the same batch as distractors for the contrastive loss. The final loss function is \(\mathcal{L}=\mathcal{L}_{\mathrm{gen}}+\alpha_{\mathrm{con}}\mathcal{L}_{ \mathrm{con}}\). The factor \(\alpha\) determines the relative weight of the loss terms. For the main experiment, we use \(\alpha=0.1\). We used a batch size of 5 to reflect human short-term memory constraints [97].

Guessing BlockIn the guessing block, we use the same loss function as in the exposure block. The contrastive loss term \(\mathcal{L}_{\mathrm{con}}\) mirrors the task in which human participants had to select the correct scene against the distractors given a label. The generative loss term \(\mathcal{L}_{\mathrm{gen}}\) is used so that the model does not "forget" how to generate [98]. Again, we use the same factor \(\alpha_{\mathrm{con}}=0.1\) to weight the loss terms.

In more detail, the latent representation \(\mathbf{z}=\textsc{Encoder}(\mathbf{x})\) of the scene \(\mathbf{x}\) should be closest to the latent representation \(\mathbf{z}^{\prime}=\textsc{Reader}(\mathbf{m})\) of the corresponding label \(\mathbf{m}\). The difference from exposure training is that in the guessing block, we use the identical distractors used in experiments with humans, whereas, in the exposure block, we use the other scenes from the same batch.

Production BlockIn the production block, a scene was presented to human participants, who had to produce a label. We again use the same generative loss as in the previous block, \(\mathcal{L}_{\mathrm{gen}}\), to model the production block. In the production block, however, we omit the contrastive loss term and train only on generation. Thus, the loss function is \(\mathcal{L}=\mathcal{L}_{\mathrm{gen}}\).

Initialization and OptimizationThe parameters are randomly initialized by _He initialization_[99], the default in PyTorch [100]. We employ the widely used _Adam optimizer_[101] to optimize the loss function with the default learning rate of \(10^{-3}\).

### Measures and Evaluation

Following experiments with human participants [32], we tested memorization and generalization. In the memorization test, we have neural agents produce labels for scenes that were part of the training data. In the generalization test, we have neural agents produce labels for scenes they have never seen, i. e., not in the training data. We performed these tests after each of the 100 rounds. To account for the random effects of the parameter initialization and training procedure, we ran 100 experiments with different random seeds for each input language. The measures are described in the following.

Production SimilarityProduction similarity measures the overlap between two sets of labels. It is computed as one minus the normalized edit distance between pairs of labels. For our analysis, we use production similarity once to quantify the similarity between the generated labels and the ground truth of the input languages, and once to quantify the similarity of labels generated by neural network agents with labels produced by human learners.

Generalization ScoreThe generalization score measures the degree of systematicity during the generalization test [32]. We take two sets of scenes: a training set, on which the agents were trained, and a test set, on which the agents were not trained. We then do the following for each agent. First, we take two sets of labels: one previously generated for each training scene by the agent and another that we let the agent generate for each test scene. Second, the difference between train and test scenes is measured by pairwise semantic difference. Semantic difference is calculated as described above under Input Languages. Third, the difference between train and test labels is measured by pairwise normalized edit distance. Finally, we compute the Pearson correlation between these two differences across all scenes. Then, we take the average correlation coefficient across all agents as the generalization score.

Convergence ScoreThe convergence score measures the similarity in the generalization test between agents that learned the same language. We take a test set of scenes on which the agents have not been trained and let each agent produce a label for each scene. We compute the pairwise normalized edit distance between all generated labels per scene so that if we have \(n\) test scenes and \(k\) agents, we compute \(n\cdot\frac{k(k-1)}{2}\) distances. We then compute the average distance across both scenes and labels and take one minus the average distance as the convergence score. Therefore, if all agents produce the same label for each test scene, we would get a convergence score of 1.
### Large Language Models

We use the following procedure to generate the results of large language models. We employ GPT-3 [7] (version text-davinci-003), or Generative Pre-trained Transformer, an autoregressive language model trained to predict the next token in a sequence of tokens using web-scale text data. GPT-3 is capable of few- and zero-shot learning. This means that only a few examples are sufficient for GPT-3 to tackle a new task. This ability, sometimes referred to as in-context learning, allows us to evaluate GPT-3's performance on our object description task without the need for fine-tuning.

Thus, we supplied the training data of the respective input language to GPT-3: 23 shape, angle, and word triples. These 23 lines were followed by a single line, with only shape and angle, and GPT-3 was asked to predict the corresponding word. In the memorization task, the target word appears at an earlier location in the training data, which means that the perfect solution is to choose this word. In the generalization task, we gave GPT-3 a combination of shape and angle that is not present in the training data. The model generated the most likely descriptive word for the new shape and angle pair.

We had to make certain technical choices when using GPT-3. First, we chose not to supply an instructive description of the task because GPT-3 is sensitive to varying input. Second, we set the sampling temperature to zero, which controls the randomness of the generation, such that we obtain deterministic generations. Third, we do not impose any restrictions on the characters that can be generated but rely on its ability to detect this from the training data. Fourth, we do not feed GPT-3 its previous predictions for memorization or generalization. Lastly, GPT-3's tokenization procedure (how text is split into subword tokens) could have been problematic for applying it to our artificial languages. However, we found that GPT-3 still reaches high memorization performance, which suggests that tokenization is not a problem.

### Statistical Analyses

For each of the ten input language, we trained 100 differently-initialized neural network models over 100 rounds. The testing in each round consisted of 23 memorization and 13 generalization examples. This makes a total of 2.3M memorization and 1.3M generalization test results subject to statistical analyses. Significance was tested using linear mixed-effects models, as implemented in the Python package statsmodels [102], for production similarity (LME 1), generalization score (LME 3), generalization convergence (LME 4), as well as production similarity to humans in memorization (LME 2) and generalization (LME 5). We use the structure score and the logarithmized round number in all measures as a fixed effect. The number of rounds was logarithmized in accordance with the scaling laws of neural language models [103]. Both the structure score and the logarithmized round number were centered and scaled As random effects, all statistical models use the random seed that was used for random initialization, which also determines the input language, and the specific scene. to unit standard deviation. For LME 5, scaling the log-transformed round number to unit variance hindered convergence, so the log rounds were only centered.

AcknowledgementsWe thank Yosef Prat, Tal Simon, Mitja Nikolaus, Marieke Woensdregt, Kadine Saraiva de Carvalho, and Sergio Pereira Soares for their comments and discussions. We thank Shijie Wu for sharing their data.
