and challengers are evaluated on a validation set that is disjoint from the training data. The fitness for each individual is its performance for its task on the validation set. In this paper, accuracy is the performance metric. If the challenger has higher fitness than the champion, then the champion is replaced, i.e.,$(\mathcal{E}_{t1}, G_{t1}, \mathcal{D}_{t1}) \coloneqq (\mathcal{E}_{t2}, G_{t2}, \mathcal{D}_{t2})$. After selection, if the average accuracy across all champions is the best achieved so far, the entire system is checkpointed, including the states of the modules. After evolution, the champions and modules from the last checkpoint constitute the final trained model, and are evaluated on a held out test set.

### An Ecological Perspective

More than most evolutionary methods, this algorithm reflects an artificial ecology. The shared modules can be viewed as a shared finite set of environmental resources that is constantly exploited and altered by the actions of different tasks, which can correspond to different species in an environment. Within each task, individuals compete and cooperate to develop mutualistic relationships with the other tasks via their interaction with this shared environment. A visualization of CTR under this perspective is shown in Figure \[fig:ctr\_diagram\].

![\[fig:ctr\_diagram\] This figure shows an instance of 