for each task is possible because an evaluation of a multitask network yields a performance metric *for each task*. The $(1+1)$-ES is chosen because it is efficient and sufficiently powerful in experiments, though it can potentially be replaced by any population-based method. To make it clear that a single set of modules is trained during evolution, and to disambiguate from the terminology of CoDeepNEAT, for CTR the term *meta-iteration* is used in place of *generation*.

### Algorithm Description

Each individual constitutes a module routing scheme for a particular task. At any point in evolution, the $i$th individual for the $t$th task is represented by a tuple $(\mathcal{E}_{ti}, G_{ti}, \mathcal{D}_{ti})$, where $\mathcal{E}_{ti}$ is an encoder, $G_{ti}$ is a DAG, which specifies the module routing scheme, and $\mathcal{D}_{ti}$ is a decoder. The complete model for an individual is then given by $$\bm{y}_t = \big(\mathcal{D}_{ti} \circ \mathcal{R}\big(G_{ti}, \big\{\mathcal{M}_k\big\}_{k=1}^{K}\big) \circ \mathcal{E}_{ti}\big)(\bm{x}_t) \, ,$$ where $\mathcal{R}$ indicates the application of the shared modules $\mathcal{M}_k$ based on the DAG $G_{ti}$. Note that $\circ$ denotes function composition and $\mathcal{E}_{ti}$, and $\mathcal{D}_{ti}$ can be any neural network functions that are compatible with the set of shared modules. In the experiments in this paper, each $\mathcal{E}_{ti}$ is an identity 