inserted into the blueprints to create a temporary population of assembled networks. Each individual in this population is then evaluated by training it on a supervised learning task, and assigning its performance as fitness. The fitnesses of the individuals (networks) are attributed back to blueprints and modules as the average fitness of all the assembled networks containing that blueprint or module. One of the advantages of CoDeepNEAT is that it is capable of discovering modular, repetitive structures seen in state of the art networks such as Googlenet and Resnet [@szegedy2015going; @szegedy:cvpr16; @he:arxiv16].

![The relationships of the five methods tested in this paper. The soft ordering method [@Meyerson:2018] is used as the starting point, extending it with CoDeepNEAT on the left and task-specific routing on the right. The CMTR on bottom right combines these two main ideas and performs the best.[]{data-label="fg:overview"}](figures/overview.pdf "fig:"){width="\linewidth"}\

**Given** fixed blueprint\
**Initialize** module population\
**Each** generation:\
**Assemble** MTL networks with modules\
**Randomly initialize all weights**\
**Train** each MTL network with backprop\
**Assign fitnesses** to modules\
**Update** module populations

**Initialize** blueprint/module populations\
**Each** generation:\
**Assemble** MTL networks with\
blueprints/modules\
**Randomly initialize all weights**\
**Train** each MTL network with backprop\
**Assign fitnesses** to modules and blueprints\
**Update** blueprint/module populations

