and have become very hard for human designers to optimize [@szegedy2015going; @szegedy:cvpr16; @zoph:arxiv16; @jaderberg2017population]

This paper develops an automated, flexible approach for evolving architectures, i.e. hyperparameters, modules, and module routing topologies, of deep multitask networks. A recent deep MTL architecture called soft ordering [@Meyerson:2018] is used as a starting point, in which a different soft sequence of modules is learned for each task. This paper extends this architecture in several ways. First, this paper proposes a novel algorithm for evolving *task specific routings* that create an unique routing between modules for each task. Second, *more general modules* with the same soft ordering architecture are evolved. Third, the general modules are evolved together with a *blueprint*, a shared routing for all tasks, that improves upon the soft ordering architecture. Fourth, as a capstone architecture, the task specific routing are evolved together with the general modules. These four approaches are evaluated in the Omniglot task [@Lake:2015] of learning to recognize characters from many different alphabets. A series of results confirms the intuition well: As a baseline, soft ordering performs significantly better in each task than single-task training (67% vs. 61% accuracy). Evolution of modules and topologies improves significantly upon soft ordering. Coevolution 