activation function, and output dropout rate. Preliminary experiments suggested that the relatively complex routings in CMTR (when compared to CM and CMSR) require more complex final layers as well, thus evolving the complexity of the final layer is optimal. Like in CTR, the weights between modules are always shared in CMTR. If modules with completely new weights are added to the task routings, they have to be trained from scratch and may even hurt performance, whereas adding a module with already partially trained weights does not. In addition, as the routings evolved by CTR are much larger than those discovered by CM and CMSR, disabling or evolving weight sharing significantly bloats the total number of weight parameters and slows training significantly.

Experiments {#sec:experiments}
===========

This section details experiments comparing the five methods in the Omniglot MTL domain.

Omniglot Character Recognition
------------------------------

The Omniglot dataset consists of 50 alphabets of handwritten characters [@Lake:2015], each of which induces its own character recognition task. There are 20 instances of each character, each a $105 \times 105$ black and white image. Omniglot is a good fit for MTL, because there is clear intuition that knowledge of several alphabets will make learning another one easier. 