the soft ordering network. The architecture of the network is interpreted as a grid of $K \times D$ slots, where $d$ indicates the depth of the network and the slots with the same $k$ value have the same module topology. For each available slot $T_{kd}$, the corresponding module $M_k$ is inserted. If $k > |M|$, then $M_{k \bmod |M|}$ is inserted instead.

Finally, each module in a particular slot has the potential to share its weights with modules that have the same architecture and are located in other slots of the blueprint. Flag $F_k$ in each module indicates whether or not the module’s weights are shared. This flag is evolved as part of the module genotype in CoDeepNEAT. Also, there is also global flag $F_d$ for each depth of the soft ordering network. If the $M_k$ is placed in $T_{kd}$ and both $F_k$ and $F_d$ are turned on, then the module is able to share its weights with any other $M_k$ whose slot have both flags turned on as well. Such an arrangement allows each slot to have sharing enabled and disabled independently.

The assembled network is attached to separate encoders and decoders for each task and trained jointly using 