all the nodes at the first level. Starting from the second level, the input feature map to a node is a weighted sum of the output feature maps of all the nodes at its previous level: $$f_{i, j}^{\text{in}} = \sum_{k=1}^{n} w_{i, j, k}^{\text{level}} \cdot f_{i-1, k}^{\text{out}},$$ where $2 \leq  i \leq m$, $1 \leq j \leq n$, $f_{i, j}^{\text{in}}$ is the input to the $j^{th}$ node at $i^{th}$ level, $f_{i-1, k}^{\text{out}}$ is the output of the $k^{th}$ node at $(i-1)^{th}$ level, and $w_{i, j}^{\text{level}}$ are the connection weights between the $j^{th}$ node at $i^{th}$ level and all the nodes at $(i-1)^{th}$ level. In practice, $w_{i, j}^{\text{level}}$ is a probability distribution obtained by softmax.

For each node in the supergraph, we also learn a probability distribution over the possible choices of activation functions. The output of a node is a weighted sum of the attended feature map under different activation functions: $$f^{\text{out}}_{i,j} = \sum_{k=1}^{|\mathcal{A}|} w_{i, j, k}^{\text{activation}} \cdot f_{i,j}^{\text{out}, \phi_k},$$ where $\mathcal{A}$ is the set of available activation functions, $\phi_k$ is the $k^{th}$ activation function in $\mathcal{A}$, $w_{i, j, k}^{\text{activation}}$ is the weighting factor to be learned for $\phi_k$, and $f_{i,j}^{\text{out}, \phi_k}$ is the attended feature map under the activation function 