\p {\mathbf{p}}l {}'}$ and $\p {\mathbf{c}}l := \p {\boldsymbol{\oldlambda}}l / \sqrt{\p {\mathbf{q}}l \p {\mathbf{q}}l{}'}$, and we will also call $\p {\mathbf{e}}l$ [**angular expressivity**]{}.

In this paper, for the ease of presentation, we assume $\p {\mathbf{p}}0 = \p {\mathbf{p}}0 {}'$. Then, as we will see, $\p {\mathbf{p}}l = \p {\mathbf{p}}l{}', \p {\mathbf{q}}l = \p {\mathbf{q}}l{}'$ for all $l$, and as a result, $\p {\mathbf{e}}l = \p {\boldsymbol{\oldgamma}}l / \p {\mathbf{p}}l$ and $\p {\mathbf{s}}l = \p {\mathbf{p}}l - \p {\boldsymbol{\oldgamma}}l = (1 - \p {\mathbf{e}}l) \p {\mathbf{p}}l$.

\[defn:grad\] Fix an input $\p x 0$ and a gradient vector $(\pd E/ \pd{\p x L_i})_i$ of some loss function $E$ with respect to the last layer $\p x L$. Then define [**the gradient quantities**]{} $\p {{\boldsymbol{\oldchi}}}l:= \la (\pd E/\pd \p x l _1)^2 \ra, \p {\boldsymbol{\oldchi}}l _\bullet := \la (\pd E/\pd \p \bullet l _1)^2 \ra$ for $\bullet = a, b$, and $\p {\boldsymbol{\oldchi}}l _\bullet := \la (\pd E/\pd \p \bullet l _{11})^2 \ra$ for $\bullet = w, v$. Here the expectations are taken with \[ass:gradInd\] in mind, over both random initialization of forward and backward weights and biases, as $N \to \infty$ (large width limit). Again, the index $1$ or $11$ does not 