made in [@schoenholz_deep_2017] for computing the mean field theory of gradients for feedforward tanh networks. This is similar to the practice of feedback alignment [@lillicrap_random_2016]. Even though we are the first to explicitly formulate \[ass:gradInd\](b), in fact it was already applied implicitly in the gradient calculations of [@schoenholz_deep_2017]. Note that a priori \[ass:gradInd\](b) is not true, as $\pd E/\pd \p x l _i$ depends on $\dot \phi(\p h {l+1} _k)$ for every $k$, which depend on $\p h {l} _j$ for each $j$, and which depends on $\p x {l-1} _k$ for every $k$. Nevertheless, in practice both subassumptions hold very well.

Now we define the central quantities studied in this paper. Inevitably, our paper involves a large amount of notation that may be confusing for the first-time reader. We have included a glossary of symbols (\[tab:glossary\]) to ameliorate notation confusion.

\[defn:length\] Fix an input $\p x 0$. Define the [**length quantities**]{} $\p {\mathbf{q}}l := \la (\p h l_1)^2 \ra$ and $\p {\mathbf{p}}l := \la (\p x l_1)^2 \ra$ for $l > 0$ and $\p {\mathbf{p}}0 = \|\p x 0\|^2/N$. Here the expectations $\la \bullet \ra$ are taken over all random initialization of weights and biases for all layers $l$, 