not log scale). (a) and (b) are made with 20 runs of resnets of width 1000. (c) is made with 25 runs of resnets of width 250.[]{data-label="fig:theory_tracks_pratice"}](graphics/tanh_full_res_theory_vs_exp-chi.pdf){height=".17\textheight"}

#### Forward dynamics.

When $\phi = \tanh$, $\p {\mathbf{p}}l$ and $\p {\mathbf{q}}l$ increase as $\Theta(l)$ in either RRN or FRN (\[thm:p\_q\_linear\]), as one might expect by observing that ${\mathrm{V}}\tanh( {\mathbf{q}}) \to 1$ as ${\mathbf{q}}\to \infty$ so that, for example in the RRN case, the recurrence ${\mathbf{p}}= {\mathrm{V}}\tanh( {\mathbf{q}}) + {\underline}{\mathbf{p}}$ becomes ${\mathbf{p}}= 1 + {\underline}{\mathbf{p}}$. This is confirmed graphically by the black lines of the leftmost chart of \[fig:theory\_tracks\_pratice\]. We carefully verify that this intuition is correct in its proof in the appendix, and find that in fact $\p {\mathbf{p}}l \sim l$ in the RRN case and $\p {\mathbf{p}}l \sim (\sigma_v^2 + \sigma_a^2)l$ in the FRN case.

What about $\p {\boldsymbol{\oldgamma}}l$? The middle chart of \[fig:theory\_tracks\_pratice\] shows that over time, $\p {\mathbf{e}}l = \p {\boldsymbol{\oldgamma}}l / \p {\mathbf{p}}l$ contracts toward the center of the interval $[0, 1]$, but from the looks of it, it is not clear whether there is a stable fixed point ${\mathbf{e}}^*$ of ${\mathbf{e}}$ or not. We prove that, in fact, [**all trajectories of ${\mathbf{e}}$ not starting at 1 do 