$\mathcal A \sqrt l$ ($\mathcal A$ given in \[thm:dalethExpSqrtTanhFullRes\]), and the colored lines are $\log(\p \bullet 1/\p \bullet l)$ for $\bullet = {{\boldsymbol{\oldchi}}}, {\boldsymbol{\oldchi}}_b, {\boldsymbol{\oldchi}}_w$. That they all converge together starting around $l=1000$ indicates that the approximation in \[thm:dalethExpSqrtTanhFullRes\] is very good for large $l$.[]{data-label="fig:edelta_plot"}](graphics/edelta_plot.pdf "fig:"){height=".16\textheight" width=".3\textwidth"} ![Left-to-right: [**(a)**]{} Plots of ${\mathbf{e}}^*$ and $\delta^*$ against $\sigma_a/\sigma_v$. [**(b)**]{} In log-log scale: the dashed line is $l^{-\delta^* - 1}$, and the colored lines are $\p {\mathbf{e}}l - \p {\mathbf{e}}{l-1}$ for different initial conditions $\p {\mathbf{e}}0$. That they become parallel at about $l = 400$ on verifies that $\p {\mathbf{e}}l = \Theta(l^{-\delta^*})$. [^6] [**(c)**]{} In log-log scale: The dashed line is $\mathcal A \sqrt l$ ($\mathcal A$ given in \[thm:dalethExpSqrtTanhFullRes\]), and the colored lines are $\log(\p \bullet 1/\p \bullet l)$ for $\bullet = {{\boldsymbol{\oldchi}}}, {\boldsymbol{\oldchi}}_b, {\boldsymbol{\oldchi}}_w$. That they all converge together starting around $l=1000$ indicates that the approximation in \[thm:dalethExpSqrtTanhFullRes\] is very good for large $l$.[]{data-label="fig:edelta_plot"}](graphics/verify_deltastar_n_grad-chi.pdf "fig:"){height=".16\textheight"}

#### Backward dynamics.

Finally, we show that the gradient is approximated by $$\begin{aligned}
\p {{\boldsymbol{\oldchi}}}{m} &= \exp(\mathcal A(\sqrt{l} - \sqrt{m}) + O(\log l - \log m))\p {{\boldsymbol{\oldchi}}}{l} \label{eqn:tanhGradEst}\tag{$\star$}\end{aligned}$$ where $\mathcal A = \f 4 3 \sqrt{\f 2 \pi} \sigma_w$ in the RRN case and $
\mathcal 