{#section_ann}
--------------------------------------------

### Architecture

We have developed a deep ANN that has the same architecture as the one used in Duben and Bauer [@dueben2018challenges] (which they employed to conduct prediction on the same multi-scale Lorenz 96 studied here). The ANN has $4$ hidden layers with $100$ neurons each, and $8$ neurons in the input and output layers (Figure \[ann\_demo\]). It must be highlighted that unlike RC-ESN (and RNN-LSTM), this ANN is stateless, i.e., there is no hidden variable such as $r(t)$ that tracks temporal evolution. Furthermore, unlike the RC-ESN, for which the input and output are $X(t)$ and $X(t+\Delta t)$, following Duben and Bauer [@dueben2018challenges], the ANN’s inputs/outputs are chosen to be pairs of $X(t)$ and $\Delta X(t)=X(t+\Delta t)-X(t)$ (Figure \[ann\_demo\]). During prediction, using $X(t)$ that is known from initial condition or previously calculated, $\Delta X(t)$ is predicted. $X(t+\Delta t)$ is then computed via Adams-Bashforth integration scheme $$\begin{aligned}
X(t+\Delta t)=X(t) + \frac{1}{2} \big[ 3\Delta X(t)-\Delta X(t-\Delta t)  \big]. \label{eq:AB}\end{aligned}$$ [More details on the ANN architecture has been reported in Appendix \[sec:ANN\_append\].]{}

![A schematic of ANN, which has $4$ hidden layers each with $100$ neurons, and an input and an output layer each with $8$ neurons (for better illustration only few 