RNNs and RNN-LSTMs (see section \[LSTM\]).

![A schematic of RC-ESN. $D \times D$ is the size of the adjacency matrix of the reservoir $\mathbf{A}$. $X(t)$ ($-T \leq t \leq 0 $) is the input during training. $\mathbf{W_{in}}$ and $\mathbf{A}$ are random matrices that are chosen at the beginning of training and fixed during training and testing. Only $\mathbf{W_{out}}$ is computed during training. During testing ($t>0$), $X(t+\Delta t)$ is predicted from a given $X(t)$ that is either known from the initial condition or has been previously predicted.[]{data-label="esn_demo"}](esn.png){width="\textwidth"}

The equations governing the RC-ESN training process are as follows: $$\begin{aligned}
\label{esn_eqn}
r(t+\Delta t)=\tanh\left(\mathbf{A}r(t)+\mathbf{W_{in}}X(t)\right), \label{eq:r} \\
\mathbf{W_{out}}=\arg\min_{\mathbf{W_{out}}} ||{\mathbf{W_{out}}\tilde{r}(t)-X(t)}||+\alpha ||\mathbf{W_{out}}||. \label{eq:wout} \end{aligned}$$ Here $|| \cdot ||$ is the $L_2$-norm of a vector and $\alpha$ is the $L_2$ regularization (ridge regression) constant. Equation (\[eq:r\]) maps the observable $X(t) \in \Re^{8}$ to the higher dimensional reservoir’s state $r(t+\Delta t) \in \Re^{D}$ (reminder: $D$ is $O(1000)$ for this problem). Note that Eq. (\[eq:wout\]) contains $\tilde{r}(t)$ rather than $r(t)$. As observed by Pathak *et al*.  [@pathak2018model] for the [Kuramoto-Sivashinsky]{} equation, and observed by us in this work, the columns of the matrix $\tilde{\mathbf{r}}$ should be chosen as nonlinear combinations of the columns of the reservoir state matrix ${\mathbf{r}}$ (each 