Recurrent neural network with long short-term memory (RNN-LSTM) {#LSTM}
---------------------------------------------------------------

### Architecture

The RNN-LSTM [@hochreiter1997long] is a deep learning algorithm most suited for prediction of sequential data such as time series, and has received a lot of attention in recent years [@goodfellow2016deep]. Variants of RNN-LSTMs are the best performing models for time series modeling in areas such as stock pricing [@chen2015lstm], supply chain [@carbonneau2008application], natural language processing [@cho2014learning], and speech recognition [@graves2013speech]. A major improvement over regular RNNs, which have issues with the vanishing and the exploding gradients [@pascanu2013difficulty], LSTMs have become the state-of-the-art approach for training RNNs in the deep learning community. Unlike regular RNNs, the RNN-LSTMs have gates that control the information flow into the neural network from previous time steps of the time series. The RNN-LSTM used here, like the RC-ESN but unlike the ANN, is stateful (i.e., actively maintains state). Our RNN-LSTM has $50$ hidden layers in each RNN-LSTM cell. More details on our RNN-LSTM are presented in \[sec:lstm\]. RNN-LSTMs have many tunable parameters and are trained with the expensive BPTT algorithm [@goodfellow2016deep].

### Training and Prediction {#training-and-prediction-1}

The input to the RNN-LSTM is a time-delay-embedded matrix of $X(t)$ with embedding dimension $q$ (also known as 