<p>Please see <a href="https://arxiv.org/abs/1704.04110" rel="nofollow noreferrer">DeepAR</a> -  a LSTM forecaster more than one step into the future.</p>
<blockquote>
<p>The main contributions of the paper are twofold: (1) we propose an RNN
architecture for probabilistic forecasting, incorporating a negative
Binomial likelihood for count data as well as special treatment for
the case when the magnitudes of the time series vary widely; (2) we
demonstrate empirically on several real-world data sets that this
model produces accurate probabilistic forecasts across a range of
input characteristics, thus showing that modern deep learning-based
approaches can effective address the probabilistic forecasting
problem, which is in contrast to common belief in the field and the
mixed results</p>
</blockquote>
<p>In this paper, they forecast multiple steps into the future, to negate exactly what you state here which is the error propagation.<br />
Skipping several steps allows to get more accurate predictions, further into the future.</p>
<p>One more thing done in this paper is predicting percentiles, and interpolating, rather than predicting the value directly. This adds stability, and an error assessment.</p>
<hr />
<p>Disclaimer - I read an older version of this paper.</p>
LSTM Evolution Forecast
<p>I have a confusion about the way the LSTM networks work when forecasting with an horizon that is not finite but I'm rather searching for a prediction in whatever time in future. In physical terms I would call it the evolution of the system.</p>
<p>Suppose I have a time series $y(t)$ (output) I want to forecast, and some external inputs $u_1(t), u_2(t),\cdots u_N(t)$ on which the series $y(t)$ depends.</p>
<p>It's common to use the lagged value of the output $y(t)$ as input for the network, such that I schematically have something like (let's consider for simplicity just lag 1 for the output and no lag for the external input):</p>
<p>[y(t-1), u_1(t), u_2(t),\cdots u_N(t)] \to y(t)</p>
<p>In this way of thinking the network, when one wants to do recursive forecast it is forced to use the predicted value at the previous step as input for the next step. In this way we have an effect of propagation of error that makes the long term forecast badly behaving.</p>
<p>Now, my confusion is, I'm thinking as a RNN as a kind of an (simple version) implementation of a state space model where I have the inputs, my output and one or more state variable responsible for the memory of the system. These variables are hidden and not observed.</p>
<p>So now the question, if there is this kind of variable taking already into account previous states of the system why would I need to use the lagged output value as input of my network/model ?</p>
<p>Getting rid of this does my long term forecast would be better, since I'm not expecting anymore the propagation of the error of the forecasted output. (I guess there will be anyway an error in the internal state propagating)</p>
<p>Thanks !</p>