<h3>Hey Ryan!</h3>

<p>I know it's late but I just came across your question hope it's not too late or that you still find some knowledge here.</p>

<p><strong>First</strong> of all, Stackoverflow may not be the best place for this kind of question. First reason to that is you have a conceptual question that is not this site's purpose. Moreover your code runs so it's not even a matter of general programming. <strong>Have a look at <a href="https://stats.stackexchange.com/">stats</a></strong>.</p>

<p><strong>Second</strong> from what I see there is no conceptual error. You're using everything necessary that is:</p>

<ul>
<li>lstm with propper dimensions</li>
<li><code>return_sequences=false</code> just before your <code>Dense</code> layer</li>
<li>linear activation for your output</li>
<li><code>mse</code> cost/loss/objective function</li>
</ul>

<p><strong>Third</strong> I however find it extremely unlikely that your network learns anything with so few pieces of data. You have to understand that you have less data than parameters here! For the great majority of supervised learning algorithm, the first thing you need is not a good model, it's good data. You can not learn from so few examples, especially not with a complex model such as LSTM networks. </p>

<p><strong>Fourth</strong> It seems like your target data is made of relatively high values. First step of pre-processing here could be to standardize the data : center it around zero - that is translate your data by its mean - and rescale by ists standard deviation. This really helps learning! </p>

<p><strong>Fifth</strong> In general here are a few things you should look into to improve learning and reduce overfitting : </p>

<ul>
<li><a href="https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf" rel="nofollow noreferrer">Dropout</a></li>
<li><a href="http://arxiv.org/pdf/1502.03167v3.pdf" rel="nofollow noreferrer">Batch Normalization</a></li>
<li>Other optimizer (such as <a href="https://arxiv.org/pdf/1412.6980" rel="nofollow noreferrer">Adam</a>)</li>
<li><a href="https://arxiv.org/pdf/1211.5063v2.pdf" rel="nofollow noreferrer">Gradient clipping</a></li>
<li><a href="http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf" rel="nofollow noreferrer">Random hyper parameter search</a></li>
<li>(This is not exhaustive, if you're reading this and think something should be added, comment it so it's useful for future readers!)</li>
</ul>

<p><strong>Last but NOT least</strong> I suggest you look at this <a href="http://vict0rsch.github.io/tutorials" rel="nofollow noreferrer">tutorial on Github</a>, especially the <a href="http://vict0rsch.github.io/tutorials/keras/recurrent" rel="nofollow noreferrer">recurrent tutorial for time series with keras</a>.</p>

<p>PS: Daniel Hnyk updated his <a href="http://danielhnyk.cz/predicting-sequences-vectors-keras-using-rnn-lstm/" rel="nofollow noreferrer">post</a> ;)</p>
Keras LSTM Time Series
<p>I have a problem and at this point I'm completely lost as to how to solve it. I'm using Keras with an LSTM layer to project a time series. I'm trying to use the previous 10 data points to predict the 11th.</p>

<p>Here's the code:</p>

<pre><code>from keras.models import Sequential
from keras.layers.core import Dense, Activation, Dropout
from keras.layers.recurrent import LSTM

def _load_data(data):
"""
data should be pd.DataFrame()
"""
n_prev = 10
docX, docY = [], []
for i in range(len(data)-n_prev):
    docX.append(data.iloc[i:i+n_prev].as_matrix())
    docY.append(data.iloc[i+n_prev].as_matrix())
if not docX:
    pass
else:
    alsX = np.array(docX)
    alsY = np.array(docY)
    return alsX, alsY

X, y = _load_data(df_test)

X_train = X[:25]
X_test = X[25:]

y_train = y[:25]
y_test = y[25:]

in_out_neurons = 2
hidden_neurons = 300
model = Sequential()
model.add(LSTM(in_out_neurons, hidden_neurons, return_sequences=False))
model.add(Dense(hidden_neurons, in_out_neurons))
model.add(Activation("linear"))
model.compile(loss="mean_squared_error", optimizer="rmsprop")
model.fit(X_train, y_train, nb_epoch=10, validation_split=0.05)

predicted = model.predict(X_test)
</code></pre>

<p>So I'm taking the input data (a two column dataframe), creating X which is an n by 10 by 2 array, and y which is an n by 2 array which is one step ahead of the last row in each array of X (labeling the data with the point directly ahead of it. </p>

<p>predicted is returning  </p>

<pre><code>[[ 7.56940445,  5.61719704],
[ 7.57328357,  5.62709032],
[ 7.56728049,  5.61216415],
[ 7.55060187,  5.60573629],
[ 7.56717342,  5.61548522],
[ 7.55866942,  5.59696181],
[ 7.57325984,  5.63150951]]
</code></pre>

<p>but I should be getting</p>

<pre><code>[[ 73,  48],
[ 74,  42],
[ 91,  51],
[102,  64],
[109,  63],
[ 93,  65],
[ 92,  58]]
</code></pre>

<p>The original data set only has 42 rows, so I'm wondering if there just isn't enough there to work with? Or am I missing a key step in the modeling process maybe? I've seen some examples using Embedding layers etc, is that something I should be looking at?</p>

<p>Thanks in advance for any help!</p>