title: Predictions using a Keras Recurrent Neural Network - accuracy is always 1.0
body: <p><strong>TLDR:</strong> How do I use a Keras RNN to predict the next value in a sequence?</p>
  
  <hr>
  
  <p>I have a list of sequential values. I want to feed them into a RNN to <em>predict</em> the next value in the sequence.</p>
  
  <pre><code>[ 0.43589744  0.44230769  0.49358974 ...,  0.71153846  0.70833333 0.69230769]
  </code></pre>
  
  <p>I'm using Keras to do this and can get a network with a decreasing loss but the accuracy is consistently 1.0. This is wrong. <code>y_tests != model.predict(x_tests)</code>.</p>
  
  <pre><code>Epoch 0
  1517/1517 [==============================] - 0s - loss: 0.0726 - acc: 1.0000 - val_loss: 0.0636 - val_acc: 1.0000
  Epoch 1
  1517/1517 [==============================] - 0s - loss: 0.0720 - acc: 1.0000 - val_loss: 0.0629 - val_acc: 1.0000
  ...
  </code></pre>
  
  <p>Here's my network.</p>
  
  <pre><code>model = Sequential()
  model.add(SimpleRNN(1, 100))
  model.add(Dense(100, 1, activation = "sigmoid"))
  model.compile(loss="mean_squared_error", optimizer = "sgd")
  </code></pre>
  
  <p>I have tried a SimpleRNN, GRU and LSTM but have had no luck. Here is how the data is formatted.</p>
  
  <pre><code># Current value
  y_train = [[ 0.60576923] [ 0.64102564] [ 0.66025641] ..., [ 0.71153846] [ 0.70833333] [ 0.69230769]]
  
  # Previous 10 values
  x_train_10 = [
      [[ 0.65064103] [ 0.66346154] [ 0.66346154] ..., [ 0.72115385] [ 0.72435897] [ 0.71153846]] ...,
      [[ 0.66346154] [ 0.66346154] [ 0.67628205] ..., [ 0.72435897] [ 0.71153846] [ 0.70833333]]
  ]
  
  # Previous value
  x_train_1 = [[ 0.58333333] [ 0.60576923] [ 0.64102564] ...,  [ 0.72435897] [ 0.71153846] [ 0.70833333]]
  
  # So here are the shapes...
  y_train.shape    = (1895, 1)
  x_train_10.shape = (1895, 10, 1)
  x_train_1.shape  = (1895, 1)
  </code></pre>
  
  <p>Each element in <code>x_train_10</code> is a list of the previous 10 values. I formatted it like this to follow Keras's documentation that recurrent layers take inputs of shape <code>(nb_samples, timesteps, input_dim)</code>.</p>
  
  <p>I have also tried using an <code>Embedding</code> layer with no luck. (This may be the wrong way to use it - I've only seen it used in classification not prediction).</p>
  
  <pre><code>model = Sequential()
  model.add(Embedding(1, 30))
  model.add(LSTM(30, 100))
  ...
  </code></pre>
  
  <p><code>pad_sequences</code> also did not work.</p>
  
  <pre><code>x_train_1 = sequence.pad_sequences(x_train_1, maxlen = None, dtype = "float32")
  </code></pre>
  
  <hr>
  
  <p>I want to get the RNN working with this simple data/architecture so I can use it for more complex problems later.</p>
  
  <p>Thanks :)</p>