title: Keras model to predict number sequence
body: <p>I am trying to train Keras LSTM model to predict next number in a sequence.</p>
  
  <ol>
  <li>What is wrong with my model below, how do I debug when a model is not learning </li>
  <li>How do I decide which layer types to use</li>
  <li>On what basis should I select loss and optimizer params while compiling</li>
  </ol>
  
  <p>My input training data is of shape (16000, 10) like below</p>
  
  <pre><code>[
      [14955 14956 14957 14958 14959 14960 14961 14962 14963 14964]
      [14731 14732 14733 14734 14735 14736 14737 14738 14739 14740]
      [35821 35822 35823 35824 35825 35826 35827 35828 35829 35830]
      [12379 12380 12381 12382 12383 12384 12385 12386 12387 12388]
      ...
  ]
  </code></pre>
  
  <p>Corresponding output training data is of shape (16000, 1) like below</p>
  
  <pre><code>[[14965] [14741] [35831] [12389] ...]
  </code></pre>
  
  <p>As LSTM is complaining, I reshaped training/test data</p>
  
  <pre><code>X_train = X_train.reshape(X_train.shape[0], X_train.shape[1], 1)
  X_test = X_test.reshape(X_test.shape[0], X_test.shape[1], 1)
  </code></pre>
  
  <p>Here is final training/test data shape</p>
  
  <pre><code>Total Samples: 20000
  X_train: (16000, 10, 1)
  y_train: (16000, 1)
  X_test: (4000, 10, 1)
  y_test: (4000, 1)
  </code></pre>
  
  <p>Here is my model</p>
  
  <pre><code># Model configuration
  epochs = 2
  batch_size = 32
  hidden_neurons = 100
  output_size = 1
  
  # Create the model
  model = Sequential()
  model.add(LSTM(hidden_neurons, input_shape=(X_train.shape[1], X_train.shape[2])))
  model.add(Dense(output_size))
  model.compile(loss='mean_squared_error', optimizer='rmsprop', metrics=['accuracy'])
  print(model.summary())
  model.fit(X_train, y_train, epochs=epochs, batch_size=batch_size)
  
  scores = model.evaluate(X_test, y_test, batch_size=batch_size, verbose=0)
  print("Model Accuracy: %.2f%%" % (scores[1]*100))
  </code></pre>
  
  <p>Here is my output</p>
  
  <pre><code>_________________________________________________________________
  Layer (type)                 Output Shape              Param #   
  =================================================================
  lstm_3 (LSTM)                (None, 100)               40800     
  _________________________________________________________________
  dense_3 (Dense)              (None, 1)                 101       
  =================================================================
  Total params: 40,901
  Trainable params: 40,901
  Non-trainable params: 0
  _________________________________________________________________
  None
  Epoch 1/2
  16000/16000 [==============================] - 11s - loss: 533418575.3600 - acc: 0.0000e+00    
  Epoch 2/2
  16000/16000 [==============================] - 10s - loss: 532474289.7280 - acc: 6.2500e-05    
  Model Accuracy: 0.00%
  </code></pre>