title: Stock price predictions of keras multilayer LSTM model converge to a constant value
body: <p><a href="https://i.stack.imgur.com/cWxhc.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/cWxhc.png" alt="enter image description here" /></a>I've made a multilayer LSTM model that uses regression to predict next frame's values of the data. The model finishes after 20 epochs. I then get some predictions and compare them to my ground truth values. As you can see them in the picture above, predictions converge to a constant value. I don't know why this happens.
  Here is my model so far:</p>
  <pre><code>from keras.models import Sequential
  from keras.layers.core import Dense, Activation, Dropout
  from keras.layers import LSTM, BatchNormalization
  from tensorflow.python.keras.initializers import RandomUniform
  
  init = RandomUniform(minval=-0.05, maxval= 0.05)
  
  model = Sequential()
  
  model.add(LSTM(kernel_initializer=init, activation='relu', return_sequences=True, units=800, dropout=0.5, recurrent_dropout=0.2, input_shape=(x_train.shape[1], x_train.shape[2]) ))
  model.add(LSTM(kernel_initializer=init, activation='relu', return_sequences=False, units=500, dropout=0.5, recurrent_dropout=0.2 ))
  
  model.add(Dense(1024, activation='linear', kernel_initializer=init))
  model.add(BatchNormalization())
  model.add(Dropout(0.5))
  model.add(Dense(1, activation='linear', kernel_initializer= 'normal'))
  
  model.compile(loss='mean_squared_error', optimizer='rmsprop' )
  model.summary()
  </code></pre>
  <p><strong>EDIT1:</strong>
  I decreased epochs from 20 to 3. results are as follows:
  <a href="https://i.stack.imgur.com/PRhh5.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/PRhh5.png" alt="prediction results after 3 epochs." /></a></p>
  <p>By comparing 2 pictures, I can conclude that when the number of epochs increases, the predictions are more likely to converge to some specific value which is around -0.1.</p>