title: Overfitting in LSTM even after using regularizers
body: <p>I am having a time series prediction problem and building an LSTM like below :</p>
  
  <pre><code>def create_model():
      model = Sequential()
      model.add(LSTM(50,kernel_regularizer=l2(0.01), recurrent_regularizer=l2(0.01), bias_regularizer=l2(0.01), input_shape=(train_X.shape[1], train_X.shape[2])))
      model.add(Dropout(0.591))
      model.add(Dense(1))
      model.compile(loss='mean_squared_error', optimizer='adam')
      return model
  </code></pre>
  
  <p>When I train the model on 5 splits like below :</p>
  
  <pre><code>tss = TimeSeriesSplit(n_splits = 5)
  X = data.drop(labels=['target_prediction'], axis=1)
  y = data['target_prediction'] 
  for train_index, test_index in tss.split(X):
     train_X, test_X = X.iloc[train_index, :].values, X.iloc[test_index,:].values
     train_y, test_y = y.iloc[train_index].values, y.iloc[test_index].values
     model=create_model()
     history = model.fit(train_X, train_y, epochs=10, batch_size=64,validation_data=(test_X, test_y), verbose=0, shuffle=False)
  </code></pre>
  
  <p>I get an overfitting problem. The graph of loss is attached <a href="https://i.stack.imgur.com/a8gLB.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/a8gLB.png" alt="enter image description here"></a></p>
  
  <p>I am not sure why there is overfitting when I use regularizers in my Keras model. Any help is appreciated .</p>
  
  <p><strong>EDIT:</strong>
  Tried the architectures </p>
  
  <pre><code>def create_model():
      model = Sequential()
      model.add(LSTM(20, input_shape=(train_X.shape[1], train_X.shape[2])))
      model.add(Dense(1))
      model.compile(loss='mean_squared_error', optimizer='adam')
      return model
  
  
  def create_model(x,y):
      # define LSTM
      model = Sequential()
      model.add(Bidirectional(LSTM(20, return_sequences=True), input_shape=(x,y)))
      model.add(TimeDistributed(Dense(1, activation='sigmoid')))
      model.compile(loss='mean_squared_error', optimizer='adam')
      return model 
  </code></pre>
  
  <p>but still it is overfitting.</p>