title: Increase the complexity of RNNs
body: <p>I have been working on a big data set on which I have been trying different machine learning algorithms. I have been getting really good results with XGBoost but the best results I got were from SVM. But as I had data spread across 8 years and it's a time series model hence I thought of using RNNs as well. I have been experimenting with the hyper parameters of RNN-LSTMs by increasing the number of layers or the number of nodes per layer as there is no rule of thumb as such. But still I am not even getting closer to my XGBoost results. And the weird thing I am noticing is that even if I increase the number of epochs from 100 to 1000 or I increase the number of hidden layers from 1 to 3 or I change the nodes per layer, the performance of the model is always the same both for training and test sets. I am sharing my RNN-LSTMs code and please let me know if I am missing anything. Theoretically it would at least be as good as SVM if not better as the data quality is also really good and lengthy. </p>
  
  <pre><code># Getting the inputs and the ouputs
  X = allData.iloc[:-48,:-1].values
  y = allData.iloc[:-48,-1].values
  y = y.reshape(-1,1)
  
  # Feature Scaling
  from sklearn.preprocessing import MinMaxScaler
  sc_x = MinMaxScaler()
  sc_y = MinMaxScaler()
  X = sc_x.fit_transform(X)
  y = sc_y.fit_transform(y)
  y = np.ravel(y)
  X = np.reshape(X, (-1,1,X.shape[1]))
  y = np.reshape(y, (-1,1,1))
  
  # Building RNN
  regressor = Sequential()
  regressor.add(LSTM(units = 8,activation = 'tanh',recurrent_activation = 'relu',return_sequences = True, input_shape = (X.shape[1],X.shape[2])))
  regressor.add(LSTM(units = 3,activation = 'tanh', recurrent_activation ='relu', return_sequences = True))
  regressor.add(TimeDistributed(Dense(units = 1)))
  regressor.compile(optimizer = 'adam', loss = 'mean_squared_error')
  regressor.fit(X, y, batch_size = 32,shuffle = False, epochs = 100,validation_split=0.1)
  </code></pre>
  
  <p>I am not able to get even comparable accuracy from this model. What other things can I try in RNNs to make to perform at least similar to other machine learning algorithms?</p>