title: validation accuracy not improving
body: <p>No matter how many epochs I use or change learning rate, my validation accuracy only remains in 50's. Im using 1 dropout layer right now and if I use 2 dropout layers, my max train accuracy is 40% with 59% validation accuracy. And currently with 1 dropout layer, here's my results:</p>
  
  <pre><code>2527/2527 [==============================] - 26s 10ms/step - loss: 1.2076 - accuracy: 0.7944 - val_loss: 3.0905 - val_accuracy: 0.5822
  Epoch 10/20
  2527/2527 [==============================] - 26s 10ms/step - loss: 1.1592 - accuracy: 0.7991 - val_loss: 3.0318 - val_accuracy: 0.5864
  Epoch 11/20
  2527/2527 [==============================] - 26s 10ms/step - loss: 1.1143 - accuracy: 0.8034 - val_loss: 3.0511 - val_accuracy: 0.5866
  Epoch 12/20
  2527/2527 [==============================] - 26s 10ms/step - loss: 1.0686 - accuracy: 0.8079 - val_loss: 3.0169 - val_accuracy: 0.5872
  Epoch 13/20
  2527/2527 [==============================] - 31s 12ms/step - loss: 1.0251 - accuracy: 0.8126 - val_loss: 3.0173 - val_accuracy: 0.5895
  Epoch 14/20
  2527/2527 [==============================] - 26s 10ms/step - loss: 0.9824 - accuracy: 0.8165 - val_loss: 3.0013 - val_accuracy: 0.5917
  Epoch 15/20
  2527/2527 [==============================] - 26s 10ms/step - loss: 0.9417 - accuracy: 0.8216 - val_loss: 2.9909 - val_accuracy: 0.5938
  Epoch 16/20
  2527/2527 [==============================] - 26s 10ms/step - loss: 0.9000 - accuracy: 0.8264 - val_loss: 3.0269 - val_accuracy: 0.5943
  Epoch 17/20
  2527/2527 [==============================] - 26s 10ms/step - loss: 0.8584 - accuracy: 0.8332 - val_loss: 3.0011 - val_accuracy: 0.5934
  Epoch 18/20
  2527/2527 [==============================] - 26s 10ms/step - loss: 0.8172 - accuracy: 0.8378 - val_loss: 2.9918 - val_accuracy: 0.5949
  Epoch 19/20
  2527/2527 [==============================] - 26s 10ms/step - loss: 0.7796 - accuracy: 0.8445 - val_loss: 2.9974 - val_accuracy: 0.5929
  Epoch 20/20
  2527/2527 [==============================] - 25s 10ms/step - loss: 0.7407 - accuracy: 0.8502 - val_loss: 3.0005 - val_accuracy: 0.5907
  </code></pre>
  
  <p>Again max, it can reach is 59%. Here's the graph obtained:</p>
  
  <p><a href="https://i.stack.imgur.com/di9AT.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/di9AT.png" alt="enter image description here"></a></p>
  
  <p>No matter how much changes I make, the validation accuracy reaches max 59%.
  Here's my code:</p>
  
  <pre><code>BATCH_SIZE = 64
  EPOCHS = 20
  LSTM_NODES = 256
  NUM_SENTENCES = 3000
  MAX_SENTENCE_LENGTH = 50
  MAX_NUM_WORDS = 5000
  EMBEDDING_SIZE = 100
  
  encoder_inputs_placeholder = Input(shape=(max_input_len,))
  x = embedding_layer(encoder_inputs_placeholder)
  encoder = LSTM(LSTM_NODES, return_state=True)
  
  encoder_outputs, h, c = encoder(x)
  encoder_states = [h, c]
  
  decoder_inputs_placeholder = Input(shape=(max_out_len,))
  
  decoder_embedding = Embedding(num_words_output, LSTM_NODES)
  decoder_inputs_x = decoder_embedding(decoder_inputs_placeholder)
  
  decoder_lstm = LSTM(LSTM_NODES, return_sequences=True, return_state=True)
  decoder_outputs, _, _ = decoder_lstm(decoder_inputs_x, initial_state=encoder_states)
  
  decoder_dropout1 = Dropout(0.2)
  decoder_outputs = decoder_dropout1(decoder_outputs)
  
  decoder_dense1 = Dense(num_words_output, activation='softmax')
  decoder_outputs = decoder_dense1(decoder_outputs)
  
  opt = tf.keras.optimizers.RMSprop()
  
  model = Model([encoder_inputs_placeholder,
    decoder_inputs_placeholder],
    decoder_outputs)
  model.compile(
      optimizer=opt,
      loss='categorical_crossentropy',
      metrics=['accuracy']
  )
  
  history = model.fit(
      [encoder_input_sequences, decoder_input_sequences],
      decoder_targets_one_hot,
      batch_size=BATCH_SIZE,
      epochs=EPOCHS,
      validation_split=0.1,
  )
  </code></pre>
  
  <p>Im very confused why only my training accuracy is updating, not the validation accuracy.</p>
  
  <p>Here's the model summary:</p>
  
  <pre><code>Model: "model_1"
  __________________________________________________________________________________________________
  Layer (type)                    Output Shape         Param #     Connected to                     
  ==================================================================================================
  input_1 (InputLayer)            (None, 25)           0                                            
  __________________________________________________________________________________________________
  input_2 (InputLayer)            (None, 23)           0                                            
  __________________________________________________________________________________________________
  embedding_1 (Embedding)         (None, 25, 100)      299100      input_1[0][0]                    
  __________________________________________________________________________________________________
  embedding_2 (Embedding)         (None, 23, 256)      838144      input_2[0][0]                    
  __________________________________________________________________________________________________
  lstm_1 (LSTM)                   [(None, 256), (None, 365568      embedding_1[0][0]                
  __________________________________________________________________________________________________
  lstm_2 (LSTM)                   [(None, 23, 256), (N 525312      embedding_2[0][0]                
                                                                   lstm_1[0][1]                     
                                                                   lstm_1[0][2]                     
  __________________________________________________________________________________________________
  dropout_1 (Dropout)             (None, 23, 256)      0           lstm_2[0][0]                     
  __________________________________________________________________________________________________
  dense_1 (Dense)                 (None, 23, 3274)     841418      dropout_1[0][0]                  
  ==================================================================================================
  Total params: 2,869,542
  Trainable params: 2,869,542
  Non-trainable params: 0
  __________________________________________________________________________________________________
  None
  </code></pre>