<p>As discussed in the comments, the problem lies in the large batch size, and - maybe also - in the optimizer used for the training.</p>

<p>It is hard to determine an exact reason why your algorithm did not converge with the current settings, but it can be argued as such:</p>

<h2>Large batch sizes have a slower convergence.</h2>

<p>Counter-intuitively, training with larger batch sizes will actually in some instances slow down your training. The reason behind this is purely speculative and depends on the exact nature and distribution of your data. Generally, though, having a smaller batch size means having more frequent updates. If your calculated gradients all point in a similar direction, having these more frequent updates will lead to a faster convergence.<br/>
Good practice is to have a batch size that is <em>never</em> larger than 1000. In most scenarios 128 is a good rule of thumb and a nice trade-off between the speed advantage of larger batches, and the nice convergence properties of smaller batch sizes. Note that this only makes sense in cases where you have a lot of training data.</p>

<p>Also note that theoretically the gradients of multiple examples in that large setting can "average out", meaning the large batch size will have only a very small and indistinct gradient. Having fewer samples in a mini-batch will reduce this chance, although it increases the risk of "going the wrong direction" (i.e. having a gradient that points in the opposite directin).</p>

<h2>SGD is a good starting point, but there exist several optimizations.</h2>

<p>One of these "smarter" variants is the suggested ADAM method. There is a highly cited <a href="https://arxiv.org/pdf/1412.6980.pdf" rel="nofollow noreferrer">paper about it</a>, which can give you a vague idea of what happens under the hood. Essentially, SGD is a very naive solution that does not have any special assumptions or built-in optimizations.
(From what I know, ADAM for example uses first-order derivatives)</p>

<p>There exist many different ones, and there is a ton of <a href="http://ruder.io/optimizing-gradient-descent/" rel="nofollow noreferrer">theoretical articles</a> (and <a href="https://www.youtube.com/watch?v=OWzkRD6MjYI" rel="nofollow noreferrer">practical comparisons</a>) of the different implementations. It is worth a lot to at least understand partially what the parameters do, and know what values make sense to set them to.</p>

<p>For example, you already set the learning rate to a sensible value (0.001); personally, I usually end up with values between 0.001-0.01, and maybe use learning rate decay over time if I have larger learning rates.</p>
Gensim doc2vec 300 dimensional vector fed into keras,lstm model not working.Loss is not diminishing
<p>This is my code snippet:</p>

<pre><code>model=keras.Sequential()
model.add(keras.layers.LSTM(28,input_shape=(300,1),return_sequences=True))
model.add(keras.layers.Dropout(0.4))
model.add(keras.layers.LSTM(14))
model.add(keras.layers.Dropout(0.5))
model.add(keras.layers.Dense(2,activation="softmax"))
sgd=keras.optimizers.SGD(lr=0.001)
model.compile(optimizer=sgd,loss=keras.losses.sparse_categorical_crossentropy)
model.fit(trainData,labeledData.sentiment,epochs=20,batch_size=3000)
</code></pre>

<p>trainData shape is [batch_size,300,1],when i begin training this model,loss is not going down.</p>

<p>Epoch 1/20
25000/25000 [==============================] - 2s 89us/step - loss: 0.6927
Epoch 2/20
25000/25000 [==============================] - 0s 8us/step - loss: 0.6928
Epoch 3/20
25000/25000 [==============================] - 0s 8us/step - loss: 0.6928
Epoch 4/20
25000/25000 [==============================] - 0s 8us/step - loss: 0.6928
Epoch 5/20
25000/25000 [==============================] - 0s 8us/step - loss: 0.6928
Epoch 6/20
25000/25000 [==============================] - 0s 8us/step - loss: 0.6926</p>

<p>what am i missing?</p>