<p>You can do something called scheduled sampling. A major problem with RNNs and with generative models in general is that during train time, they are not trained on their own predictions. Rather, they are trained on the gold labels. During inference time, there are no gold labels available and the model is fed it's own generations. This is something the model has not done before and if the model makes a mistake during the beginning of the generation phase, it will be more likely to make more mistakes.</p>
<p>The idea is to allow the model to be trained on it's own outputs gradually using a decay parameter which increases the probability of a predicted token being fed into the RNN rather than the gold label.</p>
<p>You can read about it in this paper : <a href="https://arxiv.org/abs/1506.03099" rel="nofollow noreferrer">Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks</a></p>
What are the possible RNN modification to improve the model?
<p>I want to improve MNIST handwritten model with RNN model/LSTM .. have any one tried modification on RNN .. if so what are the different ways I can improve model ..please suggest</p>