title: Loss increases after 4000 Epochs
body: <p>I am trying to train a RNN Network for stock price prediction for my Master Thesis. I have additional input values (6), not just the stock prices by itself.</p>
  <p>Using an LSTM Network with the &quot;optimal&quot; structure based on Hyperparameter Tuning with Keras Tuner, i observed a significant increase in the losses for training and validation in my case after 4000 Epochs.</p>
  <p>My dataset consists of about 12 000 datapoints and i use the <code>Adam</code> optimizer with <code>mean_absolute_error</code> loss function</p>
  <p>The Network is quite deep with the following layers:</p>
  <p>LSTM (24 unit)<br />
  Dropout<br />
  LSTM (366 unit)<br />
  Dropout<br />
  LSTM (150 unit)<br />
  Dropout<br />
  Dense (1 unit)</p>
  <p>I attached a graph of the loss (sorry for the german)</p>
  <p>I would really like to understand what leads to this (for me) unexpected behaviour.</p>
  <p><a href="https://i.stack.imgur.com/4gmdj.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/4gmdj.png" alt="Loss development" /></a></p>