I am trying to train a RNN Network for stock price prediction for my Master Thesis. I have additional input values (6), not just the stock prices by itself.
Using an LSTM Network with the “optimal” structure based on Hyperparameter Tuning with Keras Tuner, i observed a significant increase in the losses for training and validation in my case after 4000 Epochs.
My dataset consists of about 12 000 datapoints and i use the Adam optimizer with mean_absolute_error loss function
The Network is quite deep with the following layers:
LSTM (24 unit)
Dropout
LSTM (366 unit)
Dropout
LSTM (150 unit)
Dropout
Dense (1 unit)
I attached a graph of the loss (sorry for the german)
I would really like to understand what leads to this (for me) unexpected behaviour.

Verlustentwicklung_LSTMonly1728×924 137 KB

I reduced the learning rate of the adam optimizer to 0.0002. Which resulted in the V2 results in the graph, which for me reduced but did not solve the issue.
Does anybody have an idea why? I am running the same network and data with the RMSProp optimizer right now. I will get the results tomorrow.