jpeg729:

The vocab is rather confusing. The “hidden states” are not truly hidden since they are output to the next layer as well as being used as input for the next timestep. The “cell states” are truly hidden, since they are used only by the LSTM as input for the next timestep.


I see. That makes a lot of sense to me now.
I also tried your suggestion to use a linear layer to distribute the gradients all over parameters of the LSTM - this caused the gradients to vanish more slowly (kind of what was excepted), but the error seems to be grounded somewhere else as the general tendency of the gradient to fall exponentially remained. However, I would prefer to keep it as was (the authors of the paper I’m re-implementing here did it like that - Deep Reinforcement Learning for Object Tracking 3), although I will try and see if this stabilizes training once I got this gradient thing working.
Regarding that, I’m a bit puzzled about this behavior at this point. The plot of the sum of LSTM gradients falling exponentially while the hidden state being reset after each sequence is pretty contradictory (this would obviously also be the case in very easy tasks where we reach some optimum really quick, but we converge to more or less random states…)
Do you have any ideas what other error this behavior might be grounded upon? I will post again in this thread for sure when I found out what happens here. Anyway, thanks for helping jpeg.