@Miguel_Suau
I agree as hidden states from the sampling phase can already be considered deprecated.
I’m still looking into the issue of observing the flow of gradients into the hidden states.
I just rendered a graph based on this code https://stackabuse.com/time-series-prediction-using-lstm-with-pytorch-in-python/ 2 .
Digraph.gv2099×6249 800 KB
The sequence length is defined as 12 and it looks like that some structure of this graph is repeated 12 times. But this behavior is not apparent for this repo: ikostrikov/pytorch-a2c-ppo-acktr-gail (see one of the graphs above). In that repo, the hidden states are recomputed during the mini batch updates.