Hi all,
I want to add memory cell/layer to my model to improve performance on Atari games. But I’m not sure if I’m doing it right!
If I understood recurrent networks correctly, they take a sequence of observations from the environment. This defies the i.i.d assumption as the observations in the batch become highly correlated, but that is fine since the memory cells are made for that specific task.
That said, what I’m struggling with is how to use the hidden state output in my training. Should I:
1- Save it after each step and feed it back to the network in step + 1 until episode end?
2- OR ignore it and just initiate zero tensor at each step?
Secondly:
Assuming the GRU is feeding a dense layers below. Which output should I pass to the FC layer (output or hn)?