Thanks for the very swift reply.
What I’m trying to say is that my replay bufffer has no chance to accumulate to a size that it will even become a problem before my computer runs out of memory. How I determined that my replay buffer is not the problem is by limiting its capacity to only a few hundred episodes, since I’m only storing (96,96) 2D arrays in it, a few hundred of those can’t have caused the memory problem.
When I don’t train, I still continue to see this behaviour. By not training I mean that I comment out all parts of the code that attempts to do any form of backprop or loss estimation, so the only time the network is being invoked is when I do the forward pass.