Rajat_Sharma1:

It is the network architecture where output of the model is fed back as input along with other input. For the very first time it is initialized as zeros.


This might be the culprit, as the computation graph might be kept alive using this approach.
Assuming you don’t want to backpropagate through the previous iterations, you could detach the tensor via op.detach_() in each iteration.