Thanks @ptrblck,
This occurs even if I don’t store the loss. Yes, I see a huge growth in memory usage; I am sure that it is storing the entire computation graph (across iterations). Is there a way to prevent this?