@googlebot You’re right, for training loop, I detached the previous model outputs when using them as next inputs, memory allocated remains the same after each prediction. For evaluation though, apparently I just needed to add this to torch.no_grad() context and memory is freed after each prediction.