Hi, @ptrblck thanks for your response. I figured out that the issue is not related to the forward pass, but rather PyTorch was caching the memory, and therefore I saw an increased memory footprint on nvidia-smi. I have figured out the solution. Thanks once again.