Well yes the gradients tensor won’t be created. But that is actually not what uses a lot of memory. For example, check the memory usage after transfering a model to the gpu (so only the weights are there) and after running a forward pass (so weights + temporary buffer). You will see that the weights are a small fraction.
In your test, it is possible that these tensors are small enough that you don’t see they’re not there.
Keep in mind that the CUDA allocator will not return to the os memory just after a tensor is freed (for speed reason), you might want to use torch.cuda.memory_allocated() 52 to get the exact memory usage by tensors.