This isn’t a memory leak as the memory is still available within PyTorch.
Roughly speaking, PyTorch allocates CUDA memory in chunks and then puts tensors in it. It can only “return” (with empty caches) chunks that are completely unused (i.e. don’t have any tensors in them).
What happens here is that with the additional intermediate the data of outp sits inside the 4GB allocation that is mostly free and available to PyTorch but cannot be returned.
You can get a glimpse of this by calling: torch.cuda.memory_snapshot 1 and it also links to a brief note on memory management (e.g. how to avoid caching allocations for debugging):
print(torch.cuda.memory_snapshot())
Best regards
Thomas