Solved by ptrblck in post #2 
              
              
                I don’t think this operation is fused, as you’ve already mentioned that you are not scripting the model. 
torch.cuda.memory_allocated() will return the currently allocated memory, not its peak (use torch.cuda.max_memory_allocated() for is or torch.cuda.memory_summary()). 
Based on your description, …