Solved by ptrblck in post #2 
              
              
                You might not have deleted all references to all parameters and tensors, so these objects might still hold the memory. 
Also, another application might of course use the GPU memory (but I assume you are sure that PyTorch uses it).