Solved by ptrblck in post #4 
              
              
                The remaining memory is used by the CUDA context (which you cannot delete unless you exit the script) as well as all other processes shown in nvidia-smi. You can add print(torch.cuda.memory_summary()) to the code before and after deleting the model and clearing the cache and would see no allocations…