Solved by albanD in post #2 
              
              
                Hi, 
How do you measure the gpu memory usage? You should be using torch.cuda.memory_allocated to get the memory actually used by Tensors. Note that this number only counts Tensors, and not the memory required by the cuda driver at initialization. 
If you check via nvidia-smi, you will see a larger n…