OK found the issue.
Apparently the gpu still works in the background and measuring your runtime only works correctly if you use torch.cuda.synchronize() :

  
    
    
    Strange speed of tensor.sum() 
  
  
    Hello, all. 
I meet a problem that speed of  function tensor.sum() is strange when calculate the same kind of Tensor. 
I want to calculate the confusion matrix of a foreground/background segmentation model, and below is my testing code: 
import time
import torch


def func(pr, gt):
    dump = 0.0
    for gt_i in range(2):
        for pr_i in range(2):
            num = (gt == gt_i) * (pr == pr_i)
            start = time.time()
            dump += num.sum()
            print("Finding Time: {} {}…