So timing in Pytorch seems to be very weird. I tried the code on this thread to verify some numbers reported in various papers like Shufflenet. Here’s the code:
cudnn.benchmark = True

model = alexnet().cuda().eval()

# Shufflenet model from https://github.com/jaxony/ShuffleNet/blob/master/model.py
# model = ShuffleNet().cuda().eval()

x = torch.rand(1, 3, 224, 224).cuda()

torch.cuda.synchronize()
# torch.cuda.synchronize()

sum_time = 0
total_runs = 100
warm_up = 50

for i in range(total_runs):
    torch.cuda.synchronize()
    a = time.perf_counter()
    # y = None
    y = model(x)
    torch.cuda.synchronize()
    b = time.perf_counter()

    if i > warm_up:  # give torch some time to warm up
        sum_time += ((b-a) * 1000)

print("Time (ms): {:.04}".format(sum_time / (total_runs-warm_up)))

For Alexnet, I get  ~1.327 ms but the Shufflenet model gives me ~5.403 ms, both of which are very different from the numbers reported in the Shufflenet paper.
This begs the question that is using wall clock time even a reliable source of measuring speed in Pytorch?
@ngimel @apaszke @Andrei_Pokrovsky