data size: [16, 3, 473, 473]
In method 1:
t0 = time.time()
batch = next(trainloader)
images, labels, _, _ = batch
images = images.to(device)
labels = labels.long().to(device)

t.append(time.time() - t0)
if i_iter % 20 == 0:
    print('batch read time: ', sum(t[-20:]) / 20)

###
    ....
    model forward and backward
    ....
###
t1.append(time.time() - t0)
if i_iter % 20 == 0:
    print('model train time: ', sum(t1[-20:]) / 20)


In method 2：
t0 = time.time()
batch = q.get() # .to() has been executed in other thread before
images, labels, _, _ = batch

t.append(time.time() - t0)
if i_iter % 20 == 0:
    print('batch read time: ', sum(t[-20:]) / 20)

###
    ....
    model forward and backward
    ....
###
t1.append(time.time() - t0)
if i_iter % 20 == 0:
    print('model train time: ', sum(t1[-20:]) / 20)

I have not ues torch.cuda.synchronize()  before starting and stopping the timer.