the “idle time” occurs in two scenarios:

at the beginning of every iteration of the training loop posted above.
at the beginning of a test operation (very similar to the training code above, just iterating on a test dataloader)

just to be clear - even if i don’t switch from train to eval at all and just run the training loop, i still wait at the beginning of every training epoch
otherwise, during an epoch the GPU is constantly loaded and working at a constant pace.