Could you move the device=cuda part into the for loop?
Usually you load your data on the CPU, while the GPU is busy training your model.
Once a training iteration is done, your data batches are waiting in the queue to be transferred onto the GPU.