Hello.
I notice a strange behavior when training.
If for instance, I have a batch size of N=(1, 2, 3, …) and 8 workers (or more),
I notice that during training, the loop pauses for a few seconds every 8th iterations. As if that pause is somehow used to load data or something of the like.
This behavior appears on a GTX 1080TI GPU.
The same code when executed on a TITAN RTX  does not exhibit the above behavior.
Has someone else encountered this issue? could this be an issue with my computer?
@ptrblck