I would expect DataLoader to load batches concurrently to the main process, refilling the buffer as soon as a batch is consumed from the buffer. However, when I track the utilization of GPU and order of loading vs execution I see some different behavior:

loading of the whole buffer (expected)
consuming the whole buffer by execution of all the batches until the buffer is empty (not expected)
loading the whole buffer again without parallel execution (not expected)
goto 2.

This obviously results in dips in GPU utilization when in step 3.
I set:
num_workers >= 1
pin_memory = True/False (doesn’t influence the described behavior)
Did anyone experience the same? What could be the issue?