I think so, too. I just don’t know how to fix this…
Another evidence is that when I start running another instance of training (that uses the same dataset) on the other GPU, the one that was previously running gets affected and slows down.
It alternates between 10-20 seconds and 1-4 seconds per loading each batch during the first epoch. In the next epoch, it maintains 0.3 seconds per batch, which seems normal.