That is why I am trying to figure this whole multithreading loader question once and for all. I found out that a lot of other practitioners have the same issues but I couldn’t find a reasonable solution yet.

So your observation (quoted below) only applies to when the memory is pinned? What do you get when you set it to False?



 halahup:

o, I use 8 workers on the host to feed the data to the device. The model works just fine, however, I have noticed that not all workers do something (in fact, all 8 don’t seem to do anything), while the main thread is the one that loads the data (loads up to 10% though):