https://pytorch.org/docs/stable/data.html#multi-process-data-loading 5 Multi process data loading to GPU is not suggested. You can use multiple process to load them and pass them to GPU together.
I’m not exactly sure whether that helps but according to my experience it helps a little.