filippobistaffa:

How is that a apparently “sub-optimal” approach yields a higher processing rate?
Am I doing something wrong, or is it related to the (relatively) small batch size?


Your approach might have a slower initialization time, but could be faster during the training.
The main drawback is that you most likely load the complete dataset onto the device, which will use (some) GPU memory which will thus not be available for your model.
Also, preprocessing each sample might not be that easy, e.g. since a lot of image processing transformations from torchvision rely on PIL.Image, which uses numpy arrays under the hood and thus needs CPU arrays.
If that’s not the case and your dataset is “quite” small, your approach is valid.
PS: We also recommend to use DistributedDataParallel 137 (as Multi-Process Single-GPU), which should be the fastest approach.