Indeed, nn.DataParallel works by splitting batches on the available GPUs. multiprocessing however is used generally for loading data with multiple workers (CPU based) onto the GPU, therefore speeding training considerably.
I’m not familiar with style transfer training, but I don’t see a reason why training would happen with single image batches . Even for inference, you might want to process multiple images at once…