Consider applications, e. g. Neural Style Transfer, where the input is not batched but rather a single image. Can an application like this benefit from multiple GPUs if the model is wrapped in a nn.DataParallel?
I’m asking, because I want to be sure before I buy new hardware. If I understand the multiprocessing correctly, for such applications multiple GPUs are useless and thus a single GPU with high memory would be the better choice.