Solved by rvarm1 in post #2 
              
              
                I’m assuming that since your tensor of shape 195HW is a single training example, you don’t want it to be split by N GPUs, since 195 is not the batch size in this case? If you pass in your training examples in the form such as batch_size * C * H * W for example then DataParallel/DDP should divide alo…