Solved by ptrblck in post #9 
              
              
                nn.DataParallel will split the input tensor in dim0 and will send each chunk to the corresponding model replica on a specific device. 
I.e. your model’s forward will get an input of the shape [batch_size//nb_gpus, *]. 
If you need to create y1 and y2 inside the forward method, you should also consi…