Solved by albanD in post #8 
              
              
                Given that you use torch.no_grad(), I guess that you don’t want this to be taken into account in the backprop. So the simplest solution is just to apply this inplace before doing the DataParallel forward.