Actually, after some thinking I realized that the problem is not about the size of the target tensor.
What I think would help me solve this problem is to know if, is there any way I could override the split of some tensors done by nn.DataParallel? @ptrblck
So that the corresponding targets for the images are sent to the specific GPU which is handling their input image.