Initializing a member tensor after creation with DataParallel (repost) 
  
  
    I have a member tensor that is created/saved during a backward hook. Hook looks like: 
def saveGrad(self, grad_input, grad_output):
        self.currentGrad = grad_output[0].detach()

This works fine on one GPU but when running on multiple it tells me 'AttributeError: (module) object has no attribute ‘currentGrad’ '. I imagine I could do a .copy_() or something, but I won’t actually know the shape until the network has been run on an image.