Hi,
I am trying to design backward hooks that depend on the loss.
In the single GPU version, i modify the hook between the forward and backward pass depending on the loss value.
I would like to scale to multi-gpu and modify each replicated hook depending on the loss of each GPU.
To do so, Is it possible to access the model replicas in torch.nn.DataParallel() between forward() and  .backward() call?
Thanks for helping