Solved by ptrblck in post #2 
              
              
                It should work and pass a zero gradient to the masked elements: 
a = torch.randn(2, 2, requires_grad=True)
b = torch.randn(2, 2)

diff = a - b
diff[ diff < 0 ] = 0
print(diff)
> tensor([[0.6480, 0.0000],
          [0.0000, 0.0272]], grad_fn=<IndexPutBackward>)

diff.mean().backward()
print(a.grad)
> …