Solved by ptrblck in post #7 
              
              
                The torch.sqrt method would create an Inf gradient for a zero input and a NaN output and gradient for a negative input, so you could add an eps value there as well or make sure the input is a positive number: 
x = torch.tensor([0.], requires_grad=True)
y = torch.sqrt(x)
y.backward()
print(x.grad)
> …