Solved by ptrblck in post #2 
              
              
                The gradients you are seeing is accumulated from the division and the max operation. 
Here is the step by step approach: 
a = torch.tensor([1., 2., 3.], requires_grad=True)

# Get div gradients
y = a / a.max().detach()
y.backward(torch.ones_like(y))
print('div grad: ', a.grad)
a.grad.zero_()

# Get …