Hi Rishi!



 snau:

the gradient is constant over the epochs


Calculate analytically the derivative of loss with respect to param,
specifically for the values of a, b, and c that you are using.  (Hint:
You will discover that it is constant.)


and becomes Nan afterwards.


In essence, you are having pytorch calculate sqrt (x**2).
sqrt (x) is singular at x = 0, and its derivative diverges (and
pytorch – rightly – doesn’t replace sqrt (x**2) with x).
Consider:
>>> x = torch.tensor ([0.0], requires_grad = True)
>>> torch.sqrt (x).backward()
>>> x.grad
tensor([inf])
>>> x = torch.tensor ([0.0], requires_grad = True)
>>> torch.sqrt (x**2).backward()
>>> x.grad
tensor([nan])

Best.
K. Frank