Replace the line loss_fn.backward(x) with loss_fn.backward(). After that change the optimization should behave as expected. To understand the difference between these two calls, please check the autograd tutorial 17 (it contains both examples).



 Martin_Lee:

However, in math, the slope of x**2 at x = 1 should be negative.


This statement does not hold. Derivative of x**2 is 2*x so at x = 1 it’s value is 2 * 1 = 1.