Your self.target always equals self.input. So self.crit(input, self,target) is always 0. 0 is a convex minima for MSE, so it’s grad is 0. Therefore, your grad will not change no matter what finite number you put there.