Machine Learning, Neural Network Algorithm, Differential Equations.
approaches 0 or infinity. Gradient descent relies on following the gradient to a decent minima of the loss function. A 0 gradient gives no path to follow and a massive gradient leads to overshooting the minima and huge instability. As introduced above, the transformation h(t+1) = h(t) + f(h(t),