Optimizer, Gradient Descent, Machine Learning, Adam, Sgd.
horizontal movement speeds up as the movements add up positively to solve our problem. The equation becomes, This ‘Vdw’ is called Moment. The update equation becomes: Another thing to notice is, W(old) is no longer equal to W(new) when dL/dW=0. It has another component that keeps the descent from