Gradient Descent, Gradient Boosting.
descent and less noisy updates than stochastic gradient descent. 4. Momentum gradient descent — Takes previous updates as an account and accelerate the update in that direction. Fast and smooth convergence. 5. Nesterov accelerated gradient(NAG) — Modification of momentum gradient descent.