Artificial Intelligence, Machine Learning, Deep Learning, Neural Networks.
solves the problem of the elongated valleys by pointing the update step slightly towards the global minima. It achieves this by scaling down the gradient along the steepest dimension. It first accumulates the square of the gradients s = s + square (∇J(θ​) or s = s + ∇J(θ​) ⦻ ∇J(θ​) ⦻ indicates