Machine Learning, Deep Learning, Neural Networks, Optimization Algorithms, Adam.
way, Adagrad deals with vanishing and exploding gradient problems. Under the hood, Adagrad accumulates element-wise squares dw² of gradients from all previous iterations. During weight update, instead of using normal learning rate α, AdaGrad scales it by dividing α by the square root of the