Artificial Intelligence, Deep Learning, Machine Learning, Gradient Descent, Optimization.
boost the gradient update with a higher learning rate. In AdaGrad, we are dividing the learning with the history of gradient value until that point. Non-sparse features will have a large history value because they would be getting frequent updates, by dividing the learning rate with the large