Artificial Intelligence, Deep Learning, Machine Learning, Gradient Descent, Optimization.
— 1 Stochastic Gradient Descent — N(Number of data points) Mini-Batch Gradient Descent — N/B(B = Batch Size) AdaGrad AdaGrad — Gradient Descent with Adaptive Learning Rate The main motivation behind the AdaGrad was the idea of Adaptive Learning rate for different features in the dataset, i.e.