Machine Learning, Gradient Descent, Data Visualization, Data Science.
algorithm) doesn’t; it can only take one step at a time and explore its surroundings, analogous to walking in the dark with only a flashlight. The vanilla gradient descent is vanilla because it just operates on the gradients. The following methods do some additional processing of the gradients to