Optimization, Learning Parameters, Gradient Descent, Stochastic Gradient, Deep Learning.
data points. If you look closely, you can see that our descent makes many tiny oscillations. Why? Because we are making greedy decisions. Each point is trying to push the parameters in a direction most favorable to it (without being aware of how this affects other points). A parameter update which