Optimization, Learning Parameters, Gradient Descent, Stochastic Gradient, Deep Learning.
guarantees hold (in other words each step guarantees that the loss will decrease). Is this desirable? Sure it is, but what is the flipside? Imagine we have a million points in the training data. To make 1 update to w, b the algorithm makes a million calculations. Obviously, this could be very