Optimization, Learning Parameters, Gradient Descent, Stochastic Gradient, Deep Learning.
at IIT-Madras. Motivation Let us look at vanilla gradient descent we talked about in part-1 of the series. If you observe the block of code between the two weirdly commented lines, you notice that the gradient is calculated across the entire dataset. Meaning, the algorithm goes over the entire data