Optimization, Learning Parameters, Gradient Descent, Stochastic Gradient, Deep Learning.
once before updating the parameters. Why? Because this is the true gradient of the loss as derived earlier in part-1 (the sum of the gradients of the losses corresponding to each data point). This is a good thing as we are not approximating anything. Hence, all theoretical assumptions and