Machine Learning, Gradient Descent, Stochastic Gradient, Batch Gradient Descent, Regression.
Batch and Stochastic gradient descent | by Sweta | Medium The contours here depict the averaged cost function, MSE loss. As you can see, we do not optimize the parameters according to the averaged cost function while using SGD. That is the reason behind the noise that is produced in its journey