Machine Learning, Deep Learning, Stochastic Gradient, Data Science, Editors Pick.
variance. This makes the cost function fluctuate more on each iteration, when compared to Gradient Descent, making it harder for the algorithm to converge. New variations of Stochastic Gradient Descent have been developed to tackle these problems. For example Mini-batch Stochastic Gradient Descent,