Artificial Intelligence, Deep Learning, Machine Learning, Gradient Descent, Optimization.
point within that batch. Comparison of Stochastic (Blue Curve) & Mini-Batch GD (Red Curve) We can see the oscillations have reduced in Mini-Batch gradient descent and oscillations are completely contained within the blue curve. In practice, we choose the batch size equal to 16, 32 and 64. Whether