Optimization, Learning Parameters, Gradient Descent, Stochastic Gradient, Deep Learning.
gradients for all the data points, we make an update with gradients of just one data point at a time. Thus, the name stochastic, as we are estimating the total gradient based on a single data point. Almost like tossing a coin only once and estimating P(heads). Now if we have a million data points