Optimization, Learning Parameters, Gradient Descent, Stochastic Gradient, Deep Learning.
somewhat better estimates of the gradient (analogy: we are now tossing the coin k=2 times to estimate P(heads)). The higher the value of k, the more accurate the estimates will be. In practice, typical values of k are 16, 32, 64. Of course, there are still oscillations, and they will always be