Machine Learning, Deep Learning, Neural Networks, Optimization, Towards Data Science.
that underlies the work of momentum. Exponentially weighed averages Exponentially weighed averages deal with sequences of numbers. Suppose, we have some sequence S which is noisy. For this example I plotted cosine function and added some Gaussian noise. It looks like this: Out sequence. Note, that