Machine Learning, Deep Learning, Neural Networks, Optimization, Towards Data Science.
bit more intuition of how momentum works. Let’s expand our definition of exponentially weighed averages for three consecutive elements of the new sequence V. V — New sequence. S — original sequence. Combining all of them together we get: And then simplifying it a bit: From this equation we see,