Machine Learning, Deep Learning, Neural Networks, Optimization, Towards Data Science.
weight and alpha — learning rate. The other way, which is most popular way to write momentum update rules, is less intuitive and just omits (1 - beta) term. This is pretty much identical to the first pair of equation, the only difference is that you need to scale learning rate by (1 - beta) factor.