Deep Learning, Regularisation, Optimization Algorithms, Deep Neural Networks, Gradient Descent.
is given by: Vt = β * V(t-1) + (1 — β) * Θt Let’s look at how we can implement this: Initialize VΘ = 0 Repeat { get next Θt VΘ = β * VΘ + (1 - β) * Θt } This step takes a lot less memory as we are overwriting the previous values. Hence, it is a computational, as well as memory efficient, process.