Pytorch, Machine Learning, Gradient Descent, Python, Hands On Tutorials.
from the last step is going to carry us, then Nesterov momentum instead calculates the current gradient by looking ahead to where that will be, giving us the scenario in figure 7(b) below: Figure 7. (a) Momentum vs. (b) Nesterov momentum. multi_plot(lr=0.1, epochs=100, momentum=0.9, nesterov=True)