Pytorch, Machine Learning, Gradient Descent, Python, Hands On Tutorials.
gradient update going froward. Normally very small values of momentum are sufficient, but for the sake of visualization we’re going to set it to the crazy value of 0.9—kids, do NOT try this at home: multi_plot(lr=0.1, epochs=100, momentum=0.9) Figure 6. Visualized gradient descent down all loss