Machine Learning, Deep Learning, Neural Networks, Optimization Algorithms, Adam.
loss function slowly oscillate towards vertical axes. These bounces occur because gradient descent does not store any history about its previous gradients making gradient steps more undeterministic on each iteration. This example can be generalized to a higher number of dimensions. As a