I use the full batch gradient decent method with Adam optimizer, obtaining the following result

1484×508 70.6 KB

as seen in the picture, the direction of the first update is not along the direction of the gradient, but during the first update, there are no momentum acculumated, why the direction of first update is the same as gradient.