Optimizer, Gradient Descent, Machine Learning, Adam, Sgd.
have been developed with two primary goals. Optimizing the learning rate Dampening the vertical movement to speed up the gradient descent. Before we move to the optimizers let’s see a concept called exponentially weighted average. Weighted Averages In the diagram, the deep blue points are the data