Machine Learning, Deep Learning, Neural Networks, Optimization Algorithms, Adam.
training a deep network because of its slow convergence rate. This became a motivation for researchers to develop optimization algorithms which accelerate gradient descent. Before reading this article, it is highly recommended that you are familiar with the exponentially moving average concept