Machine Learning, Deep Learning, Neural Networks, Optimization Algorithms, Adam.
(Adaptive Moment Estimation) For the moment, Adam is the most famous optimization algorithm in deep learning. At a high level, Adam combines Momentum and RMSProp algorithms. To achieve it, it simply keeps track of the exponentially moving averages for computed gradients and squared gradients