Machine Learning, Deep Learning, Neural Networks, Optimization Algorithms, Adam.
absolute magnitudes of gradient are ignored which can make the method tremendously slow to converge. A positive aspect about this algorithm is the fact only a single bit is required to store signs of gradietns which can be handy in distributed computations with strict memory requirements. Adam