Artificial Intelligence, Deep Learning, Machine Learning, Gradient Descent, Optimization.
its rapid growth. The way these algorithms uses the history is different, In Momentum GD, we use history to compute the current update whereas in case of RMSProp history was used to adjust the learning rate (shrink or boost). Adam combines these two separate histories into one algorithm. Adam