Machine Learning, Gradient Descent, Data Visualization, Data Science.
relatedly, Momentum) takes the approach of “let me slide down the steep slope first and maybe worry about the slower direction later”. Sometimes, vanilla gradient descent might just stop at the saddle point where gradients in both directions are 0 and be perfectly content there. RMSProp The problem