Artificial Intelligence, Deep Learning, Machine Learning, Gradient Descent, Optimization.
vanilla GD on the flat surface because it is not only using the current gradient value to update the parameters but also a history of gradients value till that point. By the time it has taken around 5 to 6 steps on the flat surface, it’s history would have grown up to a certain extent, allowing the