Artificial Intelligence, Machine Learning, Deep Learning, Neural Networks.
elongated valleys: In cases where the scale of the model features is different, the loss function has an elongated valley. In such cases, the model convergence slows down. This is because the steepest slope does not point directly towards the optima. AdaGrad: Adapt to Speed Up AdaGrad algorithm