Artificial Intelligence, Deep Learning, Machine Learning, Gradient Descent, Optimization.
Momentum gradient descent to take larger and larger steps on the flat surface. Is moving fast always good? To analyze the limitations of Momentum based gradient descent on a different error surface, I have taken some other toy data set and used single sigmoid neuron to find the loss value at