Artificial Intelligence, Machine Learning, Deep Learning, Neural Networks.
axis represents the Loss J) | Source: Image by the author. Jump, Leap, or Baby Steps — Learning Rate α The size of the step for the update of parameter values is, perhaps, the most crucial factor in Gradient Descent. This size is decided by a hyperparameter called the learning rate (represented as