Artificial Intelligence, Machine Learning, Deep Learning, Neural Networks.
learning rate (α). The length of the ‘step’ is α times the slope ∇J(θ​). The new value of the parameter becomes the old value minus the length of the step. θnew​=θold​−α∇J(θold​) where θ represents parameters, α is the learning rate, and ∇J(θ) is the gradient of the loss function with respect to θ.