Deep Learning, Regularisation, Optimization Algorithms, Deep Neural Networks, Gradient Descent.
rate, there is a higher chance of coming close to the global minima. Learning rate decay can be given as: α = [1 / (1 + decay_rate * epoch_number)] * α0 Let’s understand it with an example. Consider: α0 = 0.2 decay_rate = 1 This is how, after each epoch, there is a decay in the learning rate which