Deep Learning, Regularisation, Optimization Algorithms, Deep Neural Networks, Gradient Descent.
techniques we have seen earlier. Learning Rate Decay If we slowly reduce the learning rate over time, we might speed up the learning process. This process is called learning rate decay. Initially, when the learning rate is not very small, training will be faster. If we slowly reduce the learning