Deep Learning, Regularisation, Optimization Algorithms, Deep Neural Networks, Gradient Descent.
exploding gradients. Suppose we are using a neural network with ‘l’ layers with two input features and we initialized the large weights: The final output at the lth layer will be (consider we are using a linear activation function): For deeper networks, L will be large, making the gradients very