Deep Learning, Regularisation, Optimization Algorithms, Deep Neural Networks, Gradient Descent.
the output. These activations change every time as the parameters of the previous layers change. Hence, we see a lot of shift in the activation values. Batch norm reduces the amount that the distribution of these hidden unit values shift around. Additionally, it turns out that batch norm has a