Machine Learning, Neural Networks, Artificial Intelligence, Google Brain, Math.
significantly. However, GoogleNet had great performance measures while saturation still being present in the network, hence the authors made the below hypothesis. Our hypothesis is that the gradients of important features are not saturated early in the training process. The gradients only saturate