Machine Learning, Neural Networks, Artificial Intelligence, Google Brain, Math.
resulting gradient we get a plot like below. Which indicates as the alpha value increase the values of final gradient decreases due to saturation. Cumulating Interior Gradients In this section, the authors extend the interior gradients by summing all of the gradient values up at different alpha