Machine Learning, Data Science, NLP, Mathematics, AI.
gradients: the gradients can be so small the error that back-propagated to that weight/bias terms in the layer is ineffective in updating any weights. This prevents learning in early layers. The deeper a neural network becomes, the greater a problem this becomes. Vanishing Gradients in RNN