Optimizer, Gradient Descent, Machine Learning, Adam, Sgd.
curves are very common in Neural networks due to complex functions. Now let’s talk about some other issues, with the gradient descent algorithms. 2. Exploding and Vanishing Gradient In Neural Networks, if we are using very deep networks, we have a problem called Vanishing and Exploding gradients.