Question: While using "Two class neural network" in Azure ML, I encountered "Momentum" property. As per [documentation](https://docs.microsoft.com/en-us/azure/machine-learning/studio-module-reference/neural-network-regression), which is not clear, it says

> 
> For The momentum, type a value to apply during learning as a weight on
> nodes from previous iterations.
> 
> 
> 

Although that is not very clear. Can someone please explain?

Answer: 
As a non formal definition and non thorough, you can understand momentum in the gradient descent as an inertia.

So when you are doing down the hill in the optimization problem you just add "momentum" to the descending and it helps with things as noise in the data, saddle points and stuff like that.

For a more thorough analysis see <https://towardsdatascience.com/stochastic-gradient-descent-with-momentum-a84097641a5d>

This is not dependent on azure but common in all NN
Momentum is a technique to prevent sensitive movement. When the gradient gets computed every iteration, it can have totally different direction and the steps make a zigzag path, which makes training very slow. Something like this.

[![enter image description here](https://i.stack.imgur.com/epW89.jpg)](https://i.stack.imgur.com/epW89.jpg)

To prevent this from happening, momentum kind of stabilizes this movement.
You can find more in the [Following Article](https://ai-pool.com/a/s/optimization-methods--gradient-descent)