Deep Learning, Optimization, Gradient Descent, Momentum, Learning Parameters.
them, please go through section 5 of my basic stuff blog post — Learning Parameters, Part-0: Basic Stuff (there is even a self-test you can take to get better at interpreting them). Let’s see how effective MBGD is using the same toy neural network we introduced in part-1. Momentum-Based Gradient