Deep Learning, Pytorch, Fine Tuning, Computer Vision, Transfer Learning.
trapped in local minima. In addition to the learning rate, there are other crucial hyperparameters to consider when defining the optimizer, such as weight decay and momentum (specific to SGD). Let’s take a quick look at both of these hyperparameters: Weight decay, also known as L2 regularization,