Machine Learning, Artificial Intelligence, Neural Networks, Data Science, Towards Data Science.
converge to M different local optima and save network parameters. Left: Illustration of SGD optimization with a typical learning rate schedule. The model converges to a minimum at the end of training. Right: Illustration of Snapshot Ensembling. The model undergoes several learning rate annealing