A Theory on Adam Instability in Large-Scale Machine Learning
Is it possible to use derivative-free&#x2F;black-box optimizers to train these large networks?<p>From what I understand, gradient descent and its cousins can&#x27;t suddenly jump to a distant global optimum.