Artificial Intelligence, Research, Machine Learning, NLP.
Adam is not the chosen optimizer because it always works best and most efficiently but because it works well enough most of the time, and relevant bottlenecks are elsewhere. That said, I would love to see learned optimizers take off and become the new defacto for challenging unstable learning