Artificial Intelligence, Research, Machine Learning, NLP.
experiments showcased in the paper, achieving around 2x training speedups when compared to commonplace vanilla optimizers such as Adam. Source: https://arxiv.org/pdf/2302.06675.pdf However, as usual with these papers, it’s very hard to assess the most important feature of optimizers: convenience.