Machine Learning, NLP, Bert, Albert, Transformers.
preferable choice over original BERT models as it outperforms them on downstream tasks. Nevertheless, ALBERT requires much more computations due to its longer structures. A good example of this issue is ALBERT xxlarge which has 235M parameters and 12 encoder layers. The majority of these 235M