Machine Learning, NLP, Bert, Albert, Transformers.
xxlarge processes a single iteration of data 3.3x slower than BERT large, experiments showed that if training both of these models for the same amount of time, then ALBERT xxlarge demonstrates a considerably better average performance on benchmarks than BERT large (88.7% vs 87.2%). Experiments