Machine Learning, NLP, Bert, Albert, Transformers.
embedding size of 128. As was shown in the ablation studies in the paper, this is the optimal value. Increasing the embedding size, for example, up to 768, improves metrics but no more than 1% in absolute values which is not so much regarding the increasing complexity of the model. Though ALBERT