Machine Learning, NLP, Bert, Albert, Transformers.
dimensionality of embeddings decreases from H to E. Nevertheless, in the case of decomposed matrices, to obtain the input for BERT, the mapped embeddings need then to be projected into hidden BERT space: this is done by multiplying a corresponding row of the left matrix by columns of the right