Machine Learning, NLP, Bert, Albert, Transformers.
transformer layers into N groups of size M each where every group shares parameters within layers it has. Researchers found out that the smaller the group size M is, the better the results are. However, decreasing group size M leads to a significant increase in total parameters. 3. Sentence Order