Machine Learning, NLP, Bert, Albert, Transformers.
matrix. 2. Cross-layer Parameter Sharing One of the ways to reduce the model’s parameters is to make them shareable. This means that they all share the same values. For the most part, it simply reduces the memory required to store weights. However, standard algorithms like backpropagation or