Machine Learning, NLP, Bert, Albert, Transformers.
during forward propagation or backpropagation will be the same. This gives more opportunities for designing an efficient computation framework. The mentioned idea is implemented in ALBERT which consists of a set of Transformer blocks with the same structure making parameter sharing more efficient.