Machine Learning, NLP, Bert, Albert, Transformers.
In fact, there exist several ways of parameter sharing in Transformers across layers: share only attention parameters; share only forward neural network (FNN) parameters; share all parameters (used in ALBERT). Different parameter sharing strategies In general, it is possible to divide all