Machine Learning, NLP, Bert, Albert, Transformers.
inference will still have to be executed on all parameters. One of the most optimal ways to share weights occurs when they are located in different but similar blocks of the model. Putting them into similar blocks results in a higher chance that most of the calculations for shareable parameters