[159] Tahaei, M.S., Charlaix, E., Nia, V.P., Ghodsi, A., Rezagholizadeh, M.: Kro-
neckerbert: Learning kronecker decomposition for pre-trained language models
via knowledge distillation. arXiv preprint arXiv:2109.06243 (2021)

[160] Edalati, A., Tahaei, M., Rashid, A., Nia, V.P., Clark, J.J., Rezagholizadeh, M.:
Kronecker decomposition for gpt compression. arXiv preprint arXiv:2110.08152
(2021)

[161] Reid, M., Marrese-Taylor, E., Matsuo, Y.: Subformer: Exploring weight
sharing for parameter eﬃciency in generative transformers. arXiv preprint
arXiv:2101.00234 (2021)

[162] Xu, M., Xu, Y.L., Mandic, D.P.: Tensorgpt: Eﬃcient compression of the embed-
ding layer in llms based on the tensor-train decomposition. arXiv preprint
arXiv:2307.00526 (2023)

[163] Teerapittayanon, S., McDanel, B., Kung, H.-T.: Branchynet: Fast inference via
early exiting from deep neural networks. In: 2016 23rd International Conference
on Pattern Recognition (ICPR), pp. 2464–2469 (2016). IEEE

53
Beyond Efficiency