[25] X. Liu, Y. Zheng, Z. Du, M. Ding, Y. Qian, Z. Yang, and J. Tang, “Gpt

understands, too,” arXiv preprint arXiv:2103.10385, 2021.

[26] T. Vu, B. Lester, N. Constant, R. Al-Rfou’, and D. Cer, “SPoT: Better
frozen model adaptation through soft prompt transfer,” in Proc. Annu.
Meeting Assoc. Comput. Linguistics, 2022, pp. 5039–5059.

[27] A. Asai, M. Salehi, M. Peters, and H. Hajishirzi, “ATTEMPT:
Parameter-efficient multi-task tuning via attentional mixtures of soft
prompts,” in Proc. Conf. Empir. Methods Natural Lang. Process., 2022,
pp. 6655–6672.

[28] Z. Wang, R. Panda, L. Karlinsky, R. Feris, H. Sun, and Y. Kim,
“Multitask prompt tuning enables parameter-efficient transfer learning,”
in Proc. Int. Conf. Learn. Representations, 2023.

[29] Y.-L. Sung, J. Cho, and M. Bansal, “LST: Ladder side-tuning for
parameter and memory efficient transfer learning,” in Proc. Adv. Neural
Inf. Process. Syst., 2022.

18
Parameter-EfficientFine-TuningMethods