present a comprehensive taxonomy scheme in Section III. By
categorizing PEFT methods into additive fine-tuning, partial
fine-tuning, reparameterized fine-tuning, hybrid fine-tuning,
and unified fine-tuning, we establish a structured framework
for understanding these PEFT approaches, as depicted in
Fig. 2. In Section IV, we conduct quantitative investigations
and analyses to assess the performance, parameters efficiency,
and memory usage of these PEFT approaches. Our quantitative
studies primarily focus on natural
language understanding
(NLU), machine translation (MT), and natural language gen-
eration (NLG) tasks. Additionally, we extensively explore

the applications of PEFT in multi-task learning, cross-lingual
transfer, and backdoor attack and defense, underscoring its
effectiveness. Furthermore, our research also unveils potential
directions for future investigations in this rapidly evolving
field. To summarize, the main contributions of this survey can
be outlined as follows:
Parameter-EfficientFine-TuningMethods