Abstract—With the continuous growth in the number of
parameters of transformer-based pretrained language models
(PLMs), particularly the emergence of large language models
(LLMs) with billions of parameters, many natural
language
processing (NLP) tasks have demonstrated remarkable success.
However, the enormous size and computational demands of
these models pose significant challenges for adapting them
to specific downstream tasks, especially in environments with
limited computational resources. Parameter Efficient Fine-Tuning
(PEFT) offers an effective solution by reducing the number
of fine-tuning parameters and memory usage while achieving
comparable performance to full fine-tuning. The demands for
fine-tuning PLMs, especially LLMs, have led to a surge in the
development of PEFT methods, as depicted in Fig. 1. In this
paper, we present a comprehensive and systematic review of
PEFT methods for PLMs. We summarize these PEFT methods,
Parameter-EfficientFine-TuningMethods