There exist many approaches [16], [35], [58], [59], [60],
[61], [62] to combine multiple PEFT methods, aiming to
leverage the distinctive advantages of each PEFT method and
achieve enhanced performance. Nevertheless, the exploration
has been limited to PEFT methods such as adapter, LoRA,
prefix-tuning, and BitFit, leaving room for further exploitation
by incorporating additional combinations of PEFT methods.
Moreover, while drawing inspiration from the NAS algorithm,
several PEFT methods [60], [61] have been investigated using
diverse optimization techniques to explore optimal neural
network architectures for configuring these PEFT methods.
There remains potential for continued exploration in utilizing
other optimization methods to automatically search for neural
network architectures and configure specific combinations of
PEFT modules at specific layers. Additionally, utilizing mul-
tiple PEFT methods typically results in increased parameter
Parameter-EfficientFine-TuningMethods