pretrained parameters, preserving the knowledge captured by
the PLM while adapting it to the target task and reducing
the risk of catastrophic forgetting. Furthermore, since the size
of the fine-tuned dataset is typically much smaller than the
pretrained dataset, performing full fine-tuning to update all
the pretrained parameters may lead to overfitting, which is
circumvented by the PEFT through selectively or not updating
pretrained parameters.
Parameter-EfficientFine-TuningMethods