RIFF: Learning to Rephrase Inputs for Few-shot Fine-tuning of Language Models
<|reference_start|>RIFF: Learning to Rephrase Inputs for Few-shot Fine-tuning of Language Models: Pre-trained Language Models (PLMs) can be accurately fine-tuned for downstream text processing tasks. Recently, researchers have introduced several parameter-efficient fine-tuning methods that optimize input prompts or adjust a small number of model parameters (e.g LoRA). In this study, we explore the impact of altering the input text of the original task in conjunction with parameter-efficient fine-tuning methods. To most effectively rewrite the input text, we train a few-shot paraphrase model with a Maximum-Marginal Likelihood objective. Using six few-shot text classification datasets, we show that enriching data with paraphrases at train and test time enhances the performance beyond what can be achieved with parameter-efficient fine-tuning alone. The code used for our experiments can be found at https://github.com/SaeedNajafi/RIFF.<|reference_end|>
@article{najafi2024riff:,
    title={RIFF: Learning to Rephrase Inputs for Few-shot Fine-tuning of Language
  Models},
    author={Saeed Najafi and Alona Fyshe},
    journal={arXiv preprint arXiv:2403.02271},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.02271},
    primaryClass={cs.CL cs.LG}
}