Enhancing Low-Resource LLMs Classification with PEFT and Synthetic Data
<|reference_start|>Enhancing Low-Resource LLMs Classification with PEFT and Synthetic Data: Large Language Models (LLMs) operating in 0-shot or few-shot settings achieve competitive results in Text Classification tasks. In-Context Learning (ICL) typically achieves better accuracy than the 0-shot setting, but it pays in terms of efficiency, due to the longer input prompt. In this paper, we propose a strategy to make LLMs as efficient as 0-shot text classifiers, while getting comparable or better accuracy than ICL. Our solution targets the low resource setting, i.e., when only 4 examples per class are available. Using a single LLM and few-shot real data we perform a sequence of generation, filtering and Parameter-Efficient Fine-Tuning steps to create a robust and efficient classifier. Experimental results show that our approach leads to competitive results on multiple text classification datasets.<|reference_end|>
@article{patwa2024enhancing,
    title={Enhancing Low-Resource LLMs Classification with PEFT and Synthetic Data},
    author={Parth Patwa, Simone Filice, Zhiyu Chen, Giuseppe Castellucci, Oleg
  Rokhlenko, Shervin Malmasi},
    journal={arXiv preprint arXiv:2404.02422},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.02422},
    primaryClass={cs.CL cs.LG}
}