AlpaGasus: Training A Better Alpaca with Fewer Data
Large language models~(LLMs) obtain instruction-following capability through
instruction-finetuning (IFT) on supervised instruction/response data. However,
widely used IFT datasets (e.g., Alpaca's 52k data) surprisingly contain many
low-quality instances with incorrect or irrelevant responses, which are
misleading and detrimental to IFT. In this paper, we propose a simple and
effective data selection strategy that automatically identifies and removes
low-quality data using a strong LLM (e.g., ChatGPT). To this end, we introduce
AlpaGasus, which is finetuned on only 9k high-quality data filtered from the
52k Alpaca data. AlpaGasus significantly outperforms the original Alpaca as
evaluated by GPT-4 on multiple test sets and its 13B variant matches >90%
performance of its teacher LLM (i.e., Text-Davinci-003) on test tasks. It also
provides 5.7x faster training, reducing the training time for a 7B variant from
80 minutes (for Alpaca) to 14 minutes We apply IFT for the same
number of epochs as Alpaca(7B) but on fewer data, using 4timesNVIDIA A100
(80GB) GPUs and following the original Alpaca setting and hyperparameters..
Overall, AlpaGasus demonstrates a novel data-centric IFT paradigm that can be
generally applied to instruction-tuning data, leading to faster training and
better instruction-following models. Our project page is available at:
https://lichang-chen.github.io/AlpaGasus/.
64b5ef9160274cbb29736e1f
Lichang Chen
admin_assigned
2023-07-18T07:47:18.473Z
62323bb408bcea92917e42ee
https://cdn-avatars.huggingface.co/v1/production/uploads/62323bb408bcea92917e42ee/2vHxkv-oSROtLteOnqa8P.jpeg
Lichang Chen
user
Lichang-Chen
64b5ef9160274cbb29736e20
Shiyang Li
64b5ef9160274cbb29736e21
Jun Yan
claimed_verified
2023-07-19T06:28:08.466Z
63a2137ef30c464227820e36
/avatars/d400d3dbf7f7233bb85d507a34b0be02.svg
Jun Yan
user
jun-yan
64b5ef9160274cbb29736e22
Hai Wang
claimed_verified
2023-07-19T06:28:12.656Z
6436f6d83d818691cf25f683
/avatars/acc08ff56095d53a36de5ae7cad4b718.svg
Hai Wang
user
sohurenxefr
64b5ef9160274cbb29736e23
Kalpa Gunaratna
64b5ef9160274cbb29736e24
Vikas Yadav
64b5ef9160274cbb29736e25
Zheng Tang
64b5ef9160274cbb29736e26
Vijay Srinivasan
claimed_verified
2023-07-21T19:25:07.179Z
6434e1c4546e16f17a1958f4
/avatars/d14600ca6f29010648ec98687b9b2bc0.svg
Vijay Srinivasan
user
svijay17
64b5ef9160274cbb29736e27
Tianyi Zhou
claimed_verified
2023-07-18T06:43:50.735Z
647f5af5b0e96764589f3b2a
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VJ4cDyjp5M3V5WmI5gPIU.jpeg
Tianyi Zhou
user
zhoutianyi
64b5ef9160274cbb29736e28
Heng Huang
64b5ef9160274cbb29736e29
Hongxia Jin
AlpaGasus: Training A Better Alpaca with Fewer Data
Large language models~(LLMs) obtain instruction-following capability through
instruction-finetuning (IFT) on supervised instruction/response data. However,
widely used IFT datasets (e.g., Alpaca's 52k data) surprisingly contain many
low-quality instances with incorrect or irrelevant responses, which are
misleading and detrimental to IFT. In this paper, we propose a simple and
effective data selection strategy that automatically identifies and removes
low-quality data using a strong LLM (e.g., ChatGPT). To this end, we introduce
AlpaGasus, which is finetuned on only 9k high-quality data filtered from the
52k Alpaca data. AlpaGasus significantly outperforms the original Alpaca as
evaluated by GPT-4 on multiple test sets and its 13B variant matches >90%
performance of its teacher LLM (i.e., Text-Davinci-003) on test tasks. It also
provides 5.7x faster training, reducing the training time for a 7B variant from
80 minutes (for Alpaca) to 14 minutes We apply IFT for the same
number of epochs as Alpaca(7B) but on fewer data, using 4timesNVIDIA A100
(80GB) GPUs and following the original Alpaca setting and hyperparameters..
Overall, AlpaGasus demonstrates a novel data-centric IFT paradigm that can be
generally applied to instruction-tuning data, leading to faster training and
better instruction-following models. Our project page is available at:
https://lichang-chen.github.io/AlpaGasus/.
A data selection strategy using a strong LLM improves instruction-following models by filtering high-quality data, enhancing performance and reducing training time.
instruction-finetuning LLMs data selection Alpaca AlpaGasus GPT-4 Text-Davinci-003 instruction-tuning