Phased Instruction Fine-Tuning for Large Language Models
<|reference_start|>Phased Instruction Fine-Tuning for Large Language Models: Instruction Fine-Tuning enhances pre-trained language models from basic next-word prediction to complex instruction-following. However, existing One-off Instruction Fine-Tuning (One-off IFT) method, applied on a diverse instruction, may not effectively boost models' adherence to instructions due to the simultaneous handling of varying instruction complexities. To improve this, Phased Instruction Fine-Tuning (Phased IFT) is proposed, based on the idea that learning to follow instructions is a gradual process. It assesses instruction difficulty using GPT-4, divides the instruction data into subsets of increasing difficulty, and uptrains the model sequentially on these subsets. Experiments with Llama-2 7B/13B/70B, Llama3 8/70B and Mistral-7B models using Alpaca data show that Phased IFT significantly outperforms One-off IFT, supporting the progressive alignment hypothesis and providing a simple and efficient way to enhance large language models. Codes and datasets from our experiments are freely available at https://github.com/xubuvd/PhasedSFT.<|reference_end|>
@article{pang2024phased,
    title={Phased Instruction Fine-Tuning for Large Language Models},
    author={Wei Pang and Chuan Zhou and Xiao-Hua Zhou and Xiaojie Wang},
    journal={arXiv preprint arXiv:2406.04371},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.04371},
    primaryClass={cs.CL cs.AI}
}