PromptIntern: Saving Inference Costs by Internalizing Recurrent Prompt during Large Language Model Fine-tuning
<|reference_start|>PromptIntern: Saving Inference Costs by Internalizing Recurrent Prompt during Large Language Model Fine-tuning: Large language models (LLMs) have played a fundamental role in various natural language processing tasks with powerful prompt techniques. However, in real-world applications, there are often similar prompt components for repeated queries, which causes significant computational burdens during inference. Existing prompt compression and direct fine-tuning methods aim to tackle these challenges, yet they frequently struggle to strike an optimal balance between cost-efficiency and performance effectiveness, especially in complex tasks such as NL2Code. In this paper, we propose a novel method namely PromptIntern to internalize the prompt knowledge into model parameters via progressive fine-tuning. Our method enables LLMs to emulate the human learning process for a new task, where detailed templates and examples in a prompt are gradually internalized and phased out progressively as the model grows accustomed to the task. Extensive experiments demonstrate that our method reduces inference tokens over 90%, speedups inference by 4.2 times, and saves 88.3% monetary cost.<|reference_end|>
@article{zou2024promptintern:,
    title={PromptIntern: Saving Inference Costs by Internalizing Recurrent Prompt
  during Large Language Model Fine-tuning},
    author={Jiaru Zou, Mengyu Zhou, Tao Li, Shi Han, Dongmei Zhang},
    journal={arXiv preprint arXiv:2407.02211},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.02211},
    primaryClass={cs.CL cs.AI cs.LG}
}