Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together
<|reference_start|>Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together: Natural Language Processing (NLP) systems are increasingly taking the form of sophisticated modular pipelines, e.g., Retrieval Augmented Generation (RAG), where each module may involve a distinct Language Model (LM) and an associated prompt template. These compound systems often lack intermediate labels or gradient flow to optimize each module, making their end-to-end optimization challenging. Here we seek strategies to optimize both the module-level LM weights and the associated prompt templates of such systems to maximize a downstream task metric. We propose for the first time combining the weight and prompt optimization strategies to optimize a modular LM pipeline by alternating between the two to get the same LM to teach itself. In experiments with multi-hop QA, mathematical reasoning, and feature-based classification using mistral-7b, llama-2-7b, and llama-3-8b, these BetterTogether strategies optimizing the weights and prompts of a pipeline together outperform directly optimizing weights alone and prompts alone by up to 60% and 6%, respectively, on average across LMs and tasks. BetterTogether optimizer is released in DSPy at http://dspy.ai<|reference_end|>
@article{soylu2024fine-tuning,
    title={Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better
  Together},
    author={Dilara Soylu, Christopher Potts, Omar Khattab},
    journal={arXiv preprint arXiv:2407.10930},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.10930},
    primaryClass={cs.CL cs.AI cs.LG}
}