Mimicking User Data: On Mitigating Fine-Tuning Risks in Closed Large Language Models
<|reference_start|>Mimicking User Data: On Mitigating Fine-Tuning Risks in Closed Large Language Models: Fine-tuning large language models on small, high-quality datasets can enhance their performance on specific downstream tasks. Recent research shows that fine-tuning on benign, instruction-following data can inadvertently undo the safety alignment process and increase a model's propensity to comply with harmful queries. Although critical, understanding and mitigating safety risks in well-defined tasks remains distinct from the instruction-following context due to structural differences in the data. Our work addresses the gap in our understanding of these risks across diverse types of data in closed models - where providers control how user data is utilized in the fine-tuning process. We demonstrate how malicious actors can subtly manipulate the structure of almost any task-specific dataset to foster significantly more dangerous model behaviors, while maintaining an appearance of innocuity and reasonable downstream task performance. To address this issue, we propose a novel mitigation strategy that mixes in safety data which mimics the task format and prompting style of the user data, showing this is more effective than existing baselines at re-establishing safety alignment while maintaining similar task performance.<|reference_end|>
@article{eiras2024mimicking,
    title={Mimicking User Data: On Mitigating Fine-Tuning Risks in Closed Large
  Language Models},
    author={Francisco Eiras, Aleksandar Petrov, Phillip H.S. Torr, M. Pawan Kumar,
  Adel Bibi},
    journal={arXiv preprint arXiv:2406.10288},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.10288},
    primaryClass={cs.CL cs.LG}
}