Exploring Scaling Trends in LLM Robustness
<|reference_start|>Exploring Scaling Trends in LLM Robustness: Language model capabilities predictably improve from scaling a model's size and training data. Motivated by this, increasingly large language models have been trained, yielding an array of impressive capabilities. Yet these models are vulnerable to adversarial prompts, such as "jailbreaks" that hijack models to perform undesired behaviors, posing a significant risk of misuse. Prior work indicates that computer vision models become more robust with model and data scaling, raising the question: does language model robustness also improve with scale? We study this question empirically, finding that larger models respond substantially better to adversarial training, but there is little to no benefit from model scale in the absence of explicit defenses.<|reference_end|>
@article{howe2024effects,
    title={Effects of Scale on Language Model Robustness},
    author={Nikolaus Howe, Ian McKenzie, Oskar Hollinsworth, Micha{l} Zajac, Tom
  Tseng, Aaron Tucker, Pierre-Luc Bacon, Adam Gleave},
    journal={arXiv preprint arXiv:2407.18213},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.18213},
    primaryClass={cs.LG cs.AI cs.CL cs.CR}
}