Trading Inference-Time Compute for Adversarial Robustness
We conduct experiments on the impact of increasing inference-time compute in
reasoning models (specifically OpenAI o1-preview and o1-mini) on their
robustness to adversarial attacks. We find that across a variety of attacks,
increased inference-time compute leads to improved robustness. In many cases
(with important exceptions), the fraction of model samples where the attack
succeeds tends to zero as the amount of test-time compute grows. We perform no
adversarial training for the tasks we study, and we increase inference-time
compute by simply allowing the models to spend more compute on reasoning,
independently of the form of attack. Our results suggest that inference-time
compute has the potential to improve adversarial robustness for Large Language
Models. We also explore new attacks directed at reasoning models, as well as
settings where inference-time compute does not improve reliability, and
speculate on the reasons for these as well as ways to address them.
67a02c75221b701e4c04da7f
Wojciech Zaremba
67a02c75221b701e4c04da80
Evgenia Nitishinskaya
admin_assigned
2025-02-03T12:56:51.252Z
6792b8359967b7f195447e43
/avatars/0fcad468c7062d003902e78975daf6ea.svg
Evgenia Nitishinskaya
user
gadzin1203
67a02c75221b701e4c04da81
Boaz Barak
67a02c75221b701e4c04da82
Stephanie Lin
67a02c75221b701e4c04da83
Sam Toyer
admin_assigned
2025-02-03T12:57:23.000Z
62952a811e87ffbe5c06e3d4
/avatars/8140d3a6cbb0f85c284a1fd388915cb2.svg
Sam Toyer
user
qxcv
67a02c75221b701e4c04da84
Yaodong Yu
admin_assigned
2025-02-03T12:57:28.373Z
6100e69a393be1b5c4c83867
/avatars/1b87098cffb9c50345789808daea4f68.svg
Yaodong Yu
user
yaodongyu
67a02c75221b701e4c04da85
Rachel Dias
67a02c75221b701e4c04da86
Eric Wallace
admin_assigned
2025-02-03T12:57:48.651Z
63112d2431257261d20d5754
/avatars/502a68c0fb2f0c6989fe2869d0a7e3f4.svg
Eric Wallace
user
EricWallace
67a02c75221b701e4c04da87
Kai Xiao
67a02c75221b701e4c04da88
Johannes Heidecke
67a02c75221b701e4c04da89
Amelia Glaese
Trading Inference-Time Compute for Adversarial Robustness
We conduct experiments on the impact of increasing inference-time compute in
reasoning models (specifically OpenAI o1-preview and o1-mini) on their
robustness to adversarial attacks. We find that across a variety of attacks,
increased inference-time compute leads to improved robustness. In many cases
(with important exceptions), the fraction of model samples where the attack
succeeds tends to zero as the amount of test-time compute grows. We perform no
adversarial training for the tasks we study, and we increase inference-time
compute by simply allowing the models to spend more compute on reasoning,
independently of the form of attack. Our results suggest that inference-time
compute has the potential to improve adversarial robustness for Large Language
Models. We also explore new attacks directed at reasoning models, as well as
settings where inference-time compute does not improve reliability, and
speculate on the reasons for these as well as ways to address them.
Increasing inference-time compute enhances the robustness of reasoning models, such as OpenAI o1-preview and o1-mini, against adversarial attacks without adversarial training.
reasoning models OpenAI o1-preview o1-mini adversarial attacks inference-time compute adversarial robustness Large Language Models