Can LLMs Follow Simple Rules?
As Large Language Models (LLMs) are deployed with increasing real-world
responsibilities, it is important to be able to specify and constrain the
behavior of these systems in a reliable manner. Model developers may wish to
set explicit rules for the model, such as "do not generate abusive content",
but these may be circumvented by jailbreaking techniques. Evaluating how well
LLMs follow developer-provided rules in the face of adversarial inputs
typically requires manual review, which slows down monitoring and methods
development. To address this issue, we propose Rule-following Language
Evaluation Scenarios (RuLES), a programmatic framework for measuring
rule-following ability in LLMs. RuLES consists of 15 simple text scenarios in
which the model is instructed to obey a set of rules in natural language while
interacting with the human user. Each scenario has a concise evaluation program
to determine whether the model has broken any rules in a conversation. Through
manual exploration of model behavior in our scenarios, we identify 6 categories
of attack strategies and collect two suites of test cases: one consisting of
unique conversations from manual testing and one that systematically implements
strategies from the 6 categories. Across various popular proprietary and open
models such as GPT-4 and Llama 2, we find that all models are susceptible to a
wide variety of adversarial hand-crafted user inputs, though GPT-4 is the
best-performing model. Additionally, we evaluate open models under
gradient-based attacks and find significant vulnerabilities. We propose RuLES
as a challenging new setting for research into exploring and defending against
both manual and automatic attacks on LLMs.
654c3dc517d83697c754441c
Norman Mu
admin_assigned
2023-11-09T09:51:43.994Z
60b08cd39600ed1aafdb2b96
https://cdn-avatars.huggingface.co/v1/production/uploads/60b08cd39600ed1aafdb2b96/MeXWoApvQrUGYAbYWVVSr.jpeg
Norman Mu
user
normster
654c3dc517d83697c754441d
Sarah Chen
654c3dc517d83697c754441e
Zifan Wang
claimed_verified
2024-10-21T20:02:40.367Z
66976d1007b36ccd01586ce5
/avatars/5811e350907a29b71f6e4d57ffd53e66.svg
Wang
user
ZifanScale
654c3dc517d83697c754441f
Sizhe Chen
claimed_verified
2023-11-09T18:11:46.123Z
654d11580351be57074866e6
/avatars/1014ef4048f865c6d9fe8d39e5354805.svg
Sizhe Chen
user
Sizhe-Chen
654c3dc517d83697c7544420
David Karamardian
admin_assigned
2023-11-09T09:52:52.089Z
64d5844abb9b5ddddc58f13c
/avatars/8aa8f03764511d5c1b68046e2c754839.svg
David Karamardian
user
dkaramardian
654c3dc517d83697c7544421
Lulwa Aljeraisy
654c3dc517d83697c7544422
Dan Hendrycks
admin_assigned
2023-11-09T09:53:03.421Z
63f55aa1b51da4d61da9c96b
/avatars/56cf9c2d8295c4549248d3b0a4933043.svg
Dan Hendrycks
user
hendrycks
654c3dc517d83697c7544423
David Wagner
Can LLMs Follow Simple Rules?
As Large Language Models (LLMs) are deployed with increasing real-world
responsibilities, it is important to be able to specify and constrain the
behavior of these systems in a reliable manner. Model developers may wish to
set explicit rules for the model, such as "do not generate abusive content",
but these may be circumvented by jailbreaking techniques. Evaluating how well
LLMs follow developer-provided rules in the face of adversarial inputs
typically requires manual review, which slows down monitoring and methods
development. To address this issue, we propose Rule-following Language
Evaluation Scenarios (RuLES), a programmatic framework for measuring
rule-following ability in LLMs. RuLES consists of 15 simple text scenarios in
which the model is instructed to obey a set of rules in natural language while
interacting with the human user. Each scenario has a concise evaluation program
to determine whether the model has broken any rules in a conversation. Through
manual exploration of model behavior in our scenarios, we identify 6 categories
of attack strategies and collect two suites of test cases: one consisting of
unique conversations from manual testing and one that systematically implements
strategies from the 6 categories. Across various popular proprietary and open
models such as GPT-4 and Llama 2, we find that all models are susceptible to a
wide variety of adversarial hand-crafted user inputs, though GPT-4 is the
best-performing model. Additionally, we evaluate open models under
gradient-based attacks and find significant vulnerabilities. We propose RuLES
as a challenging new setting for research into exploring and defending against
both manual and automatic attacks on LLMs.
RuLES is a framework for evaluating rule-following behaviors in LLMs using programmatically defined scenarios and adversarial inputs, revealing vulnerabilities across different models and attack strategies.
Large Language Models LLMs Rule-following Language Evaluation Scenarios RuLES adversarial inputs attack strategies gradient-based attacks