Additionally, we expand the comparative scope
of our analysis to encompass the performance of
the instruction-tuned T5 version (Flan-T5) in the
zero-shot adversarial setting. This aids us in in-
vestigating whether the effectiveness of instruc-
tion following contributes to the observed success.
Note that in the zero-shot adversarial setting, ac-
curately selecting the appropriate option (e.g., ’a’
or ’b’) requires the ability to follow instructions,
a competence that the non-instruction-tuned ver-
sion of T5 with its 770 million parameters has yet
to fully acquire through pre-training alone (as it
is not instruction-tuned). While our prior exper-
iments relied on non-instruction-tuned models to
handle the task of choosing the correct option letter,
this was possible due to their significantly larger
scale at 175 billion parameters. This comparison is
presented in Figure 7.

We find that there is a remarkable and substantial
overlap in the tasks where these models perform

19
AreEmergentAbilitiesinLarge Language Models just In-Context