The second change we make to our prompting
strategy involves the exploration of two types of
completion-style prompts: closed and open. In the
closed prompt format, we provide answer choices
alongside the prompt, while in the open prompt for-
mat, the answer choices are withheld. We find that
when models are prompted using the open prompt
strategy, their generated results often exhibit little
or no resemblance to the provided answer choices.
Consequently, evaluating the correctness of the gen-
erated answers becomes challenging. As a result,
experiments utilising the open prompt setting are
completely excluded from our analysis. However,
we provide access to these responses in the data ac-
companying this study, allowing other researchers

9

Prev. Emergent Competence type Memorisable (of 50)
AreEmergentAbilitiesinLarge Language Models just In-Context