9.3. LLM developers have limited inﬂuence over what

is developed

Because many important LLM capabilities are emergent
and difﬁcult to predict, LLM developers have relatively lit-
tle inﬂuence on precisely what capabilities future LLMs
will have, and efforts to make predictions about future LLM
capabilities based on the economic incentives, values, or
personalities of their developers are likely to fail. GPT-4,
for example, appears to have many skills, like those involv-
ing programming, that its creators were likely hoping for.
However, it also appears to have initially shown several un-
wanted skills, like teaching laypeople to prepare biological
weapons, that its creators had to spend signiﬁcant effort to
try to remove (OpenAI, 2023b).
Beyond this, LLM developers inevitably also have limited
awareness of what capabilities an LLM has when they’re
deciding whether to deploy it: There is no known evaluation
or analysis procedure that can rule out surprises like chain-
Eight Things to Know about Large Language Models