123I believe I heard this example from Ben Garﬁnkel.
124We might see GPT-3’s usefulness as bottlenecked in part by something like alignment, too. GPT-3’s outputs
suggest that it has lots of knowledge—for example, about basic medicine—that could be useful to users (see
Cotra (2021)). But extracting that knowledge is difﬁcult, partly because GPT-3 was trained, not to be useful to
humans, but to predict the next token in human-written text. And its outputs reﬂect biases that make it a less
attractive product to humans.

125I mean the term “testing” here to refer to something still under control of the developers, akin to the type of
“safety test” one might run on a car or a rocket. In this sense, testing is part of “training” more broadly. This
contrasts with other possible uses of a “train/test” distinction.
Is Power-Seeking AI an Existential Risk?