Not all misaligned AI behavior seems relevant to existential risk. Consider, for example, an AI system
in charge of an electrical grid, whose designers intend it to send electricity to both town A and town
B, but whose objectives have problems that cause it, during particular sorts of storms, to only send
electricity to town A. This is misaligned behavior, and it may be quite harmful, but it poses no threat
to the entire future.
Rather, as I discussed in section 1.2.4, the type of misaligned AI behavior that I think creates the
most existential risk involves misaligned power-seeking in particular: that is, active efforts by an AI
system to gain and maintain power in ways that designers didn’t intend, arising from problems with
that system’s objectives. In the electrical grid case, the AI system hasn’t been described as trying to
gain power (for example, by trying to hack into more computing resources to better calculate how to
Is Power-Seeking AI an Existential Risk?