system plans strategically in order to direct electricity only to town A, but any plans that involve
unintended power-seeking would score very low on its criteria for choice. The ease with which we
can create systems of this type seems closely related to the ease with which we can create systems
with non-problematic objectives more generally (again, see section 4.3.1 for discussion).
Let’s look at two other objections to instrumental convergence.
The ﬁrst is that humans don’t always seem particularly “power-seeking.”81 Humans care a lot about
survival, and about certain resources (food, shelter, etc), but beyond that, we associate many forms
of power-seeking with a certain kind of greed, ambition, or voraciousness, and with intuitively
“resource-hungry” goals, like “maximize X across all of space and time.” Some strategically-aware
Is Power-Seeking AI an Existential Risk?