4.3.2 Controlling capabilities

AI alignment research typically focuses on controlling a system’s objectives. But controlling its
capabilities can play a role in practical PS-alignment, too.
In particular: the less capable a system, the more easily its behavior—including its tendencies to
misaligned power-seeking—can be anticipated and corrected. Less capable systems will also have a
harder time getting and keeping power, and a harder time making use of it, so they will have stronger
incentives to cooperate with humans (rather than trying to e.g. deceive or overpower them), and to
make do with the power and opportunities that humans provide them by default.
Preventing agentic planning and strategic awareness in the ﬁrst place would be one example of
“controlling capabilities” (see section 3); but there are other options, too.
Is Power-Seeking AI an Existential Risk?