38

6 Correction

In many contexts, if an AI system starts seeking to gain/maintain power in unintended ways, the
behavior may well be noticed, and the system prevented from gaining/maintaining the power it seeks.
Let’s call this “correction.”
Some types of correction might be easy (e.g., a lab notices that an AI system tried to open a Bitcoin
wallet, and shuts it down). Others might be much more difﬁcult and costly (for example, an AI system
that has successfully hacked into and copied itself onto an unknown number of computers around the
world might be quite difﬁcult to eradicate).140
Confronted with post-deployment PS-alignment failures, will humanity’s corrective efforts be enough
to avert catastrophe? I think they might well; but it doesn’t seem guaranteed. Let’s look at some
considerations.

6.1 Take-off
Is Power-Seeking AI an Existential Risk?