6.5 Sharing power

I’ve been assuming, here, that humans will, by default, be unwilling to let AI systems keep any power
they succeed in gaining via misaligned behavior. But especially in multipolar scenarios, we can also
imagine cases in which humans are either unable to correct a given type of misaligned power-seeking,
or unwilling to pay the costs required to do so.163
In such scenarios, it may be possible to reach various types of compromise arrangements, or to limit
the impact of a given PS-misaligned system or systems to some contained domain, such that humans
end up sharing power with practically misaligned AI systems, but not losing power entirely. These
are somewhat strange scenarios to imagine, and I won’t analyze them in any depth here.
I’ll note, though, that if we reach such a point, the situation has likely become quite dire; and we might
wonder, too, about its long-term stability. In particular, if the relevant PS-misaligned systems have
Is Power-Seeking AI an Existential Risk?