Once some actors can create APS systems, then over time, and absent active efforts to the contrary, a
larger and larger number of actors around the world will likely become able to do so as well. Absent
signiﬁcant coordination among such actors, then, it seems likely that we will see substantial variation
in the beliefs, values, and incentives that inform their decision-making.
Of course, safety-relevant properties of AI systems can be correlated, even without active coordination
amongst relevant actors. The ease of ensuring practical PS-alignment, for example, represents one
source of correlation, and there may be others (see Hubinger (2020) for discussion).
But to the extent that such correlation does not ensure practical PS-alignment by default (such that
e.g. some substantive level of caution and social responsibility is required), then larger numbers of
relevant actors increase the risk that some sort of failure will occur. That is: even if many actors
Is Power-Seeking AI an Existential Risk?