systems than to build misaligned (and relevantly powerful/agentic) AI systems that
are still superﬁcially attractive to deploy; (4) some such misaligned systems will
seek power over humans in high-impact ways; (5) this problem will scale to the
full disempowerment of humanity; and (6) such disempowerment will constitute
an existential catastrophe. I assign rough subjective credences to the premises in
this argument, and I end up with an overall estimate of ~5% that an existential
catastrophe of this kind will occur by 2070. (May 2022 update: since making this
report public in April 2021, my estimate here has gone up, and is now at >10%.)
Is Power-Seeking AI an Existential Risk?