3

• My emphasis and approach differs from that of others in the literature in various ways.3 In
particular: I’m less focused than some on the possibility of an extremely rapid escalation in
frontier AI capabilities, on “recursive self-improvement,” or on scenarios in which one actor
comes to dominate the world; I’m focusing on power-seeking in particular (as opposed to
“misalignment” more broadly); I’m aiming to incorporate and respond to various objections
and re-framings introduced since early work on the topic, and since machine learning has
become a prominent AI paradigm;4 I’m aiming to avoid reliance on models of “utility
function maximization” and related concepts; and I present and assign probabilities to
premises in a complete argument for catastrophe. That said, I still think of this as an
articulation and analysis of a certain kind of “core argument”—one that has animated, and
continues to animate, much concern about existential risk from misaligned AI.5
Is Power-Seeking AI an Existential Risk?