140This is a point made in Ord (2020), and by Bostrom in his TED talk.
141See e.g. Bostrom (2014), Chapter 4. Obviously, the “level” of AI capability available is highly multidimen-

sional; but how we deﬁne it doesn’t much matter at present.
142See the AI Impacts project on discontinuities for more.
143This could in principle proceed via non-artiﬁcial forms of intelligence, too; but I’m leaving that aside.
144Bostrom (2014)’s notion of a “hardware overhang” is an example of fast take-off not driven by feedback

loops.

39
Is Power-Seeking AI an Existential Risk?