Humans can also be agentic—that is (loosely), we pursue objectives, guided by models of the world
(see section 2.1.2 for more). You want to travel to New York, so you buy a ﬂight, double-check when
it leaves, wake up early to pack, and so forth.
Non-human cognitive systems can be agentic, too. And depending on what you count as an agent,
humans can already create novel non-human agents.9 We breed and genetically modify non-humans
animals, for example; and some of our artiﬁcial systems display agent-like behavior in speciﬁc
environments (see e.g. here and here). But these agents can’t learn, plan, communicate, etc. like we
can.
At some point, though—absent catastrophe, deliberate choice, or other disruption of scientiﬁc
progress—we will likely be in a position, if we so choose, to create non-human agents whose
abilities in these respects rival or exceed our own. And in the context of artiﬁcial agents, the
Is Power-Seeking AI an Existential Risk?