75Ben Garﬁnkel suggests the example of a robo-cop pinning down the wrong person, but remaining amenable
to human instruction otherwise. And in general, it seems most dangerous if an APS system is using its advanced
capabilities in pursuing power; e.g., if I’m a greater hacker, but a poor writer, and I’m trying to get power via my
journalism, I’m less threatening.

76Thanks to Ben Garﬁnkel for suggesting examples in this vein.
77Thanks to Ben Garﬁnkel, Rohin Shah, and Tom Davidson for emphasizing points in this vicinity.
78We can also imagine cases where the system’s tendency to rule out certain plans is best interpreted as part

of its objectives (e.g., such plans in fact aren’t good, on its values).
Is Power-Seeking AI an Existential Risk?