a new strategy, to probabilistically perform tasks that would accelerate their replication, thus slipping through
the test environment some percentage of the time and continuing to accelerate their replication thereafter.” See
also Lewis (2017) for examples of possible deceptive dynamics learned in the context of bargaining: “Analysing
the performance of our agents, we ﬁnd evidence of sophisticated negotiation strategies. For example, we ﬁnd
instances of the model feigning interest in a valueless issue, so that it can later ‘compromise’ by conceding it.
Deceit is a complex skill that requires hypothesising the other agent’s beliefs, and is learnt relatively late in
child development (Talwar and Lee, 2002). Our agents have learnt to deceive without any explicit human design,
simply by trying to achieve their goals.. . . Deception can be an effective negotiation tactic. We found numerous
Is Power-Seeking AI an Existential Risk?