88Though what sorts of misaligned power-seeking in particular these behaviors would give rise to is a further

question.

89Thus, paying railroad builders by the mile of track that they lay incentivizes them to lay unnecessary track
(from Wikipedia, here); trying to lower the cobra population by paying people to turn in dead cobras leads
to people breeding cobras (from Wikipedia, here); if teachers take “cause my students to get high scores on
standardized tests” as their objective, they’re incentivized to “teach to the test”—an incentive that can work to
the detriment of student education more broadly; and so on (see here for more examples). See Manheim and
Garrabrant (2018) for an abstract categorization of dynamics of this kind.

90Krakovna et al (2020) make this point well.
91This paragraph draws heavily on discussion in Ngo (2020).
92See e.g. Christiano et al (2017).

23
Is Power-Seeking AI an Existential Risk?