will be strongly inclined to use them, even if there are risks involved—and those who don’t use them
will end up losing their senate races, falling behind their business and military competitors, and so
forth.
And even if the chimps, at the beginning, are appropriately contained and incentivized to be genuinely
cooperative, it seems unsurprising if, as people draw on their capacities in more and more ways
around the world, they get exposed to opportunities and circumstances that incentivize them to seek
power for themselves, instead.
Something similar, I think, might apply to APS AI systems. Indeed, even if people know, or strongly
suspect, that such systems would seek power in misaligned ways in some not-out-of-the-question
circumstances, the pull towards using them for goals that matter a lot to us may simply be too great.
When pandemics are raging, oceans are rising, parents and grandparents are dying of cancer, rival
Is Power-Seeking AI an Existential Risk?