51Here I’m drawing on the framework used by Christiano (2020).
52Following the framework in Critch and Krueger (2020), we can also draw additional distinctions, related
to the number of human stake-holders and AI systems involved. I’m focused here on what that framework
would call “single-single” and “multi-single” delegation: that is, the project of aligning AI behavior with the
intentions of some human designer or set of designers. Critch argues (e.g., here) that multi-multi delegation
should be our focus. I disagree with this (indeed, if we solve single-single alignment, I personally would be
feeling substantially better about the situation), but I won’t argue the point here.
Is Power-Seeking AI an Existential Risk?