Much work on technical alignment focuses on controlling the objectives the AI systems we build
pursue. I’ll start there.
At any given point in the progress of AI technology, we will have some particular level of control
in this respect, grounded in currently available techniques. Thus, for example, some methods of
AI development require coding objectives by hand. Others (more prominent at present) shape an
AI’s objectives via different sorts of training signals—whether generated algorithmically, or using
human feedback/demonstration. And we can imagine future methods that allows us to e.g. control an
AI’s objectives by editing the weights of a neural network directly; or to cause a system to pursue
(for its own sake) any objective that we can articulate using English-language sentences that will be
accurately, common-sensically, and charitably construed.
The challenge of (1) is to use what methods in this respect are available to ensure PS-alignment
Is Power-Seeking AI an Existential Risk?