  Neural language models (LMs) represent facts about the world described by
text. Sometimes these facts derive from training data (in most LMs, a
representation of the word "banana" encodes the fact that bananas are fruits).
Sometimes facts derive from input text itself (a representation of the sentence
"I poured out the bottle" encodes the fact that the bottle became empty). We
describe REMEDI, a method for learning to map statements in natural language to
fact encodings in an LM's internal representation system. REMEDI encodings can
be used as knowledge editors: when added to LM hidden representations, they
modify downstream generation to be consistent with new facts. REMEDI encodings
may also be used as probes: when compared to LM representations, they reveal
which properties LMs already attribute to mentioned entities, in some cases
making it possible to predict when LMs will generate outputs that conflict with
background knowledge or input text. REMEDI thus links work on probing,
prompting, and LM editing, and offers steps toward general tools for
fine-grained inspection and control of knowledge in LMs.

Computation and Language