  Large Language Models (LLMs) are trained on vast amounts of text from the
internet, which contains both factual and misleading information about the
world. Can language models discern truth from falsehood in this contradicting
data? Expanding on the view that LLMs can model different communicative agents,
we present the persona hypothesis: LLMs can cluster agents into personas using
common features of their generations. For instance, a truthful persona is a
group of agents that are likely to produce truthful text and that share similar
features like formal writing styles and scientific references. By modeling this
persona, LLMs can generalize truthfulness beyond the specific contexts in which
each agent generated the training text. For example, the model can infer that
the agent "Wikipedia" will behave truthfully on topics that were only generated
by "Science" because they both belong to the truthful persona. We show evidence
for the persona hypothesis via two observations: (1) we can probe whether a
model's answer will be truthful before it is generated; (2) finetuning a model
on a set of facts improves its truthfulness on unseen topics. Next, using
arithmetics as a synthetic environment, we show that language models can
separate true and false statements, and generalize truthfulness across agents;
but only if agents in the training data share a truthful generative process
that enables the creation of a truthful persona. Overall, our findings suggest
that models can exploit hierarchical structures in the data to learn abstract
concepts like truthfulness.

Computation and Language;Artificial Intelligence;Machine Learning