Methods We use the Know Your Data tool and classiﬁers similar to those available in Google Cloud’s documentation
on Classifying Content. These analyses are run on the data in optimized formats used to train the model, so we report
relative frequencies within those optimized formats. For example, English pronouns are found in 66% of optimized
English training data documents, and within that subset, there are female pronouns in 48% of those documents. And
while terms for “Americans” are found in 48% of the subset of all documents mentioning race, religious, ethnicity and
nationality terms, this subset is only 29% of all pre-training documents. In most cases, we only report values that make
up more than 5% of a subset of documents, focusing the analysis on major differences or patterns. We also note that
single-digit percentages of training data is still a signiﬁcant number of tokens.

62
PaLM 2 Technical Report