When the answer is an entity, in our entity vo-
cabulary, we link the answer string to an entity ID
using the SLING phrase table12 (Ringgaard et al.,
2017). If the answer is not an entity in our vocabu-
lary, we discard the question from the training set,
though we keep it in the development and test set
to ensure fair comparisons with prior work. Ta-
ble 8 shows the share of answers that were linked
using our procedure. This means that Oracle perfor-
mance for our model on the TriviaQA development
set is only 85%, which is due to non entity-answers
and entities not in our vocabulary.

11Nominal mentions in questions typically refer to the an-
swer entity. This is unlike Wikipedia text, where nominal
mentions refer to entities that have previously been named.
We only link proper name mentions in questions so that the
model is not forced to hallucinate links for entities that have
not been properly introduced to the discourse.

12https://github.com/google/sling
Entities as Experts- Sparse Memory Access with Entity Supervision