6.2 Manual Analysis
from
We randomly sampled 100 examples
unﬁltered-dev set, and analysed the ability of
EAE’s to correctly identify and link the question’s
entities. We ﬁnd that 87% of questions do not have
any incorrectly predicted entities.7

We ﬁnd that when there is at least one incor-
rectly linked entity, the performance of EAE is
considerably reduced. On this small sample, the
performance is even lower than for examples in
which there are no named entities in the question.
Table 4 illustrates three representative examples
from EAE and T5. In the ﬁrst example, the ques-
tion contains a date but no proper names. Since
EAE does not have a representation for dates in the
entity memory, this question is challenging and the
model predicts an incorrect answer of the correct
type (annual event). The second example demon-
strates EAE’s ability to model connections between
entities. In this case, ‘The Master’ only occurs 38
Entities as Experts- Sparse Memory Access with Entity Supervision