17.7
12.8
16.3
19.3
17.0
15.7

TQA
30.9
24.2
30.6
33.2
30.1
33.9

Table 6: Comparison of learned representations to knowledge graph embeddings (TransE) and pre-trained repre-
sentations of entity descriptions and contexts (Deep-Ed). All experiments use same 200k entities.

from three different sources and two different train-
ing scenarios. The initial embedddings are either
the TransE-Wikidata embeddings used by ERNIE
(Bordes et al., 2013); the Deep-Ed embeddings
used by KNOWBERT (Ganea and Hofmann, 2017);
or the random embeddings used by EAE. The em-
beddings are either frozen, following ERNIE and
KNOWBERT, or trained along with all other net-
work parameters.8 Along with the knowledge prob-
ing tasks from Section 5, we report performance
on the on the 9-way entity typing task from Choi
et al. 2018 (Appendix D).
Entities as Experts- Sparse Memory Access with Entity Supervision