  Ultra-fine entity typing (UFET) aims to predict a wide range of type phrases
that correctly describe the categories of a given entity mention in a sentence.
Most recent works infer each entity type independently, ignoring the
correlations between types, e.g., when an entity is inferred as a president, it
should also be a politician and a leader. To this end, we use an undirected
graphical model called pairwise conditional random field (PCRF) to formulate
the UFET problem, in which the type variables are not only unarily influenced
by the input but also pairwisely relate to all the other type variables. We use
various modern backbones for entity typing to compute unary potentials, and
derive pairwise potentials from type phrase representations that both capture
prior semantic information and facilitate accelerated inference. We use
mean-field variational inference for efficient type inference on very large
type sets and unfold it as a neural network module to enable end-to-end
training. Experiments on UFET show that the Neural-PCRF consistently
outperforms its backbones with little cost and results in a competitive
performance against cross-encoder based SOTA while being thousands of times
faster. We also find Neural- PCRF effective on a widely used fine-grained
entity typing dataset with a smaller type set. We pack Neural-PCRF as a network
module that can be plugged onto multi-label type classifiers with ease and
release it in https://github.com/modelscope/adaseq/tree/master/examples/NPCRF.

Computation and Language