In this paper, we present WiC-TSV (Target Sense Verification for Words in Context ) , a new multi-domain evaluation benchmark for  Word Sense Disambiguation (WSD) and Entity Linking (EL). Our benchmark is different from conventional WSD and EL benchmarks for it  being independent of a general sense inventory, making it highly flexible for the evaluation of a diverse set of models and systems in different domains. WiC-TSV is split into three tasks (systems get hypernymy or definitional or both hypernymy and definitional information about the target sense). Test data is available in four domains: general (WordNet), computer science, cocktails and medical concepts. Results show that existing state-of-the-art language models such as BERT can achieve a high performancein both in-domain data and out-of-domain data, but they still have room for improvement . WiC-TSV task data is available at URL/competitions/ 23683 .
We present WiC-TSV , a new multi-domain evaluation benchmark for Target Sense Verification of Words in Context. The benchmark is a new framework for evaluating Word Sense Disambiguation and Entity Linking , which is different from conventional benchmarks for it covering various domains, while being independent of external sense inventories. This makes the dataset highly flexible for the evaluation of a diverse set of models and systems in and across domains. WiC-TSV provides three different evaluation settings, depending on the input signals provided to the model. We set baseline performance on the dataset using state-of-the-art language models. Experimental results show that even though these models can perform decently on the task, there remains a gap between machine and human performance, especially in out-of-domain settings . WiC-TSV data is available at URL/competitions/ 23683.
R
In this paper, we
We
D
(
D
Target Sense Verification for Words in Context
D
)
A
Target Sense Verification of Words in Context. The benchmark is a new framework for evaluating
D
(WSD)
R
(EL). Our benchmark
, which
D
WSD and EL
A
covering various domains, while
R
a general sense inventory, making it
external sense inventories. This makes the dataset
R
different
and across
R
is split into three tasks (systems get hypernymy or definitional or both hypernymy and definitional information about the target sense). Test data is available in four domains: general (WordNet), computer science, cocktails and medical concepts. Results show that existing
provides three different evaluation settings, depending on the input signals provided to the model. We set baseline performance on the dataset using
R
language models such as BERT can achieve a high performancein both in-domain data and
language models. Experimental results show that even though these models can perform decently on the task, there remains a gap between machine and human performance, especially in
R
data, but they still have room for improvement
settings
D
task
D
23683
R
.
23683.