Commonsense reasoning requires a model to make presumptions about world events via language understanding. Many methods couple pre-trained language models with knowledge graphs in order to combine the merits in language modeling and entity-based relational learning . However, although a knowledge graph contains rich structural information, it lacks the context to provide a more precise understanding of the concepts and relations . This creates a gap when fusing knowledge graphs into language modeling, especially in the scenario of insufficient paired text-knowledge data. In this paper , we propose to utilize external entity description to provide contextual information for graph entities. For the CommonsenseQA task, our model first extracts concepts from the question and choice, and then finds a related triple between these concepts. Next, it retrieves the descriptions of these concepts from Wiktionary and feed them as additional input to a pre-trained language model, together with the triple . The resulting model can attain much more effective commonsense reasoning capability, achieving state-of-the-art results in the CommonsenseQA dataset with an accuracy of 80.7\% (single model) and 83.3\% (ensemble model) on the official leaderboard .
Commonsense question answering (QA) requires a model to grasp commonsense and factual knowledge to answer questions about world events . Many prior methods couple language modeling with knowledge graphs (KG) . However, although a KG contains rich structural information, it lacks the context to provide a more precise understanding of the concepts . This creates a gap when fusing knowledge graphs into language modeling, especially when there is insufficient labeled data. Thus , we propose to employ external entity descriptions to provide contextual information for knowledge understanding. We retrieve descriptions of related concepts from Wiktionary and feed them as additional input to pre-trained language models . The resulting model achieves state-of-the-art result in the CommonsenseQA dataset and the best result among non-generative models in OpenBookQA .
R
reasoning
question answering (QA)
R
make presumptions
grasp commonsense and factual knowledge to answer questions
R
via language understanding. Many methods couple pre-trained language models
. Many prior methods couple language modeling
R
in order to combine the merits in language modeling and entity-based relational learning
(KG)
R
knowledge graph
KG
D
and relations
R
in the scenario of insufficient paired text-knowledge data. In this paper
when there is insufficient labeled data. Thus
R
utilize external entity description
employ external entity descriptions
R
graph entities. For the CommonsenseQA task, our model first extracts concepts from the question and choice, and then finds a related triple between these concepts. Next, it retrieves the descriptions of these
knowledge understanding. We retrieve descriptions of related
D
a
R
model, together with the triple
models
R
can attain much more effective commonsense reasoning capability, achieving
achieves
R
results
result
R
with an accuracy of 80.7\% (single model) and 83.3\% (ensemble model) on the official leaderboard
and the best result among non-generative models in OpenBookQA