_:n6921723ffdd64b259fcfb202929b506axb77
<http://schema.org/description>
"Commonsense knowledge is critical in human reading comprehension. While\nmachine comprehension has made significant progress in recent years, the\nability in handling commonsense knowledge remains limited. Synonyms are one of\nthe most widely used commonsense knowledge. Constructing adversarial dataset is\nan important approach to find weak points of machine comprehension models and\nsupport the design of solutions. To investigate machine comprehension models'\nability in handling the commonsense knowledge, we created a Question and Answer\nDataset with common knowledge of Synonyms (QADS). QADS are questions generated\nbased on SQuAD 2.0 by applying commonsense knowledge of synonyms. The synonyms\nare extracted from WordNet. Words often have multiple meanings and synonyms. We\nused an enhanced Lesk algorithm to perform word sense disambiguation to\nidentify synonyms for the context. ELECTRA achieves the state-of-art result on\nthe SQuAD 2.0 dataset in 2019. With scale, ELECTRA can achieve similar\nperformance as BERT does. However, QADS shows that ELECTRA has little ability\nto handle commonsense knowledge of synonyms. In our experiment, ELECTRA-small\ncan achieve 70\nnot perform much better. Its accuracy on SQuAD 2.0 is 88\nsignificantly to 26\nfailed badly on QADS, was not as bad as ELECTRA. The result shows that even\ntop-performing NLP models have little ability to handle commonsense knowledge\nwhich is essential in reading comprehension."
<https://deepai.org/profile/lizhen-cui>