Hey!
I had such a blast at ACL2020 this week! So many cool works, and lots of very interesting discussions both in the chat and in the zoom Q&A sessions!
Here’s a pick of 3 of my highlights (there are extremely biased towards what I’m currently interested in):
(1) Inherent Disagreements in Human Textual Inferences 8
by Ellie Pavlick, Tom Kwiatkowski
Natural Language Inference (sometimes referred to as textual entailment) has become fundamental in evaluating language understanding and semantics. The central question of this paper is “what should we use as ground truth labels for textual inference?” The authors show that the apparent “annotation noise” often results from a multi-modality among the annotators’ labels. They discuss the implication of this uncertainty and argue for a refined evaluation that better captures the diversity of human judgments.
(2) Unsupervised Domain Clusters in Pretrained Language Models 9
by Roee Aharoni, Yoav Goldberg
The authors propose a “data-driven” approach to define what a domain is in NLP and to select in-domain data. They show that large pre-trained language models are able to capture these domains in an unsupervised way and leverage this insight to select in-domain data to train neural machine translation models.
(3) Syntactic Data Augmentation Increases Robustness to Inference Heuristics 14
by Junghyun Min, R. Thomas McCoy, Dipanjan Das, Emily Pitler, Tal Linzen
Natural Language Inference models fine-tuned on top of models like BERT show high accuracy on standard test datasets but fail on challenge sets. The authors propose a simple syntactic data augmentation procedure to augment the standard training set to up to a thousand examples. Results show great improvement (and generalization) by just exposing the model to these controlled syntactic examples supporting that hypothesis that BERT contains knowledge that simply needs to be “activated”. Cases failures (like passive) support that the idea there is also knowledge pre-trained BERT is not aware of.
How about you? Did any work change your perspective?