What was their state of the art accuracy score?
In 2-way classification precision score was 88% and recall 82%. In 4-way classification on LUN-dev F1-score was 91% and on LUN-test F1-score was 65%.
The rising growth of fake news and misleading information through online media outlets demands an automatic method for detecting such news articles. Of the few limited works which differentiate between trusted vs other types of news article (satire, propaganda, hoax), none of them model sentence interactions within a document. We observe an interesting pattern in the way sentences interact with each other across different kind of news articles. To capture this kind of information for long news articles, we propose a graph neural network-based model which does away with the need of feature engineering for fine grained fake news classification. Through experiments, we show that our proposed method beats strong neural baselines and achieves state-of-the-art accuracy on existing datasets. Moreover, we establish the generalizability of our model by evaluating its performance in out-of-domain scenarios. Code is available at this https URL
In today's day and age of social media, there are ample opportunities for fake news production, dissemination and consumption. BIBREF0 break down fake news into three categories, hoax, propaganda and satire. A hoax article typically tries to convince the reader about a cooked-up story while propaganda ones usually mislead the reader into believing a false political or social agenda. BIBREF1 defines a satirical article as the one which deliberately exposes real-world individuals, organisations and events to ridicule. Previous works BIBREF2, BIBREF0 rely on various linguistic and hand-crafted semantic features for differentiating between news articles. However, none of them try to model the interaction of sentences within the document. We observed a pattern in the way sentences cluster in different kind of news articles. Specifically, satirical articles had a more coherent story and thus all the sentences in the document seemed similar to each other. On the other hand, the trusted news articles were also coherent but the similarity between sentences from different parts of the document was not that strong, as depicted in Figure FIGREF1. We believe that the reason for such kind of behaviour is the presence of factual jumps across sections in a trusted document. In this work, we propose a graph neural network-based model to classify news articles while capturing the interaction of sentences across the document. We present a series of experiments on News Corpus with Varying Reliability dataset BIBREF0 and Satirical Legitimate News dataset BIBREF2. Our results demonstrate that the proposed model achieves state-of-the-art performance on these datasets and provides interesting insights. Experiments performed in out-of-domain settings establish the generalizability of our proposed method.