What languages do they evaluate their methods on?
Mandarin Chinese (see table 3)
Rumour detection is hard because the most accurate systems operate retrospectively, only recognising rumours once they have collected repeated signals. By then the rumours might have already spread and caused harm. We introduce a new category of features based on novelty, tailored to detect rumours early on. To compensate for the absence of repeated signals, we make use of news wire as an additional data source. Unconfirmed (novel) information with respect to the news articles is considered as an indication of rumours. Additionally we introduce pseudo feedback, which assumes that documents that are similar to previous rumours, are more likely to also be a rumour. Comparison with other real-time approaches shows that novelty based features in conjunction with pseudo feedback perform significantly better, when detecting rumours instantly after their publication.
Social Media has evolved from friendship based networks to become a major source for the consumption of news (NIST, 2008). On social media, news is decentralised as it provides everyone the means to efficiently report and spread information. In contrast to traditional news wire, information on social media is spread without intensive investigation, fact and background checking. The combination of ease and fast pace of sharing information provides a fertile breeding ground for rumours, false- and disinformation. Social media users tend to share controversial information in-order to verify it, while asking about for the opinions of their followers (Zhao et. al, 2015). This further amplifies the pace of a rumour's spread and reach. Rumours and deliberate disinformation have already caused panic and influenced public opinion. The cases in Germany and Austria in 2016, show how misleading and false information about crimes committed by refugees negatively influenced the opinion of citizens. Detecting these rumours allows debunking them to prevent them from further spreading and causing harm. The further a rumour has spread, the more likely it is to be debunked by users or traditional media (Liu et. al, 2015). However, by then rumours might have already caused harm. This highlights the importance and necessity of recognizing rumours as early as possible - preferably instantaneously. Rumour detection on social media is challenging due to the short texts, creative lexical variations and high volume of the streams. The task becomes even harder if we attempt to perform rumour detection on-the-fly, without looking into the future. We provide an effective and highly scalable approach to detect rumours instantly after they were posted with zero delay. We introduce a new features category called novelty based features. Novelty based features compensate the absence of repeated information by consulting additional data sources - news wire articles. We hypothesize that information not confirmed by official news is an indication of rumours. Additionally we introduce pseudo feedback for classification. In a nutshell, documents that are similar to previously detected rumours are considered to be more likely to also be a rumour. The proposed features can be computed in constant time and space allowing us to process high-volume streams in real-time (Muthukrishnan, 2005). Our experiments reveal that novelty based features and pseudo feedback significantly increases detection performance for early rumour detection. The contributions of this paper include: Novelty based Features We introduced a new category of features for instant rumour detection that harnesses trusted resources. Unconfirmed (novel) information with respect to trusted resources is considered as an indication of rumours. Pseudo Feedback for Detection/Classification Pseudo feedback increases detection accuracy by harnessing repeated signals, without the need of retrospective operation.