Which psychological features are used?
summary variable - analytically thinking, clout, tone, authentic, Big five variable - openness, conscientiousness, extraversion, agreeableness, neuroticism, Emotional variables - positive emotions in the text, negative emotions in the text, personal drives - power, reward, risk, achievement, affiliation, personal pronouns -  counts the number of 1st, 2nd, and 3rd personal pronouns used, Minkowski distance between each profile and average values of these features created from the ISIS magazines
The Internet and, in particular, Online Social Networks have changed the way that terrorist and extremist groups can influence and radicalise individuals. Recent reports show that the mode of operation of these groups starts by exposing a wide audience to extremist material online, before migrating them to less open online platforms for further radicalization. Thus, identifying radical content online is crucial to limit the reach and spread of the extremist narrative. In this paper, our aim is to identify measures to automatically detect radical content in social media. We identify several signals, including textual, psychological and behavioural, that together allow for the classification of radical messages. Our contribution is three-fold: (1) we analyze propaganda material published by extremist groups and create a contextual text-based model of radical content, (2) we build a model of psychological properties inferred from these material, and (3) we evaluate these models on Twitter to determine the extent to which it is possible to automatically identify online radical tweets. Our results show that radical users do exhibit distinguishable textual, psychological, and behavioural properties. We find that the psychological properties are among the most distinguishing features. Additionally, our results show that textual models using vector embedding features significantly improves the detection over TF-IDF features. We validate our approach on two experiments achieving high accuracy. Our findings can be utilized as signals for detecting online radicalization activities.
The rise of Online Social Networks (OSN) has facilitated a wide application of its data as sensors for information to solve different problems. For example, Twitter data has been used for predicting election results, detecting the spread of flu epidemics, and a source for finding eye-witnesses during criminal incidents and crises BIBREF0 , BIBREF1 . This phenomenon is possible due to the great overlap between our online and offline worlds. Such seamless shift between both worlds has also affected the modus operandi of cyber-criminals and extremist groups BIBREF2 . They have benefited tremendously from the Internet and OSN platforms as it provides them with opportunities to spread their propaganda, widen their reach for victims, and facilitate potential recruitment opportunities. For instance, recent studies show that the Internet and social media played an important role in the increased amount of violent, right-wing extremism BIBREF3 . Similarly, radical groups such as Al-Qaeda and ISIS have used social media to spread their propaganda and promoted their digital magazine, which inspired the Boston Marathon bombers in 2010 BIBREF4 . To limit the reach of cyber-terrorists, several private and governmental organizations are policing online content and utilising big data technologies to minimize the damage and counter the spread of such information. For example, the UK launched a Counter Terrorism Internet Referral Unit in 2010 aiming to remove unlawful Internet content and it supports the police in investigating terrorist and radicalizing activities online. The Unit reports that among the most frequently referred links were those coming from several OSNs, such as Facebook and Twitter BIBREF2 . Similarly, several OSNs are constantly working on detecting and removing users promoting extremist content. In 2018, Twitter announced that over INLINEFORM0 million accounts were suspended for terrorist content BIBREF5 . Realizing the danger of violent extremism and radicalization and how it is becoming a major challenge to societies worldwide, many researchers have attempted to study the behaviour of pro-extremist users online. Looking at existing literature, we find that a number of existing studies incorporate methods to identify distinguishing properties that can aid in automatic detection of these users BIBREF6 , BIBREF7 . However, many of them depend on performing a keyword-based textual analysis which, if used alone, may have several shortcomings, such as producing a large number of false positives and having a high dependency on the data being studied. In addition, it can be evaded using automated tools to adjust the writing style. Another angle for analyzing written text is by looking at the psychological properties that can be inferred regarding their authors. This is typically called psycholinguistics, where one examines how the use of the language can be indicative of different psychological states. Examples of such psychological properties include introversion, extroversion, sensitivity, and emotions. One of the tools that automates the process of extracting psychological meaning from text is the Linguistic Inquiry and Word Count (LIWC) BIBREF8 tool. This approach has been used in the literature to study the behaviour of different groups and to predict their psychological states, such as predicting depression BIBREF9 . More recently, it has also been applied to uncover different psychological properties of extremist groups and understand their intentions behind the recruitment campaigns BIBREF10 . Building on the findings of previous research efforts, this paper aims to study the effects of using new textual and psycholinguistic signals to detect extremist content online. These signals are developed based on insights gathered from analyzing propaganda material published by known extremist groups. In this study, we focus mainly on the ISIS group as they are one of the leading terrorist groups that utilise social media to share their propaganda and recruit individuals. We analyze the propaganda material they publish in their online English magazine called Dabiq, and use data-mining techniques to computationally uncover contextual text and psychological properties associated with these groups. From our analysis of these texts, we are able to extract a set of signals that provide some insight into the mindset of the radical group. This allows us to create a general radical profile that we apply as a signal to detect pro-ISIS supporters on Twitter. Our results show that these identified signals are indeed critical to help improve existing efforts to detect online radicalization.