Recently, many efforts have been spent on observing individual's psychological states through analyzing users' social activities on SNS. In this paper, we propose a novel method for identifying the users with depressive moods by analyzing their daily tweets for a long period of time. Then, for more accurately understand their tweets, we exploit all media types of tweets, i.e., images and emoticons as well as texts, thus develop a multimodal method for analyzing them. In the proposed method, three single-modal analyses are first performed for extract the hidden users' moods from text, emoticon, and images: a learning based text analysis, a word-based emoticon analysis, and a SVM based image classifier. Thereafter, the extracted moods from the respective analyses are integrated into a mood and again aggregated per a day, which allows for continuous monitoring of user's mood trends. To assess the validity of the proposed method, two types of experiments were performed: 1) the proposed multimodal analysis was tested with a number of tweets, and its performance was compared to SentiStrength; 2) it was applied to classify 45 users' mental states as depressive and non-depressive ones. Then, the results demonstrated that the proposed method outperforms the baseline, and it is effective in finding depressive moods for users.
['Keumhee Kang', 'C. G. Yoon', 'Eun Yi Kim']
Identifying depressive users in Twitter using multimodal analysis