  Social media, as a major platform for communication and information exchange,
is a rich repository of the opinions and sentiments of 2.3 billion users about
a vast spectrum of topics. To sense the whys of certain social user's demands
and cultural-driven interests, however, the knowledge embedded in the 1.8
billion pictures which are uploaded daily in public profiles has just started
to be exploited since this process has been typically been text-based.
Following this trend on visual-based social analysis, we present a novel
methodology based on Deep Learning to build a combined image-and-text based
personality trait model, trained with images posted together with words found
highly correlated to specific personality traits. So the key contribution here
is to explore whether OCEAN personality trait modeling can be addressed based
on images, here called \emph{Mind{P}ics}, appearing with certain tags with
psychological insights. We found that there is a correlation between those
posted images and their accompanying texts, which can be successfully modeled
using deep neural networks for personality estimation. The experimental results
are consistent with previous cyber-psychology results based on texts or images.
In addition, classification results on some traits show that some patterns
emerge in the set of images corresponding to a specific text, in essence to
those representing an abstract concept. These results open new avenues of
research for further refining the proposed personality model under the
supervision of psychology experts.

Computers and Society;Computation and Language;Computer Vision and Pattern Recognition