  Automatic recognition of emotion from facial expressions is an intense area
of research, with a potentially long list of important application. Yet, the
study of emotion requires knowing which facial expressions are used within and
across cultures in the wild, not in controlled lab conditions; but such studies
do not exist. Which and how many cross-cultural and cultural-specific facial
expressions do people commonly use? And, what affect variables does each
expression communicate to observers? If we are to design technology that
understands the emotion of users, we need answers to these two fundamental
questions. In this paper, we present the first large-scale study of the
production and visual perception of facial expressions of emotion in the wild.
We find that of the 16,384 possible facial configurations that people can
theoretically produce, only 35 are successfully used to transmit emotive
information across cultures, and only 8 within a smaller number of cultures.
Crucially, we find that visual analysis of cross-cultural expressions yields
consistent perception of emotion categories and valence, but not arousal. In
contrast, visual analysis of cultural-specific expressions yields consistent
perception of valence and arousal, but not of emotion categories. Additionally,
we find that the number of expressions used to communicate each emotion is also
different, e.g., 17 expressions transmit happiness, but only 1 is used to
convey disgust.

Computer Vision and Pattern Recognition