Visual content is playing an increasingly important role in social media, due to its effectiveness in attracting attention and conveying emotions. However, it remains difficult to measure the emotions expressed in images and predict the viewer responses likely to be evoked. Results of such studies will be useful for applications such as community sentiment tracking, advertising, and publishing. In this talk, we present a systematic approach to this problem. We first apply web mining to discover the visual content (objects, scenes, and their attributes) frequently used in expressing various types of emotions in social media. We then apply machine learning techniques to develop a large detector library, called SentiBank, that recognizes 1,200 sentiment-related visual concepts contained in images. We show how such analytics tools may be used to predict the sentiments shared in photo tweets. Finally, by analyzing the correlations between the intended emotions and the viewer response comments on social forum like Flickr, we explore the feasibility of developing comment robots that are able to automatically select suitable comments for an image post.
Simultaneously segmenting and labeling images is a fundamental problem in Computer Vision. In this talk, I will introduce a hierarchical framework to deal with the problem of labeling images of Google StreetView scenes by several distinctive object classes. This framework takes advantage of the scale and density of StreetView image coverage. In addition to learning a classifier model from all the labeled images, we group images into clusters of similar images and learn a classifier model from each cluster separately. When labeling a new image, we pick the closest cluster and use the associated classifier model to label this image. In addition to segmentation and labeling results, I will also show how to apply the image labeling result to rerank Google similar images and the extension of this work to business recognition, image segmentation by similar patch search, and multiple image co-saliency detection.
Most multimedia applications try to deliver multimedia content to end users according to their information needs. Thus, multimedia computing actually plays the role of a bridge between multimedia data and user needs. In the past years, however, the multimedia research community mostly focus on multimedia content analysis and understanding; while the user needs over multimedia data are rarely researched, which results in the well-known Intention Gap problem. With the emergence of online social networks, billions of users proactively interact with huge volumes of multimedia data, thereby user behaviors, user relations, user influences etc. can be sensed from the social networks. In this talk, we first will present the social-sensed multimedia computing framework, which tries to inject social factors into traditional multimedia computing and bridges the multimedia content with users. Then, several exemplary applications of this framework, such as social-sensed image search, social-sensed multimedia summarization, social-sensed video communications, social-sensed media recommendation will be presented. Finally, the future research directions will be discussed.
In this presentation we demonstrate a platform for personalized television news to replace the traditional one-broadcast-fits-all model. We forecast that next-generation video news consumption will be more personalized, device agnostic, and pooled from many different information sources. The technology for our research represents a major step in this direction, providing each viewer with a personalized newscast with stories that matter most to them. We believe that such a model can provide a vastly superior user experience and provide fine-grained analytics to content providers. While personalized viewing is increasingly popular for text-based news, personalized real-time video news streams are a critically missing technology.