Relevance Topic Model for Unstructured Social Group Activity Recognition
Unstructured social group activity recognition in web videos is a challenging task due to 1) the semantic gap between class labels and low-level visual features and 2) the lack of labeled training data. To tackle this problem, we propose a "relevance topic model" for jointly learning meaningful mid-level representations upon bagof-words (BoW) video representations and a classifier with sparse weights. In our approach, sparse Bayesian learning is incorporated into an undirected topic model (i.e., Replicated Softmax) to discover topics which are relevant to video classes and suitable for prediction. Rectified linear units are utilized to increase the expressive power of topics so as to explain better video data containing complex contents and make variational inference tractable for the proposed model. An efficient variational EM algorithm is presented for model parameter estimation and inference. Experimental results on the Unstructured Social Activity Attribute dataset show that our model achieves state of the art performance and outperforms other supervised topic model in terms of classification accuracy, particularly in the case of a very small number of labeled training videos.
General: The paper proposes a supervised (hybrid) 
      topic model for unstructured activity recognition. The model is named: 
      relevance topic model (RTM). The supervised part is the label of the 
      classes of the training videos. RTM is an integration of sparse 
      Bayesian learning and Replicated Softmax. The main concept is to jointly 
      learn discriminative topics as mid-level video representations and 
      discriminant function as a video classifier. RTM is composed of an 
      undirected part to model the marginal distribution of video words and 
      a directed part to model the conditional distribution of video classes 
      given the latent topics. Also, the authors propose the parameter 
      estimation and inference methods. The authors evaluate their algorithm 
      in the Unstructured Social Activity Attribute (USAA) presenting 
      quantitative results of RTM. They compare the method with the literature 
      improving in all cases the activity classification accuracy. Also the 
      authors show an interesting comparison of the correlation of topics of two 
      different classes. The idea seems interesting. The usage of SIFT, 
      STIP and MFCC which are very low level features mostly based on pixel 
      representations. It can be interesting a small discussion on how more 
      descriptive features (object detection) can be added to the model. 
      Relevant work that could be added in the Bibliography, [1] Similar 
      subject [2] interesting approach on structured activity discovery. 
      [1] Social Role Discovery in Human Events Vignesh Ramananthan, 
      Bangpeng Yao, and Li Fei-Fei IEEE Conference on Computer Vision and 
      Pattern Recognition (CVPR). Portland, OR, USA. June 23-28, 2013 [2] J. 
      Varadarajan, R. Emonet and J.-M. Odobez Int. Journal of Computer 
      Vision (IJCV), Vol. 103, Num. 1, pages 100-126, May 2013. Quality: 
      The quality of the paper is good. Clarity: The paper is 
      clear, the problem is well defined. Originality: The 
      originality is Medium. This is not the first approach that aims at 
      building intermediate features for activity recognition. Nor the first one 
      that uses some full or semi-supervision. Unsupervised methods such 
      as [2] for structured scenarios have strong assumptions in the temporal 
      structure of the words (observation vectors) but in unstructured 
      activities as the ones target in this work temporal constraints can be 
      relaxed. Minimal supervision in the learning is acceptable for a 
      difficult task as activity recognition in wild videos. An interesting 
      question would be how many is the minimal human labeled data that is 
      required to classify wild videos decently. Significance: The 
      topic is exciting research, I would say of big significance.