Submodular Attribute Selection for Action Recognition in Video
In real-world action recognition problems, low-level features cannot adequately characterize the rich spatial-temporal structures in action videos. In this work, we encode actions based on attributes that describes actions as high-level concepts e.g., jump forward or motion in the air. We base our analysis on two types of action attributes. One type of action attributes is generated by humans. The second type is data-driven attributes, which are learned from data using dictionary learning methods. Attribute-based representation may exhibit high variance due to noisy and redundant attributes. We propose a discriminative and compact attribute-based representation by selecting a subset of discriminative attributes from a large attribute set. Three attribute selection criteria are proposed and formulated as a submodular optimization problem. A greedy optimization algorithm is presented and guaranteed to be at least (1-1/e)-approximation to the optimum. Experimental results on the Olympic Sports and UCF101 datasets demonstrate that the proposed attribute-based representation can significantly boost the performance of action recognition algorithms and outperform most recently proposed recognition approaches.
This paper presents an algorithm for attribute      (feature) selection, applied to activity recognition.  The      approach defines 3 criteria for choosing attributes for      recognition: high discrimination between pairwise classes, similarity      in the amount of discrimination over all pairs of classes, and      coverage of all pairs of classes.  The first two of these      criteria are formulated using a random walk approach, the latter      using a set-cover approach.  Finally, a greedy optimization      strategy is used to choose an attribute subset, starting from      the empty set.      This paper presents interesting research.  The main contribution of      the paper is the algorithm for feature selection.  This      algorithm seems novel, and is an interesting combination of      random walk, set-cover, and greedy optimization.  The      experimental results are also good -- the method shows      improvements over existing methods and baselines including no      feature selection and alternative strategies.  There are some      missing details in the experiments but the results seem solid.      My main concern about the paper is the motivation/intuition for      the approach -- the pieces seem chosen to try to use submodular      optimization.  The discussion on lines 133-145 describes 3      criteria.  I wasn't clear on the motivation for criterion 2 --      why can't some attributes be better for certain classes than      others?  Can't a final classifier choose combinations?  It seems      that this second criterion complicates the optimization, and      necessitates the use of the proposed techniques.      The resulting submodular optimization amounts to a      greedy approach.  It seems one could do similar greedy      optimization for other fomulations, for example ones with      criterion 2.  It would be interesting to know whether this      criterion is important empirically.      Overall, this concern is not fundamental; I think the paper is      very good, and should be accepted.      Other comments:      - I haven't gone over the details in supplementary material --      the intuitive explanations in the paper for these seemed      reasonable though.      - Typo "Combing"      - I didn't understand where the sparse coding (Eq. 5) is used in      the paper.  The experiments refer to KSVD for the DDA set.      Sparse coding of attributes doesn't seem to be a key component      of the paper, but if this is used it should be compared to      k-means or alternatives.  (Though there isn't space for this in      a conference version.)      - How are hyperparamters set in the experiments?  Are they tuned      by looking at the test (i.e. cross-validation) accuracies?      - The experimental results seem better than previous work.      Where are the numbers in Table 2 from?  [20] is shown with a      per-activity AP list, and mAP of 71.6.  In [20], Fig. 10(b)      shows mAP of 74.38%.  This would seem to be better than what is      in this paper.      - The related work section is thorough, though the work of Fu et      al., who also do similar attribute learning could be added:Learning multi-modal latent attributesY. Fu, T. Hospedales, T. Xiang and S. GongIEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI 2013)