Self-Adaptable Templates for Feature Coding
Hierarchical feed-forward networks have been successfully applied in object recognition. At each level of the hierarchy, features are extracted and encoded, followed by a pooling step. Within this processing pipeline, the common trend is to learn the feature coding templates, often referred as codebook entries, filters, or over-complete basis. Recently, an approach that apparently does not use templates has been shown to obtain very promising results. This is the second-order pooling (O2P) [1,2,3,4,5]. In this paper, we analyze O2P as a coding-pooling scheme. We find that at testing phase, O2P automatically adapts the feature coding templates to the input features, rather than using templates learned during the training phase. From this finding, we are able to bring common concepts of coding-pooling schemes to O2P, such as feature quantization. This allows for significant accuracy improvements of O2P in standard benchmarks of image classification, namely Caltech101 and VOC07.
The paper aims to provide a more general interpretation of a recent class of `tensor representations’ based on second order information. In particular the authors provide derivations in support of the claim that such second order pooling methodologies can be viewed as coding-pooling schemes that adapt the feature coding pattern to the input features. Moreover, the authors show that by incorporating coding into the new interpretation some improvements can be achieved in benchmarks (N.B. the results on VOC07 are significant; however, the results on Caltech101 by no means represent a significant improvement over the state of the art).In general this is an interesting, clearly written paper (modulo some issues, having to do with the presentation of the contributions, described below) which provides valuable insight into a class of successful, emerging techniques. Overall I find this paper to be relevant and worth publishing at NIPS. I would appreciate if the authors considered the following comments.- CommentsI would say that some of the main contributions associated with the (second-order) tensor representation are probably more attributable to [1],[16], than to [1],[2] (line 133). Equally so, line 142 should read `…celebrated Log-Euclidean metric [4,5]’ instead of just [4] as these have been introduced in the vision context simultaneously at ECCV12. Overall I would make it clear also that [3] uses a different metric than [4,5], for example. That’s why the approach authors take in baptizing slightly different methods as TR could be misleading in places like Table 1 or 2. Moreover, as far as I can understand the various approaches denoted Pyr TR w/o SQ = O2P [5]. Particularly as some of the code in [5] appears to be used (do the authors use the enriched SIFT features of [5] in their experiments with Caltech 101, or in the coded features they propose?), it would be appropriate to not `remove all traces’ and present existing methods as if they were proposed by the authors, under a new name. Best would be to conserve the original names and use new ones only when new elements are proposed by the authors, e.g. Pyr. Tr + SQ.The list of references is good. The authors can consider adding: The Devil Is in the Details: an Evaluation of Recent Feature Encoding Methods: Chatffiled et al., BMVC 2011.Efficient Match Kernels between Sets of Features for Visual recognition, Bo et al., NIPS 2009.which also discuss different encoding strategies (former) and provide potentially useful kernel interpretations for bag of words models (latter).