Zero-Shot Recognition with Unreliable Attributes
In principle, zero-shot learning makes it possible to train a recognition model simply by specifying the category's attributes. For example, with classifiers for generic attributes like striped and four-legged, one can construct a classifier for the zebra category by enumerating which properties it possesses-even without providing zebra training images. In practice, however, the standard zero-shot paradigm suffers because attribute predictions in novel images are hard to get right. We propose a novel random forest approach to train zero-shot models that explicitly accounts for the unreliability of attribute predictions. By leveraging statistics about each attribute's error tendencies, our method obtains more robust discriminative models for the unseen classes. We further devise extensions to handle the few-shot scenario and unreliable attribute descriptions. On three datasets, we demonstrate the benefit for visual category learning with zero or few training examples, a critical domain for rare categories or categories defined on the fly.
This paper presents a method for attribute-based zero-shot learning using random forests.  Attribute classifiers are first trained using SVMs.  A random forest is trained on class-attribute signature vectors, while using a validation set of images with attribute-labels is used to estimate attribute reliability at each node of the tree.  An extension to few-shot learning is also proposed.Clarity:  The paper is very clear and well-written.  It does a good job summarizing related work.Originality:  The novelty of this paper is somewhat low, but not too low to be accepted.  One issue is that the claim that the proposed approach deals with unreliable attribute predictions while earlier work does not is a bit overstated or misleading.  For example, direct attribute prediction [7] (DAP)--the baseline method used in this paper and many others--has some similar abilities to deal with attribute uncertainty.  Both approaches use the basic approach: a) Train attribute classifiers on image features (identical for both papers)b) Train a 2nd class probability estimator as a function of attribute predictions, using a validation set to estimate the reliability of attribute classifiers ([6] uses Platt scaling and a probabilistic model to combine attributes, whereas the proposed approach uses a new random forest-based method)In this way, I don't find the proposed approach to be a fundamentally new way of handling attribute uncertainty.  What it does differently than earlier work is that it uses a more complex model for modelling attribute noise: 1) it uses an ROC-based estimate instead of a sigmoidal model like Platt scaling, and 2) whereas [6] treats attributes as independent, the proposed approach effectively models joint statistics in attribute errors, since the validation set is propagated down trees.  In this way, I think it would be a little more appropriate to describe the approach as a better way to model attribute correlation (I don't know this area well, but Scheirer et al. "Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search" might be one related paper that could be cited).At the same time, one possible weakness of the proposed approach in comparison to methods that treat attributes independently is that it seems like one would need a large validation set to obtain valid probability estimates as it gets subdivided when propagated down decision trees.  This issue was glossed over in the paper, and proposed approach for dealing with this (line 323) seems heuristic.  It would have been nice to see more experiments evaluating the effect of training and validation set size.Quality: In general, the quality of the paper is good and the technical choices and descriptions make sense.  The experiments are appropriate and support the claims of the paper, although the level of quantitative improvement isn't high enough to accept the paper solely on the merit of empirical results.  The extension to few-shot learning (Section 3.3) felt less polished than the rest of the paper.  Whereas the rest of the paper was clear and explanations were lengthy, this section was very brief.  To me, a method that could combine class-attribute priors and image attribute estimates when estimating probabilities p(a|...) might make more intuitive sense then a weighted combination of information gains (Eq. 6).