Discriminative Robust Transformation Learning
This paper proposes a framework for learning features that are robust to data variation, which is particularly important when only a limited number of training samples are available. The framework makes it possible to tradeoff the discriminative value of learned features against the generalization error of the learning algorithm. Robustness is achieved by encouraging the transform that maps data to features to be a local isometry. This geometric property is shown to improve (K, )-robustness, thereby providing theoretical justification for reductions in generalization error observed in experiments. The proposed optimization framework is used to train standard learning algorithms such as deep neural networks. Experimental results obtained on benchmark datasets, such as labeled faces in the wild, demonstrate the value of being able to balance discrimination and robustness.
This paper presents a method to learn a transformation using a loss function in which it is possible to control the trade-off between robustness and class discrimination. The loss function is a linear combination of a discriminative element, which tries to maximise the between-class separation and a robustness element, which tries to preserve isometry between source feature space and target (classification) mapping. The method is theoretically justified based on the metric learning method of [1], thought the focus here is on learning a mapping function, rather than a metric. The proposed loss function is differentiable and can be used in a Gradient descent framework. It was applied as part of a Neural Network, enabling non-linearity by having multiple layers.The experiments prove the concept, for instance, they show that when a reduced training set is available, one benefits from using smaller values of lambda (the discrimination ratio), which is quite intuitive.However, I felt that the experiments are a bit limited. It would have been interesting to see more practical applications of the proposed method, showing when it is relevant to be able to tune between robustness and discrimination power. Perhaps this method could be applied in domain adaptation datasets. Minor style issues: - sentences starting with "And..." - missing spaces between words and brackets, e.g. "decreases(Fig. 3b)" - "attentions have" -> "attention has" Note: I'm pleased to see that clarity has improved in the final version of the paper and that more details and evaluations are presented in the experimental results section.