What Makes Objects Similar: A Unified Multi-Metric Learning Approach
Linkages are essentially determined by similarity measures that may be derived from multiple perspectives. For example, spatial linkages are usually generated based on localities of heterogeneous data, whereas semantic linkages can come from various properties, such as different physical meanings behind social relations. Many existing metric learning models focus on spatial linkages, but leave the rich semantic factors unconsidered. Similarities based on these models are usually overdetermined on linkages. We propose a Unified Multi-Metric Learning (UM 2 L) framework to exploit multiple types of metrics. In UM 2 L, a type of combination operator is introduced for distance characterization from multiple perspectives, and thus can introduce flexibilities for representing and utilizing both spatial and semantic linkages. Besides, we propose a uniform solver for UM Visualization results also validate its ability on physical meanings discovery.
This paper presents a framework - UM2L - for learning multiple metrics at the same time for then applying a kind of nearest neighbor classification or a majority voting. The framework is tailored to triplet-based constraints and allows to integrate different loss functions, different functions to combine the metrics that can depend be adapted to similar instances or  impostors and various regularizers even though the paper considers mainly the L-2,1 norm. The problem is formalized as a general optimization problem for which a theorem of convergence is given in the context of gradient descent. Some examples illustrating the interest of the framework are given. A large experimental evaluation is presented. 
Pros -Generality of the framework for multiple metric learning allowing to deal with different metric combination -Wide range of applicability and good empirical results -Theoretical result on the convergence of the optimization method -paper well written Cons -The framework does not really provide a solution for combining metrics learned in different regions that may suffer from discontinuity between two regions. So there is no particular novelty in the combination of local metrics. -The method requires a specific initialization that is not really discussed or analyzed -No theoretical guarantees in terms of generalization capability  Overall: Interesting paper for combining multiple metrics that offer a wide range of application. The paper has no theoretical guarantees and does not really bring a solution to the problems posed by local metric learning methods.  Comments -I like the proposed framework because it embeds a large set of possible way of combining metrics.  The idea of using different functions for combining metrics could have been used in other papers, for me the real novelty is to provide a sufficiently general framework for combining these metrics.  Even though, it must be noticed that combining multiple metrics with a max function has not really been studied, as far as I know.  -The initialization of the model is evoked quickly in the supplementary material but it is not really discussed in the paper.  This initialization is based on a clustering step which has a probably an important impact on the result and deserves more attention/analysis.  -Some local metric learning approaches try to learn a metric for a local region to capture local regularities. However, at test time, the choice of the metric to use is a critical issue because of the discontinuity between the learned metrics in case where the two instances to compare may belong to two different regions. Indeed, even for two close regions the metric can be rather different and combining them is not really easy. This is a difficult task, but this paper does not seem to provide a solution to this problem, check this reference for example: *V. Zantedeschi, R. Emonet, M. Sebban. Metric Learning as Convex Combinations of Local Models With Generalization Guarantees. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.   At test time, if I understood correctly, the paper uses all the metrics for all the pairs of examples without penalizing the metrics wrt to spatial information, which seems not natural for taking into account local similarities. I understand that each metric used in UM2L can capture different semantic information in the data, but I do not see how local information can be captured if the learning(or testing) step is global to the whole dataset.  -In the experimental comparison - Section 4.2 in particular, UM2L is used with a specific setting: use of the mixed norm and a majority vote over 3*K distances. The other approaches, in particular the local ones, do not use such a setting and it would be interesting to evaluate if this really the framework that is better or the procedure to take the decision.  For example, it could be interesting to evaluate the performance of each individual metric which can help to assess the expressiveness of the model.  The impact of regularization can also be analyzed.  -In the experimental setup, it is mentioned that 3 targets en 10 impostors are built for each dataset, but we do not know how these triplet are chosen (randomly, by a landmark selection method, cluster centers, ...)  -In the multiview setting, an assumption often made is that some views may compensate other views when they are wrong. In this setting, if one can not bring different weights to each view, it not clear if the combination can bring a better information than individual view.  -When considering the average similarity (ACS), in the batch setting, it is not clear for me if the framework is useful in a 1-view setting if one initializes the model to the same metric before optimization. I do not see how the model can learn different metrics unless we force  the triplets to be optimized only on some particular metrics.  -There is no theoretical guarantees in terms of generalization ability to new instances.  I think that this is probably an important line of work to consider. In particular, nothing is said about the importance of maximizing the margin or in other words on the separation ability brought by each metric. However, this seems an important aspect in particular with RGS and OVS settings.  In OVS, for example, trying to maximize the margin can be way to avoid trivial solutions.  On the other hand, using a ||M_k-I|| regularization can be a good idea in problems where the Euclidean distance works well.   -l.90, notation "\kappa s" has not been introduced, is it  \kappa_v ?.