Online and Stochastic Gradient Methods for Non-decomposable Loss Functions
Modern applications in sensitive domains such as biometrics and medicine frequently require the use of non-decomposable loss functions such as precision@k, F-measure etc. Compared to point loss functions such as hinge-loss, these offer much more fine grained control over prediction, but at the same time present novel challenges in terms of algorithm design and analysis. In this work we initiate a study of online learning techniques for such non-decomposable loss functions with an aim to enable incremental learning as well as design scalable solvers for batch problems. To this end, we propose an online learning framework for such loss functions. Our model enjoys several nice properties, chief amongst them being the existence of efficient online learning algorithms with sublinear regret and online to batch conversion bounds. Our model is a provable extension of existing online learning models for point loss functions. We instantiate two popular losses, Prec @k and pAUC, in our model and prove sublinear regret bounds for both of them. Our proofs require a novel structural lemma over ranked lists which may be of independent interest. We then develop scalable stochastic gradient descent solvers for non-decomposable loss functions. We show that for a large family of loss functions satisfying a certain uniform convergence property (that includes Prec @k , pAUC, and F-measure), our methods provably converge to the empirical risk minimizer. Such uniform convergence results were not known for these losses and we establish these using novel proof techniques. We then use extensive experimentation on real life and benchmark datasets to establish that our method can be orders of magnitude faster than a recently proposed cutting plane method.
Summary:This paper studies online learning techniques for optimizing non-decomposable loss functions, such as precision@k and partial AUC, aiming to enable incremental learning as well as design scalable solvers for batch learning tasks. They prove sublinear regret and online to batch conversion bounds for the proposed algorithms. This paper develops scalable stochastic gradient descent solvers for non-decomposable loss functions and validates its efficiency via empirical studies.Quality:This paper proposes a practical online learning framework for non-decomposable loss functions. It presents two algorithms with mini batches, and gives theoretical guarantee with sublinear regret and online to batch conversion bounds. Empirically, the two proposed algorithms are compared with two existing batch learning algorithms, and found better results on several real-world data sets. In general, the work appears to be very similar to the series of studies in [13,14]. However, the authors did not clearly clarify the differences and contributions as compared to the similar work in [13,14], making it difficult to justify the novel contributions.  Clarity:The manuscript is generally well written and easy to follow, except for some minor typos. For example, In line 170, it should be “not clear and remains” instead of “not clear an remains”. Originality:This paper extends the ideas in [13, 14] to tackle the general online learning tasks with non-decomposable loss functions. Instead of optimizing AUC, the authors consider two types of other losses, precision@k and partial AUC, and propose two specific online learning algorithms (single-pass and two-pass online learning) using the idea of mini-batches (which are essentially very similar to the buffering techniques as used in [13,14]). The extensions are OK, but not particularly novel. Significance:This paper addresses an important problem, online learning with non-decomposable loss functions, which could be more difficult than regular online learning with pointwise loss functions. The paper has presented two new online/stochastic learning algorithms for optimizing partial AUC and precision@k, and found better results in comparison to two existing batch learning algorithms. The limitations of this work is that the authors did not justify the significance of the proposed algorithms over the existing baselines in literature (e.g., [13,14]). Comments on specific issues:1.	The empirical study has not been done in a thorough way. Since the paper aims to address online learning tasks, it is not sufficient to compare only with two batch learning algorithms. In particular, one would expect a direct comparison of the proposed algorithms with some closely related online learning algorithms in literature (e.g., [13,14]). 2.	The two proposed algorithms (1PMB and 2PMB) have their obvious drawbacks although they seem to outperform the batch learning algorithms. In particular, for the first algorithm could fail to exploit relationships between data points, as admitted by the authors; for the second algorithm, simply collect a (single) random sample of positive example B+ may be neither effective (if sampling size s is too small) nor memory efficient (if sampling size s is too large). I think the buffer sampling treatments proposed in [13, 14] seem to be more practical to address such drawbacks of the proposed algorithms in this work. 3.	Tuning parameters is not well explained. The authors only stated “all tunable parameters were chosen using a small validation set”. As the proposed algorithm depends on several important parameters such as the buffer size s of B+, B-, the step scale length \eta, which could affect the performance. However, there are no discussions or experiments for examining how these parameters.  4.	The empirical results fail to validate efficacy of the proposed two-pass algorithm (2PMB). It seems that 2PBM is similar or even worse than the Single-pass algorithm (1PMB) for most cases. Since this algorithm needs to take multiple passes, the value of the 2PMB algorithm seems in doubt.