Spectral k-Support Norm Regularization
The k-support norm has successfully been applied to sparse vector prediction problems. We observe that it belongs to a wider class of norms, which we call the box-norms. Within this framework we derive an efficient algorithm to compute the proximity operator of the squared norm, improving upon the original method for the k-support norm. We extend the norms from the vector to the matrix setting and we introduce the spectral k-support norm. We study its properties and show that it is closely related to the multitask learning cluster norm. We apply the norms to real and synthetic matrix completion datasets. Our findings indicate that spectral k-support norm regularization gives state of the art performance, consistently improving over trace norm regularization and the matrix elastic net.
In this paper, the authors derive new properties of the k-support norm [1], devise a new efficient algorithm for computing its proximal operator and generalize it to matrices.First, they show that the k-support norm can be written as a box norm. Box norms are defined using a variational formulation (infimum of quadratics: |w| = \inf_{t \in T} \sum w_i^2 / t_i) and are a special case of the norms described in [2]. Conversely, squared box norm can be written as the sum of squared l2 norm and k-support norm.Since the box norms are symmetric gauge function, they propose to extend them (and the k-support norm as a special case) to matrices. They also show that the cluster norm is equivalent to spectral box norm. They devise efficient algorithms to compute both the norm and their associated proximal operator, with a complexity of O(d log(d)), where d is the dimension. Finally, they report experimental results for matrix completion problems on synthetic and real datasets, showing that the spectral k-support norm perform slightly better than the trace norm.This is a solid and well written article about the relations between the k-support norm and the box norms. The extension of those norms to matrices is pretty straightforward. The algorithm proposed in the paper for computing the proximal operator of the k-support norm should be useful to the NIPS community, as it is more efficient than previously known methods: the authors demonstrate that their approach is several order of magnitude faster than the algorithm in [1]. My only small concern is the lack of really convincing experiments about the usefulness of the spectral k-support norm, compared to the trace norm or the matrix elastic net.[1] A. Argyriou, R. Foygel, N. Srebro. Sparse Prediction with the k-Support Norm. 2012[2] C. Micchelli, J. Morales, M. Pontil. Regularizers for Structured Sparsity. 2010.