Structured Estimation with Atomic Norms: General Bounds and Applications
For structured estimation problems with atomic norms, recent advances in the literature express sample complexity and estimation error bounds in terms of certain geometric measures, in particular Gaussian width of the unit norm ball, Gaussian width of a spherical cap induced by a tangent cone, and a restricted norm compatibility constant. However, given an atomic norm, bounding these geometric measures can be difficult. In this paper, we present general upper bounds for such geometric measures, which only require simple information of the atomic norm under consideration, and we establish tightness of these bounds by providing the corresponding lower bounds. We show applications of our analysis to certain atomic norms, especially k-support norm, for which existing result is incomplete.
=== Summary ===In this paper, the authors propose to bound geometric quantities that appears in statistical error bounds of structured estimation problems with atomic norms.It was previously shown that the estimation error of Dantzig-like estimator, with random Gaussian design, could be bounded by using three key quantities: the Gaussian width of the unit ball of the regularizer, the Gaussian width of the spherical cap of the tangent cone and the restricted norm compatibility constant. The goal of this article is to provide easy to compute bounds for these three quantities, in the case of sign-invariant atomic norms.In section 3, the authors introduce upper bounds on the three geometric quantities, which are the main contribution of the paper. The Gaussian width of the unit ball can be bounded by decomposing the set of atoms as the union of simpler sets, for which the Gaussian width are easy to compute. This result already appeared in [14]. The Gaussian width of the spherical cap of the tangent cone can be bounded using a single subgradient of the regularizer, taken at the true parameter vector w^*. Finally, the restricted norm compatibility constant can be bounded using two other "norm compatibility constants" which are simpler to compute (because the set on which they are defined are simpler). The authors show on two examples, the L1 and OWL norms, how these results can easily be applied.In section 4, the authors provide lower bounds for the two last quantities. The lower bounds does not match the upper bounds, but show that the upper bound are relatively tight, and thus usefull.In section 5, the bounds obtained in this paper are applied to the k-support norm, improving previously known results.=== Significance ===I believe this paper is addressing an important problem, providing interesting and useful new tools for studying sparse estimation problems with atomic norms. The paper is clearly written and well organized. The use of examples helps to understand the results and how to apply those in practice. Finally, the bounds derived in the paper allowed to obtain new results for the k-support norm, showing the relevance of the results.