Solving Non-smooth Constrained Programs with Lower Complexity than O(1/"): A Primal-Dual Homotopy Smoothing Approach
We propose a new primal-dual homotopy smoothing algorithm for a linearly constrained convex program, where neither the primal nor the dual function has to be smooth or strongly convex. The best known iteration complexity solving such a non-smooth problem is O(" 1 ). In this paper, we show that by leveraging a local error bound condition on the dual function, the proposed algorithm can achieve a better primal convergence time of O " 2/(2+ ) log 2 (" 1 ) , where 2 (0, 1] is a local error bound parameter. As an example application of the general algorithm, we show that the distributed geometric median problem, which can be formulated as a constrained convex program, has its dual function non-smooth but satisfying the aforementioned local error bound condition with = 1/2, therefore enjoying a convergence time of O " 4/5 log 2 (" 1 ) . This result improves upon the O(" 1 ) convergence time bound achieved by existing distributed optimization algorithms. Simulation experiments also demonstrate the performance of our proposed algorithm.
The authors propose a primal dual homotopy smoothing algorithm for the minimization of a convex function over a convex compact set with additional equality constraints. Being based on the primal dual algorithm, the method requires an accuracy level which needs to be carefully tuned in practice and is critical for the convergence of the algorithm. It is difficult to parse Theorem 3.1. I would like to see an explicit convergence rate in terms of feasibility and suboptimality. In particular, the algorithm does not converge to the optimum, but only to an epsilon accurate solution (as in the universal primal-dual). This should be carefully discussed. In particular, the whole method should be put in perspective with “Frank-Wolfe Splitting via Augmented Lagrangian Method” for the problem in equation 16-17 and “A conditional Gradient Framework for Composite Convex Minimization with Applications to Semidefinite Programming”. In particular, the latter seem to propose the same smoothing technique. While they obtain a slower rate, their subproblem is easier to solve and they prove convergence to the optimal value and feasibility. Therefore, a discussion on the differences of the two approaches and an empirical comparison would be important. -------------------------------------------------------------------- After the rebuttal I decided to increase my score to 6 as the approach can deal with non smooth f as well.