Dykstra's Algorithm, ADMM, and Coordinate Descent: Connections, Insights, and Extensions
We study connections between Dykstra's algorithm for projecting onto an intersection of convex sets, the augmented Lagrangian method of multipliers or ADMM, and block coordinate descent. We prove that coordinate descent for a regularized regression problem, in which the penalty is a separable sum of support functions, is exactly equivalent to Dykstra's algorithm applied to the dual problem. ADMM on the dual problem is also seen to be equivalent, in the special case of two sets, with one being a linear subspace. These connections, aside from being interesting in their own right, suggest new ways of analyzing and extending coordinate descent. For example, from existing convergence theory on Dykstra's algorithm over polyhedra, we discern that coordinate descent for the lasso problem converges at an (asymptotically) linear rate. We also develop two parallel versions of coordinate descent, based on the Dykstra and ADMM connections.
This paper studies connections/equivalences between Dykstra's algorithm, coordinate descent (à la Gauss-Seidel block alternating minimization). Many of these equivalences were already known 
      (indeed essentially in the optimization literature as stated by the author). The authors investigates this for the soft feasibility (best approximation) problem and block-seperable regularized regression,
      where the regularizers are positively homogeneous (i.e. supports of closed convex sets containing the origin). 
      The author claims that this is the first work to investigate this for the casen where the design is not unitary.
      Actually, the extension to arbitrary design is very straightforward (almost trivial) through Fenchel-Rockafellar duality.
      In fact, positive homogeneity is not even needed as Dykstra's algorithm has been extended to tje proximal setting (beyond indicator functions of closed convex sets). 
      Having said this, I have several concerns on this paper.
      + My first concern pertains to the actual novelty of the work, which I beleive is quite limited. The manuscript has  a flavour of a review paper with some incremental extensions.
      + The paper contains some important inaccuracies that should be addressed.
      Other detailed comments are as follows:
      + (1): intersection of sets should be non-empty. Otherwise many of the statements made do not rigorously hold.
      + What the author is calling seminorms are actually support functions (symmetry is NOT needed), or equivalently gauges of polar sets.
      + Extension to the non-euclidean case: several key details are missing for all this to make sense. In particular the notion of a Legendre function. Qualification conditions for strong duality to hold are not stated either.