Alternating Estimation for Structured High-Dimensional Multi-Response Models
We consider the problem of learning high-dimensional multi-response linear models with structured parameters. By exploiting the noise correlations among different responses, we propose an alternating estimation (AltEst) procedure to estimate the model parameters based on the generalized Dantzig selector (GDS). Under suitable sample size and resampling assumptions, we show that the error of the estimates generated by AltEst, with high probability, converges linearly to certain minimum achievable level, which can be tersely expressed by a few geometric measures, such as Gaussian width of sets related to the parameter structure. To the best of our knowledge, this is the first non-asymptotic statistical guarantee for such AltEst-type algorithm applied to estimation with general structures.
Bounds on the estimation error in a multi-response structured Gaussian linear model is considered, where the parameter to be estimated is shared by all m responses and where the noise covariance may be different from identity. The authors propose an alternating procedure for estimation of the coefficient vector that essentially extends the GDS (generalized Dantzig selector). The new procedure accommodates general "low-complexity" structures on the parameter and non-identity covariance of the noise term. 
The paper is outside my field of expertise, so I apologize in advance for any mistakes due to my misunderstanding. 
The paper is very technical, and without the appropriate background, I was not able to really appreciate the significance of the contribution. That said, the theme of the results is quite clear, especially with the effort of the authors to supplement the results with intuitive explanations. 
1. The paper is very technical, maybe too technical, and it is hard for a non-expert like myself to make sense of the conditions/assumptions made in the different theorems (despite the authors' evident effort to help with making sense of these conditions). 
2. Again, I was not able to appreciate if the different assumptions and results are sensible or whether they are mostly ad-hoc or artifacts of the proofs, but overall the contribution over GDS and existing methods does not appear to be sufficiently significant for acceptance to NIPS. 
3. The manuscript is not free of grammatical errors or bad wording, for example:
-l.168: "at the same order"
-caption of Fig 2: "The plots for oracle...implies..."
-caption of Fig 2: "keep unchanged" (better: "remain unchanged")
## comments after reading author rebuttal:
The authors have provided a clear non-technical summary of the paper, which is much appreciated. I would like to apologize for not being able to give a fair evaluation when originally reviewing the paper (this is because I did not have the technical background to appreciate the results).