Near-Optimal Entrywise Sampling for Data Matrices
We consider the problem of selecting non-zero entries of a matrix A in order to produce a sparse sketch of it, B, that minimizes A B 2 . For large m n matrices, such that n m (for example, representing n observations over m attributes) we give sampling distributions that exhibit four important properties. First, they have closed forms computable from minimal information regarding A. Second, they allow sketching of matrices whose non-zeros are presented to the algorithm in arbitrary order as a stream, with O 1 computation per non-zero. Third, the resulting sketch matrices are not only sparse, but their non-zero entries are highly compressible. Lastly, and most importantly, under mild assumptions, our distributions are provably competitive with the optimal offline distribution. Note that the probabilities in the optimal offline distribution may be complex functions of all the entries in the matrix. Therefore, regardless of computational complexity, the optimal distribution might be impossible to compute in the streaming model.
Near-optimal Distribution for Data Matrix Sampling 
      The paper presents a new sampling methods for generating (random) 
      sparse matrices that best approximate a given matrix in terms of the 
      spectral norm. Several assumptions are made and well motivated, such as 
      the "data matrix" properties and the constraints of the computational 
      model (i.e. streaming data). The exposition is very readable, although 
      some topics such as compressibility and fast indexing are brought up that 
      are then never related back to the method proposed. More to the heart of 
      the matter, the author(s) start with discussing the commonly known L1 and 
      L2 sampling schemes, which serve as reference points throughout the 
      development of the paper and then move on to the matrix Berstein 
      inequality and explain its significance in the context of sparse matrix 
      sampling. A lot of attention is given to proper explanation and intuitive 
      arguments, which makes the paper accessible for non-specialists. The only 
      thing that has to be said is that the text becomes somewhat "talkative" 
      and lengthy, in particular in comparison to the very condensed 
      experimental section. I would recommend shortening these passage, while 
      keeping the gist of the arguments and explanations. The main 
      contributions of the paper are algorithm 1 together theorem 2.2, which 
      justifies it. I find eq 6 and what feeds into it somewhat hard to digest. 
      Some properties (e.g. certain regimes for s: small, large) are explained 
      in the surrounding text, but it would be nice to find a way of relating 
      this back to the Bernstein inequality that supposedly motivates it. Some 
      of this happens in section 5, but overall the algebraic complexity and 
      notation make this difficult and it would be nice to preserve the crucial 
      link between how even the simplified row-L1 distribution and the Bernstein 
      inequality. With all these caveats around the structure and clarity of the 
      presentation of the main contributions, I deem the result to be 
      significant and the key idea as well as the execution of the proof and 
      algorithmic detail to be highly non-trivial. The experimental 
      section and the empirical analysis are somewhat disappointing by 
      comparison. First, I cannot even read the graphs in the printed paper 
      (which is a nuisance). Second, there is really no diving deeper on any 
      aspect. Third, no connection to any relevant application is made. Fourth, 
      the spectral norm criteria is thrown out ("it turns out to be more 
      fruitful to consider…") and replaced by a new criterion based on 
      projections to the top singular vectors. The spectral norm objective is 
      not even plotted or mentioned any more. Given how much care has been spent 
      on the theory part, how detailed things have been motivated and then 
      worked through in the proof, this feels like quite a surprise. Shouldn't 
      one go back and change the objective to address the "scaling" issue, for 
      instance?