A Block-Coordinate Descent Approach for Large-scale Sparse Inverse Covariance Estimation
The sparse inverse covariance estimation problem arises in many statistical applications in machine learning and signal processing. In this problem, the inverse of a covariance matrix of a multivariate normal distribution is estimated, assuming that it is sparse. An 1 regularized log-determinant optimization problem is typically solved to approximate such matrices. Because of memory limitations, most existing algorithms are unable to handle large scale instances of this problem. In this paper we present a new block-coordinate descent approach for solving the problem for large-scale data sets. Our method treats the sought matrix block-by-block using quadratic approximations, and we show that this approach has advantages over existing methods in several aspects. Numerical experiments on both synthetic and real gene expression data demonstrate that our approach outperforms the existing state of the art methods, especially for large-scale problems.
This paper proposes a more efficient, block decomposition, algorithm for reconstruction of large sparse inverse covariance matrices. The paper is written clearly. The experimental results show considerable gains in speed and memory for comparable reconstruction accuracy. I am not an expert in this area so I cannot speak to the work's originality, but as far as I could understand the details, the method appears sound. The experimental results seem solid but not ground-breaking because the problems are still small enough to run on a single machine. It would have been nice to see an extension to the really large problems (>10^6 variables) that one sees in some applications by exploting distributed computation and sharding of the various relevant matrices. One concern for replicating the work is that there seems to be some "magic" in setting up the block decomposition and in the line search. It would have been nice to have a discussion of the setting of those choices.