Fast Algorithms for Gaussian Noise Invariant Independent Component Analysis
The performance of standard algorithms for Independent Component Analysis quickly deteriorates under the addition of Gaussian noise. This is partially due to a common first step that typically consists of whitening, i.e., applying Principal Component Analysis (PCA) and rescaling the components to have identity covariance, which is not invariant under Gaussian noise. In our paper we develop the first practical algorithm for Independent Component Analysis that is provably invariant under Gaussian noise. The two main contributions of this work are as follows: 1. We develop and implement an efficient, Gaussian noise invariant decorrelation (quasi-orthogonalization) algorithm using Hessians of the cumulant functions. 2. We propose a very simple and efficient fixed-point GI-ICA (Gradient Iteration ICA) algorithm, which is compatible with quasi-orthogonalization, as well as with the usual PCA-based whitening in the noiseless case. The algorithm is based on a special form of gradient iteration (different from gradient descent). We provide an analysis of our algorithm demonstrating fast convergence following from the basic properties of cumulants. We also present a number of experimental comparisons with the existing methods, showing superior results on noisy data and very competitive performance in the noiseless case.
The authors propose new algorithms for determining 
      independent components, when the data is corrupted by Gaussian noise. The 
      results of alternative methods are affected by noise. The algorithm 
      improves on the runtime of a method based on the 4th cumulant. The 
      problem is relevant. Quality of the paper is high. The approach seems 
      novel and the contributions significant. The experimental results 
      are not totally convincing. In synthetic experiments most of the variants 
      of ICA perform similarly. For a small number of samples, the 
      quasi-orthogonalization based method performs significantly worse than all 
      other methods. For a large number of samples, though, the method performs 
      at least comparable, even in the noiseless setting. A benefit is 
      demonstrated for a high Gaussian noise setting, if sufficient data is 
      provided. It would be interesting how the different ICA methods 
      compare in various real world applications, where the noise distribution 
      is unknown a priori. Runtime experiments are provided only for the 
      method at hand. To set these into context, the runtime of alternative 
      methods should be included.