On Algorithms for Sparse Multi-factor NMF
Nonnegative matrix factorization (NMF) is a popular data analysis method, the objective of which is to approximate a matrix with all nonnegative components into the product of two nonnegative matrices. In this work, we describe a new simple and efficient algorithm for multi-factor nonnegative matrix factorization (mfNMF) problem that generalizes the original NMF problem to more than two factors. Furthermore, we extend the mfNMF algorithm to incorporate a regularizer based on the Dirichlet distribution to encourage the sparsity of the components of the obtained factors. Our sparse mfNMF algorithm affords a closed form and an intuitive interpretation, and is more efficient in comparison with previous works that use fix point iterations. We demonstrate the effectiveness and efficiency of our algorithms on both synthetic and real data sets.
The paper proposes a new multiplicative 
      algorithm for multi-factor nonnegative matrix factorization (mfNMF). 
      The closest previous research is [3] (in Section 4.4), where the 
      target matrix is approximated based on the general Bregman 
      divergenceminimized by a multiplicative algorithm.The current 
      paper proposes an efficient algorithm specialized for the general 
      Kullback-Leibler (KL) divergence, a special case of Bregman 
      divergence.Experiments show that the proposed algorithm outperforms 
      the algorithm proposed in [3] for the general KL divergence, as well 
      as another type of mfNMF called multilayer NMF [7-9].The paper 
      also proposes a sparse variant of mfNMF with a Dirichlet 
      regularizer.In the proposed framework, the Dirichlet regularizer 
      behaves like a conjugate prior,and the sparse variant is built with a 
      simple modification (Eq.(8) to Eq.(12)) of the objective.The 
      obtained algorithm (Lemma 4) seems like ones for Bayesian automatic 
      relevance determination.I think this is an interesting paper. The 
      main idea for deriving a new algorithm is to decompose the target 
      matrix into the product of a stochastic matrix and a diagonal scale 
      matrix, and then simplify the objective based on the decomposition (2) 
      of the generalized KL divergence.This alleviates the ill-posedness 
      of mfNMF, and results in better performance (better quality of the 
      found local minima), and efficient computation, as shown in the 
      experiments. The authors also elaborate on making the algorithm fully 
      multiplicative by introducing an auxiliary function (Lemma 2) for the 
      partial problem (4).The obtained algorithm seems simple to 
      implement, and work fine in the experiments.Pros:- The 
      proposed algorithm is simple and experimentally shown to outperform 
      the previous methods.- Unique ideas are found: focusing on 
      stochastic matrices, and using decomposition (2) of the generalized KL 
      divergence. These techniques might be used for other 
      problems.Cons:- Experiments on the MNIST data set do not 
      appeal the usefulness of mfNMF and sparse mfNMF. I do not understand 
      what mfNMF found in Fig.3other than what two-factor NMF can find. In 
      other words,why do you want to decompose W2W3 into W2 and W3? 
      Quality:The paper is technically sound, and the claims are 
      supported by the experiments.Clarity:The paper is clearly 
      written. But I would suggest the authors to focus onthe closest 
      research [3] in Section 2, and make readers understand what the 
      differences between [3] and the current paper without need of checking 
      the reference.Originality:The idea to focus on the stochastic 
      matrices (in a problem where the purpose is not to find stochastic 
      matrices) seems new.Significance:Since the proposed algorithm 
      is simple and experimentally shown to outperform the previous methods, 
      it would be used by practitioners.The idea of focusing on the 
      stochastic part of the target matrixmight be used by other researchers 
      when they develop new algorithms.Minor comments:- In Lemma 1, 
      S = D^-1 V should be S = V D^-1. Lemma 1 seems trivial becauseits just 
      a column-wise normalization, so just stating that 'we decompose V as V 
      = SD' might be ok.- I suppose that Eq.(3) is the maximization of the 
      similarity between the stochastic matrices, S^{V} and S^{W}, weighted 
      by D^{V}. If so, noting this would make it clear that you are focusing 
      on approximating the stochastic part.