Gen-Oja: A Simple and Efficient Algorithm for Streaming Generalized Eigenvector Computation
In this paper, we study the problems of principal Generalized Eigenvector computation and Canonical Correlation Analysis in the stochastic setting. We propose a simple and efficient algorithm, Gen-Oja, for these problems. We prove the global convergence of our algorithm, borrowing ideas from the theory of fastmixing Markov chains and two-time-scale stochastic approximation, showing that it achieves the optimal rate of convergence. In the process, we develop tools for understanding stochastic processes with Markovian noise which might be of independent interest.
Gen-Oja: A Simple and Efficient Algorithm for Streaming Generalized Eigenvector Computation  Principal Generalized Eigenvector computation and Canonical Correlation Analysis in the stochastic setting.  Propose a simple and efficient algorithm for these problems. Overall the approach appears to be reasonable and is scientific sound. The experimental part is however quite limited - the paper is widely a theoretical proposal and it remains unclear if it is really effective in a practical setting.  The paper is also closely linked to streaming PCA and in parts it is not so clear if there is some strong novelty in this proposal or just a lot of  mathematical wrapping.  comments: - how critical is the i.i.d. condition after Eq. 5 - in my view it is a bit   unlikely to assume that a data stream can be sampled in this way under realistic   conditions - so one may only 'hope' that it holds, or it can be applied only    in some scenaria - as the experiments are very limited the i.i.d. assumption is a key problem and   it is not shown in the experiments that this assumption is not a particular problem - related work is discussed but not much addressed in the experimental part   as outlined in the paper there is a lot related work - but not in your experiments - although the theory looks convincing it would be good to show the effectiveness   also in a number of practical application. Numerical problems are common for such   type of 'approximations' and behind theory the practical results may be different - it may also not harm to add some discussions to practical results (--> in the supplementary material) - I have not checked in detail the supplementary material    --> journal submission?  Comments - to author feedback: - thanks for addressing the comments - some points are clarified - I (now) understand that the authors are focusing on a theoretical improvement   -- I think it would be good to adapt the wording in the paper and maybe also in th title to make this more explicit