Deep Generalized Canonical Correlation Analysis
We present Deep Generalized Canonical Correlation Analysis (DGCCA) – a method for learning nonlinear transformations of arbitrarily many views of data, such that the resulting transformations are maximally informative of each other. While methods for nonlinear two-view representation learning (Deep CCA, (Andrew et al., 2013)) and linear many-view representation learning (Generalized CCA (Horst, 1961)) exist, DGCCA is the first CCA-style multiview representation learning technique that combines the flexibility of nonlinear (deep) representation learning with the statistical power of incorporating information from many independent sources, or views. We present the DGCCA formulation as well as an efficient stochastic optimization algorithm for solving it. We learn DGCCA representations on two distinct datasets for three downstream tasks: phonetic transcription from acoustic and articulatory measurements, and recommending hashtags and friends on a dataset of Twitter users. We find that DGCCA representations soundly beat existing methods at phonetic transcription and hashtag recommendation, and in general perform no worse than standard linear many-view techniques.
The proposed method is simple and elegant; it builds upon the huge success of gradient based optimization for deep non-linear function approximators and combines it with established (linear) many-view CCA methods. A major contribution of this paper is the derivation of the gradients with respect to the non-linear encoding networks which project the different views into a common space. The derivation seems correct. In general this approach seems very interesting and I could imagine that it might be applicable to many other similarly structured problems. The paper is well written; but it could be enhanced with an explicit description of the complete algorithm which also highlights how the joint embeddings G and U are updated. I don’t have prior experience with CCA-style many-view techniques and it is therefore hard for me to judge the practical/empirical progress presented here. But the experiments seem reasonable convincing; although generally only performed on small and medium sized datasets. Detailed comments: The colours or the sign of the x-axis in figure 3b seem to be flipped compared to figure 4. It would be nice to additionally see a continuous (rainbow-coloured) version for Figures 2, 3 and 4 to better identify neighbouring datapoints; but more importantly: I’d like to see how the average reconstruction error between the individual network outputs and the learned representation develop during training. Is the mismatch between different views on a validation/test-set a useful metric for cross validation? In general, it seems the method is sensitive to regularization and hyperparameter selection (because it has many more parameters compared to GCCA and different regularization parameters have been chosen for different views) and I wonder if there is a clear metric to optimize these.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Deep Generalized Canonical Correlation Analysis
We present Deep Generalized Canonical Correlation Analysis (DGCCA) – a method for learning nonlinear transformations of arbitrarily many views of data, such that the resulting transformations are maximally informative of each other. While methods for nonlinear two-view representation learning (Deep CCA, (Andrew et al., 2013)) and linear many-view representation learning (Generalized CCA (Horst, 1961)) exist, DGCCA is the first CCA-style multiview representation learning technique that combines the flexibility of nonlinear (deep) representation learning with the statistical power of incorporating information from many independent sources, or views. We present the DGCCA formulation as well as an efficient stochastic optimization algorithm for solving it. We learn DGCCA representations on two distinct datasets for three downstream tasks: phonetic transcription from acoustic and articulatory measurements, and recommending hashtags and friends on a dataset of Twitter users. We find that DGCCA representations soundly beat existing methods at phonetic transcription and hashtag recommendation, and in general perform no worse than standard linear many-view techniques.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The proposed method is simple and elegant; it builds upon the huge success of gradient based optimization for deep non-linear function approximators and combines it with established (linear) many-view CCA methods. A major contribution of this paper is the derivation of the gradients with respect to the non-linear encoding networks which project the different views into a common space. The derivation seems correct. In general this approach seems very interesting and I could imagine that it might be applicable to many other similarly structured problems. The paper is well written; but it could be enhanced with an explicit description of the complete algorithm which also highlights how the joint embeddings G and U are updated. I don’t have prior experience with CCA-style many-view techniques and it is therefore hard for me to judge the practical/empirical progress presented here. But the experiments seem reasonable convincing; although generally only performed on small and medium sized datasets. Detailed comments: The colours or the sign of the x-axis in figure 3b seem to be flipped compared to figure 4. It would be nice to additionally see a continuous (rainbow-coloured) version for Figures 2, 3 and 4 to better identify neighbouring datapoints; but more importantly: I’d like to see how the average reconstruction error between the individual network outputs and the learned representation develop during training. Is the mismatch between different views on a validation/test-set a useful metric for cross validation? In general, it seems the method is sensitive to regularization and hyperparameter selection (because it has many more parameters compared to GCCA and different regularization parameters have been chosen for different views) and I wonder if there is a clear metric to optimize these.
assistant
The proposed method is simple and elegant; it builds upon the huge success of gradient based optimization for deep non-linear function approximators and combines it with established (linear) many-view CCA methods. A major contribution of this paper is the derivation of the gradients with respect to the non-linear encoding networks which project the different views into a common space. The derivation seems correct. In general this approach seems very interesting and I could imagine that it might be applicable to many other similarly structured problems. The paper is well written; but it could be enhanced with an explicit description of the complete algorithm which also highlights how the joint embeddings G and U are updated. I don’t have prior experience with CCA-style many-view techniques and it is therefore hard for me to judge the practical/empirical progress presented here. But the experiments seem reasonable convincing; although generally only performed on small and medium sized datasets. Detailed comments: The colours or the sign of the x-axis in figure 3b seem to be flipped compared to figure 4. It would be nice to additionally see a continuous (rainbow-coloured) version for Figures 2, 3 and 4 to better identify neighbouring datapoints; but more importantly: I’d like to see how the average reconstruction error between the individual network outputs and the learned representation develop during training. Is the mismatch between different views on a validation/test-set a useful metric for cross validation? In general, it seems the method is sensitive to regularization and hyperparameter selection (because it has many more parameters compared to GCCA and different regularization parameters have been chosen for different views) and I wonder if there is a clear metric to optimize these.