Representation learning is typically applied to only one mode of a data matrix, either its rows or columns. Yet in many applications, there is an underlying geometry to both the rows and the columns. We propose utilizing this coupled structure to perform co-manifold learning: uncovering the underlying geometry of both the rows and the columns of a given matrix, where we focus on a missing data setting. Our unsupervised approach consists of three components. We first solve a family of optimization problems to estimate a complete matrix at multiple scales of smoothness. We then use this collection of smooth matrix estimates to compute pairwise distances on the rows and columns based on a new multi-scale metric that implicitly introduces a coupling between the rows and the columns. Finally, we construct row and column representations from these multi-scale metrics. We demonstrate that our approach outperforms competing methods in both data visualization and clustering.
Review for CO-MANIFOLD LEARNING WITH MISSING DATA
Summary:
This paper proposes a two-stage method to recovering the underlying structure of a data manifold using both the rows and columns of an incomplete data matrix. In the first stage they impute the missing values using their proposed co-clustering algorithm and in the second stage they propose a new metric for dimension reduction.
The overall motivation for how they construct the algorithm and the intuition behind how all the pieces of the algorithm work together are not great. The paper also has significant specific clarity issues (listed below). Currently these issues seem to imply the proposed algorithm has significant logic issues (mainly on the convex/concave confusions); however depending on how they are addressed, this may end up not being an issue. The experimental results for the two simulated datasets look very good. However for the lung dataset, the results are less promising and it is less clear of the advantage of the proposed algorithm to the two competing ones. 
Novelty/Significance:
The overall idea of the algorithm is sufficiently novel. It is very interesting to consider both rows and column correlations. Each piece of the algorithm seems to draw heavily on previous work; bi-clustering, diffusion maps, but overall the idea is novel enough. The algorithm is significant in that it addresses a relatively open problem that currently doesn’t have a well established solution.
Questions/Clarity:
Smooth is not clearly defined and not an obvious measure for a matrix. Figure 1 shows smooth matrices at various levels, but still doesn’t define explicitly what smoothness is. Does smoothness imply all entries are closer to the same value? 
 “Replacing Jr(U) and Jc(U) by quadratic row and column Laplacian penalties” – The sentence is kind of strange as Laplacian penalties is not a thing. Graph Laplacian can be used as an empirical estimate for the Laplace Beltrami operator which gives a measure of smoothness in terms of divergence of the gradient of a function on a manifold; however the penalty is one on a function’s complexity in the intrinsic geometry of a manifold. It is not clear how the proposed penalty is an estimator for the intrinsic geometry penalty. It seems like the equation that is listed is just the function map Omega(x) = x^2, which also is not a concave function (it is convex), so it does not fit the requirements of Assumption 2.2.
Proposition 1 is kind of strangely presented. At first glance, it is not clear where the proof is, and it takes some looking to figure out it is Appendix B because it is reference before, not after the proposition. Or it might be more helpful if it is clearly stated at the beginning of Appendix B that this is the proof for Proposition 1.
The authors write: “Missing values can sabotage efforts to learn the low dimensional manifold underlying the data. … As the number of missing entries grows, the distances between points are increasingly distorted, resulting in poor representation of the data in the low-dimensional space.” However, they use the observed values to build the knn graph used for the row/column penalties, which is counter-intuitive because this knn graph is essentially estimating a property of a manifold and the distances have the same distortion issue.
Why do the author’s want Omega to be concave functions as this makes the objective not convex. Additionally the penalty sqrt(|| ||_2) is approximately doing a square root twice because the l2-norm already is the square root of the sum of squares. Also what is the point of approximating the square root function instead of just using the square root function? It is overall not clear what the nature of the penalty term g2 is; Appendix A, implies it must be overall a convex function because of the upper bound.
Equation 5 is not clear that it is the first order taylor approximation. Omega’ is the derivative of the Omega function? Do the other terms cancel out? Also what is the derivative with respect to; each Ui. for all Uj. ?
 “first-order Taylor approximation of a differentiable concave function provides a tight bound on the function” – Tight bound is not an appropriate term and requires being provable. Unless the function is close to linear, a first order Taylor approximation won’t be anything close to tight.
The authors state the objective in 1 is not convex. Do they mean it is not strictly convex? In which case, by stationary points, they are specifically referring to local minima? Otherwise, what benefits does the MM algorithm have on an indefinite objective i.e. couldn’t you end up converging to a saddle point or a local maxima instead of a local minima, as these are all fixed points. 
It is not clear what the sub/super scripts l, k mean. Maybe with these defined, the proposed multi-scale metric would have obvious advantages, but currently it is not clear what the point of this metric is.
Figure 4 appears before it is mentioned and is displayed as part of the previous section.
For the Lung data, it does not look like the proposed algorithm is better than the other two. None of the algorithms seem to do great at capturing any of the underlying structure, especially in the rows. It also is not super clear that the normal patients are significantly further from the cancer patients. Additionally are the linkage results from figure 3 from one trial? Without multiple trials it is hard to argue that this not just trial noise.
How big are N1 and N2 in the linkage simulations. The Lung dataset is not very large, and it seems like the proposed algorithm has large computation complexity (it is not clear). Will the algorithm work on even medium-large sized matrices (10^4 x 10^4)?