The ability to represent and compare machine learning models is crucial in order to quantify subtle model changes, evaluate generative models, and gather insights on neural network architectures. Existing techniques for comparing data distributions focus on global data properties such as mean and covariance; in that sense, they are extrinsic and uni-scale. We develop a first-of-its-kind intrinsic and multi-scale method for characterizing and comparing data manifolds, using a lower-bound of the spectral Gromov-Wasserstein inter-manifold distance, which compares all data moments. In a thorough experimental study, we demonstrate that our method effectively discerns the structure of data manifolds even on unaligned data of different dimensionality, and showcase its efficacy in evaluating the quality of generative models.
1. Define/explain data manifold: in the abstract, ``data moments" suggests the data is treated as random variables; the authors could explain more on how they make the connection between random variables and manifolds 
2. introduction: include an example of ``models having different representation space" 
3. introduction: elaborate more on ``unaligned data manifolds" 
4. introduction: the distinction between ``intrinsic" and ``extrinsic" can be made more explicit; as currently written, the difference between ``extrinsic"/``multi-scale"/``utilizing higher moments" isn't clear 
5. the authors mentioned the hypothesis that high-dimensional data lies on a low-dimensional manifold, but in (3.1) they only considered $\mathbb{R}^d$ without justifying this restriction 
6. (3.1) ``$u(t, \cdot) \to \delta(\cdot -x)$" change the dot to a variable 
7. (3.1) ``$u(t, x) = 0 \forall x \in \partial X$ and for all $t$" change to ``for all $t$ and $x \in \partial X$ 
8. equation 1, write ``we restrict our exposition to Euclidean spaces $X = \mathbb{R}^d$" and remove ``$\forall x, x' \in \mathbb{R}^d, t \in \mathbb{R}^+$" 
9.  (3.1) the phrase ``a compact X including $\mathbb{R}^d$" is confusing
    as $\mathbb{R}^d$ is not compact under Euclidean topology 
10.  (3.1) ``for a local domain with Dirichlet condition $D$" change to ``for a local domain $D$ with Dirichlet condition" 
11.  (3.1) ``Definition 1" is more of a proposition/theorem. 
12.  (3.1) notation: in the body paragraph the authors use $k(x, x', t)$ but in ``definition 1" they used $k(x, y, t)$ 
13. (3.1) ``$c$ disconnected components" to ``$c$ connected components" 
14. (3.1) the authors could elaborate more about how HKT contains all the information in the graph's spectrum. Currently, they only stated the limit equal to the number of connected components. 
15. Github code is helpful but it would be better if they can include an algorithm section
16. A figure of the architecture could be included under the ``putting IMD together" section