A Locally Adaptive Normal Distribution
The multivariate normal density is a monotonic function of the distance to the mean, and its ellipsoidal shape is due to the underlying Euclidean metric. We suggest to replace this metric with a locally adaptive, smoothly changing (Riemannian) metric that favors regions of high local density. The resulting locally adaptive normal distribution (LAND) is a generalization of the normal distribution to the "manifold" setting, where data is assumed to lie near a potentially low-dimensional manifold embedded in R D . The LAND is parametric, depending only on a mean and a covariance, and is the maximum entropy distribution under the given metric. The underlying metric is, however, non-parametric. We develop a maximum likelihood algorithm to infer the distribution parameters that relies on a combination of gradient descent and Monte Carlo integration. We further extend the LAND to mixture models, and provide the corresponding EM algorithm. We demonstrate the efficiency of the LAND to fit non-trivial probability distributions over both synthetic data, and EEG measurements of human sleep.
In this paper the authors present a locally adaptive normal distribution. Instead of measuring distances via a Euclidean distance they define a Riemannian metric M(x) which varies smoothly across the data set. Given the R. Metric M(x) geodesic distances along the manifold can be calculated via the exponential and logarithmic map. The authors use the Euler-Lagrange equations which define a set of ordinary differential equations for estimating the geodesic distances. The logarithm map takes points from the manifold (y) into the tangent plane (at x) such that the length of the resulting vector is the geodesic distance from x to y along the manifold.   The authors then apply a normal distribution computing a Mahalanobis distance on points mapped by the logarithm map. This inner product is used to compute distances from a point x to the mean instead of Euclidean distances. The authors then derive estimators for the mean and covariance via maximum likelihood. Since the solutions are not available in closed form they present a steepest descent algorithm for estimating them. Finally, since the normalization constant is also not available in closed form the authors present a Monte-Carlo approximation method.  They apply this method to synthetic data generated on a manifold as well as in a clustering task. They compare to the standard mixture of Gaussians.
Overall I feel that the paper provides an interesting and useful extension of the normal distribution which adapts to local manifold structure. The authors combine work in metric estimation, computing geodesic distances, and learning distributions. I believe this work is worthy of publication. I have a few comments on my scores for the above:  Technical quality Comments:  The manifold techniques (using Euler-Lagrange equations, estimating logarithm maps) are sound and well explained. The Monte-Carlo estimation of the normalization constant and steepest descent algorithms are clever and well described. Analysis of the applications is lacking (sigma parameter appears to be picked by optimizing on training data?). Explanation of the usefulness of the learned distribution is also somewhat lacking.  Novelty Comments:  It appears the main contribution of this paper is combining previous work (using Kernels for a metric M(x), estimating geodesic distances via Euler-Lagrange, applying a normal distribution) and providing an algorithm for estimating the parameters (mean, covariance, normalization constant).      Potential Impact Comments:  The authors provide a fair evaluation of the GMM and LAND in the synthetic data (looking at log-likelihood on data generated by the learned distributions) but it's not clear how they chose the sigma parameter. The authors show that the LAND fits well on data generated along a manifold but don't show how well it compares to a GMM when data is generated according to a single normal distribution or GMM.   In the real life (clustering) data they appear to choose the sigma value for the LAND via optimizing the F measure and then present that same F measure. What they should have done (or if they did made this clearer) is provide the F measure on a held-out test set to be fair. It is, therefore, hard to see if there is or is not a problem with over-fitting. It also would have been nice to examine very high dimensional data.  One potentially useful application that the authors present is that the LAND could be used as a generative model for manifold data, however, they make the rather unusual manifold assumption that the data X is observed in D dimensions *and* the manifold dimension is also D (that is, points on the tangent plane are D-dimensional and the Metric M(x) is full rank). In manifold learning we're usually interested in data that lie on a d-dimensional manifold embedded into a D-dimensional Euclidean space where D is much larger than d.          Clarity Comments:  There's one particular notation point I found confusing: the overloading of the Sigma notation. In particular they say M = Sigma^-1 but then estimate it via a (local) diagonal Kernel matrix.  Is this Simga also the same as the (fitted) covariance matrix? If so, is the kernel just an initialization? This would be strange since M(x) should vary with x but the fitted covariance matrix is a single parameter. In the proof of the steepest decent they explicitly say say that "in our case M = Sigma^-1" and use that substitution. Clarification here would be nice.  In Sec 4.2 they don't state but I assume cluster membership was chosen by picking 2 clusters and computing closest (geodesic?) distance to the cluster means.    Overall I felt the manifold introduction was well explained (at least for someone familiar with Riemannian manifolds).