Learning from uncertain curves: The 2-Wasserstein metric for Gaussian processes
We introduce a novel framework for statistical analysis of populations of nondegenerate Gaussian processes (GPs), which are natural representations of uncertain curves. This allows inherent variation or uncertainty in function-valued data to be properly incorporated in the population analysis. Using the 2-Wasserstein metric we geometrize the space of GPs with L 2 mean and covariance functions over compact index spaces. We prove uniqueness of the barycenter of a population of GPs, as well as convergence of the metric and the barycenter of their finite-dimensional counterparts. This justifies practical computations. Finally, we demonstrate our framework through experimental validation on GP datasets representing brain connectivity and climate development. A MATLAB library for relevant computations will be published at https://sites.google.com/view/antonmallasto/software.
The paper extends the Wasserstein (earth-mover's) metric from
finite-dimensional Gaussians to GPs, defining a geometry on spaces of
GPs and in particular the notion of a barycenter, or 'average' GP. It
establishes that 2-Wasserstein distances, and barycenters, between GPs
can be computed as the limit of equivalent computations on finite
Gaussians. These results are applied to analysis of uncertain
white-matter tract data and annual temperature curves from multiple
stations.
The writing is serviceable and relevant background is presented
clearly. I found no errors in the results themselves, though did not
check all of them thoroughly. My main concerns are with motivation and
significance. I agree that it is in some sense natural to extend the
Wasserstein geometry on Gaussians to GPs, and it's reassuring to see
that this can be done and (as I understand the results of the paper)
basically nothing surprising happens -- this is definitely a
contribution. But there is not much motivation for why this is a
useful thing to do. The experiments in the paper are (as is standard 
for theory papers) fairly contrived, and the conclusions they arrive 
at (the earth is getting warmer) could likely be gained by many other
forms of analysis --- what makes the Wasserstein machinery
particularly meaningful here?
For example: from a hierarchical Bayesian perspective it would seem
reasonable to model the climate data using a latent, unobserved global
temperature curve f_global ~ GP(m_global, k_global), and then assume
that each station i observes that curve with additional local GP noise,
f_i ~ GP(f_global + mi, ki). Then the latent global temperature curve
could be analytically computed through simple Gaussian conditioning,
would be interpretable in terms of this simple generative model, and
would (correctly) become more certain as we gathered evidence from
additional stations. By contrast I don't understand how to
statistically interpret the barycenter of independently estimated GPs,
or what meaning to assign to its uncertainty (am I correct in
understanding that uncertainty would *not* decrease given additional
observations?  what are the circumstances in which this is desirable
behavior?).
Perhaps other reviewers with a more theoretical bent will find this
work exciting. But in the absence of any really surprising results or
novel theoretical tools, I would like to see evidence that the machinery 
it builds is well-motivated and has real advantages for practical data 
analysis -- can it reach conclusions more efficiently or uncover structure 
more effectively than existing techniques? As currently written the paper
doesn't cross my personal NIPS significance bar. 
misc notes:
- Fig 1: the 'naive mean' column claims to average the pointwise
standard deviations, but the stddevs shown are smaller than *any* of
the example curves: how can this be true? Also why is averaging 
pointwise stddevs, rather than (co)variances or precisions, the relevant
comparison? 
- I wonder if the Wasserstein metric could be useful for variational
fitting of sparse/approximate GP models? Currently this is mostly done
by minimizing KL divergence (as in Matthews et al. 2015,
https://arxiv.org/abs/1504.07027) but one could imagine minimizing the
Wasserstein distance between a target GP and a fast approximating
GP. If that turned out to yield a practical method with statistical or
computational advantages over KL divergence for fitting large-scale
GPs, that'd be a noteworthy contribution.
130-131: missing close-paren in inf_\mu\inP2 (H
152: 'whe' -> 'we'
155: 'This set is a convex'
170: why is 'Covariance' capitalized?
178: 'GDss' -> 'GDs'?