GP Kernels for Cross-Spectrum Analysis
Multi-output Gaussian processes provide a convenient framework for multi-task problems. An illustrative and motivating example of a multi-task problem is multi-region electrophysiological time-series data, where experimentalists are interested in both power and phase coherence between channels. Recently, Wilson and Adams (2013) proposed the spectral mixture (SM) kernel to model the spectral density of a single task in a Gaussian process framework. In this paper, we develop a novel covariance kernel for multiple outputs, called the cross-spectral mixture (CSM) kernel. This new, flexible kernel represents both the power and phase relationship between multiple observation channels. We demonstrate the expressive capabilities of the CSM kernel through implementation of a Bayesian hidden Markov model, where the emission distribution is a multi-output Gaussian process with a CSM covariance kernel. Results are presented for measured multi-region electrophysiological data.
The authors propose a flexible and interpretable kernel (the CSM kernel), building on spectral mixture kernels, for learning relationships between multiple tasks.The starting point is to use Gaussian processes with 1 component spectral mixture kernels as the basis functions in a linear model of coregionalisation (SM-LMC). However, SM-LMC does not contain information about the phases between channels.Thus the authors propose the cross spectral mixture kernel, which mixes phase shifted versions of spectral mixture kernels across channels.The resulting kernel is interpretable and flexible. The authors note that if they have an evenly spaced temporal grid, this gives rise to covariance matrix structure which can be exploited for fast computations.In particular, they approximate their covariance matrices with circulant matrices, and use discrete Fourier transforms to speed up inference and learning.They implement the CSM kernel as part of a Bayesian HMM, and apply the model to a neuroscience application. The paper proposes a useful technically sound method, with a compelling application.Detailed comments:- A generalized version of the "SM-LMC" was discussed in chapter 4.4, p. 137 of Wilson [2014], "Covariance kernels for fast automatic pattern discovery and extrapolation with Gaussian processes", PhD thesis.In particular, it was proposed to model $\bm{y}(x) = W(x) \bm{f}(x)$, where $\bm{y}(x)$ are the responses, $\bm{f}(x)$ is a vector of GPs each with 1 component SM kernels, and W(x) is an input dependent matrix of mixing weights.Setting W(x) = W, one recovers the SM-LMC.    Quality:The paper is technically sound, and the application is very compelling.I would expect, however, the computational complexity to be O(C^3 N log N) rather than O(C^3 N).An alternative to the proposed approach would be to exploit the Toeplitz structure of your matrices and perform linear conjugate gradients, which could be exact to within numerical precision, but somewhat slower than your approach.Evaluating the log determinant would require O(N^2) computations and O(N) memory. By following an inference strategy similar to "Gaussian Process Regression Networks", ICML, you may be able to further reduce your complexity to O(C N).In general, I was a little unsure of how much the scalability of the proposed approach was necessary for this application.It would help to clarify this in a final version. ClarityThe paper was a pleasure to read.  OriginalityThere is valuable novelty in decoupling the phase shifts between different tasks, as proposed in the CSM kernel, which usefully builds on the SM kernel.   SignificanceI can easily imagine others finding this methodology widely applicable.I really enjoyed the neuroscience application, and only wish the paper went into this application more deeply, with more interpretation of results.It's very exciting that we can use these kernel learning approaches to gain new significant insights into our data, in addition to improved predictive performance.