A State-Space Model for Decoding Auditory Attentional Modulation from MEG in a Competing-Speaker Environment
Humans are able to segregate auditory objects in a complex acoustic scene, through an interplay of bottom-up feature extraction and top-down selective attention in the brain. The detailed mechanism underlying this process is largely unknown and the ability to mimic this procedure is an important problem in artificial intelligence and computational neuroscience. We consider the problem of decoding the attentional state of a listener in a competing-speaker environment from magnetoencephalographic (MEG) recordings from the human brain. We develop a behaviorally inspired state-space model to account for the modulation of the MEG with respect to attentional state of the listener. We construct a decoder based on the maximum a posteriori (MAP) estimate of the state parameters via the Expectation-Maximization (EM) algorithm. The resulting decoder is able to track the attentional modulation of the listener with multi-second resolution using only the envelopes of the two speech streams as covariates. We present simulation studies as well as application to real MEG data from two human subjects. Our results reveal that the proposed decoder provides substantial gains in terms of temporal resolution, complexity, and decoding accuracy.
In the last few years, there has been considerable interest in the neuroscience community in decoding high-level latent brain states from noisy neural recordings. One such state variable is attention: primates and other high-level organisms can preferentially distribute resources to encode and process a selective set of incoming stimuli in a way that is typically not externally visible. In the case of auditory selective attention, empirical studies have identified a set of neural variables that can be measured that provide information about which sound a subject is currently directing attention to. Some of these variables can be measured via magnetoencephalography (MEG). This study capitalises on these prior observations to build a statistically-principled decoder of human auditory attentional states under competing-speaker auditory stimulation. The authors explain the underlying measurements, represent the problem as one of Bayesian inference, then present an EM-based inference procedure for inferring the latent variables. The authors demonstrate the performance of this algorithm on simulated data, then test it on real MEG data where subjects were directed to attend in a specified way.At a high level, this is a well-motivated study on a topical problem, and the authors have executed it well.Quality: As far as I can tell, the inference procedure is technically sound. The authors demonstrate its application to synthetic data where ground truth is known, and they are able to recover the ground truth reasonably well. The results on real data are reasonably convincing, insofar as these are very limited and noisy measurements. It would be useful if the authors added a comment on how to relate the SNRs in the synthetic data with the real data, given that the noise in the real MEG signals (Fig 3, top two rows of each panel) are rather high.Clarity: Generally, this is well explained, though there were a few points in the text which could have been better motivated :-    - There's some confusion on spectrotemporal vs temporal features of speech streams in this paper. It is true that some previous work uses spectrotemporal information from the speech stream (e.g. Mesgarani & Chang, Ding and Simon). The model presented in this paper only makes use of temporal information. The authors do make this clear initially. But Line 49 refers to Fig 1A as being about the spectrotemporal information (while the image only shows the temporal information). On Line 71, the authors wonder whether one can decode using a "more parsimonious set of spectrotemporal features", but really they are asking whether they can rely on temporal features alone. On Line 94, the authors claim they will decode the attentional state given the spectrotemporal features of the two speech streams, but only do so in terms of the temporal speech streams.    - DSS: I'm not an expert on this denoising technique, but describing the method as "removing inconsistent temporal components that are not phase-locked to the stimulus" sounds an awful lot like it would suppress markers of attentional shifts (which are not typically stimulus-locked). I suspect that this is just a poor choice of wording.    - Section 2.2.1. It really wasn't clear to me until much later on that this section is describing the likelihood. This is perhaps the most confusing part of the paper. The authors should clarify that that the generative model encapsulated in equation (3) is never inverted directly, but rather tested via the surrogate, theta. The authors do need to explain carefully why they rely on this surrogate measure, and not just, for example, consider $y_{t,r}$ as conditionally Gaussian with mean $e_{i,t}$. My suspicions are either that there are slow gain fluctuations in the neural signal, due to changes in global states such as arousal, as well as stimulus-driven gain changes, which the correlation "detrends", or it introduces a nonlinearity that assists in the temporal smoothing. Whether it is because of these or other reasons, the authors should explain why.    - It was not clear to me why the von Mises (prior/posterior) distributions on theta (e.g. eqn (8)) should be zero-mean/mode. Figure 2B of Ding & Simon 2012 suggest that the correlations (across time) between $e_{i,t}$ and $y_{t,r}$ should be positive for both attended and unattended streams. Originality: This provides a more rigorous and principled approach to auditory attentional-state decoding than the other papers I have seen on this topic (e.g. by Ding & Simon, Mesgarani & Chang), and would likely be a welcome step forward in this domain. Some references would be appreciated for the statement "Previous studies employ time-averaging..." on Line 50. Significance: This kind of decoding has a strong practical component for scientists interested in studying attentional states, and engineers developing tools for interacting with the attending brain. There's clearly a fair way to go on this topic still (the performance of this algorithm is ultimately limited), but it's a good step forward.Some additional comments:- Line 125. If the authors have room, they should insert a paragraph break before "In a competing-speaker environment."- Line 110: "It is known that the TRF is a sparse filter". The TRF is not known, but estimated. For future work, the authors may consider a different regularisation scheme -- I suspect that the TRF is better described as temporally localised. The components of Fig 1C after 250ms look very artefactual, and are unrealistic given the functional characteristics of the auditory system. The authors may wish to consider Park & Pillow's ALD as an alternative method for estimating tau.- Lines 169-171 "The subjective experience... by the listener." This is irrelevant, and should be cut. - Line 416. It's a bit rich to blame the variability in estimated switching time on the subject. It's certainly possible, but it would be more parsimonious to blame the algorithm. If the authors want to comment on this, they should sample from the individual-trial posteriors $p_{k,r}$ to estimate the strength of this prediction. Also, it appears to be that the model's assumption of slowly-varying attentional state (via the AR1 dynamics on $z_k$) breaks down under the instruction that the subject switch attention during the pause. The authors should note this.- I suggest changing the Bernoulli success probability symbol from $p_k$ to $q_k$, to disambiguate from the pdf symbol $p$. Also equation (10) should start with $p$