Latent Gaussian Activity Propagation: Using Smoothness and Structure to Separate and Localize Sounds in Large Noisy Environments
We present an approach for simultaneously separating and localizing multiple sound sources using recorded microphone data. Inspired by topic models, our approach is based on a probabilistic model of inter-microphone phase differences, and poses separation and localization as a Bayesian inference problem. We assume sound activity is locally smooth across time, frequency, and location, and use the known position of the microphones to obtain a consistent separation. We compare the performance of our method against existing algorithms on simulated anechoic voice data and find that it obtains high performance across a variety of input conditions.
This paper describes a system for localization and separation of sound sources recorded by distributed microphone arrays.  It is similar to MESSL (Mandel et al, 2009) in its building blocks, but builds upon it by employing a Gaussian process prior to encourage smoothness in time, frequency, and location.  It also is targeted to the same application as DALAS (Dorfan et al, 2015) of microphones being distributed in pairs around a large space, but adds the smoothness over time and frequency via the Gaussian process prior.  It is evaluated on synthetic sound mixtures that are anechoic, but contain a reasonable amount of additive uncorrelated noise.  Its performance is quantified using a new metric that compares the masks it estimates at each microphone to the ideal ratio mask at each microphone.  Results show that it generally provides a higher level of "power precision" compared to other systems while achieving a similar level of "power recall".   Quality:  This paper is thorough in its description, derivations, and evaluation.  The targeted situation is a little contrived, but the same system seems like it should be able to apply straightforwardly to other situations.  The approach is sound and introduces a very nice and flexible framework for incorporating prior assumptions into this model.  Evaluation is carried out carefully and shows good results for the proposed system.  One relatively minor shortcoming of the paper is that it is not clear that new metrics are necessary for evaluating the performance of this system, and make it slightly harder to contextualize the results.  This is especially true with the introduced precision/recall-style metrics which do not provide a single figure of merit with which to compare systems.  There are several methods that would be appropriate for comparing synthetic multi-channel inputs and outputs, such as BSS_EVAL or PEASS.  Single-channel metrics such as STOI might also be used by finding the best microphone signal to analyze, as was done with the MESSL results in the current paper.   There are a improvements to the overall approach that could be addressed in subsequent work:  It would also be interesting to use the derived masks to perform beamforming, which would produce a single-channel estimate of each source that could be compared to the original using STOI or subjective evaluations.  It would be more realistic to use diffuse noise rather than noise that is uncorrelated across sensors.  Diffuse noise is correlated at low frequencies, see for example  Habets and Gannot (2007).   Habets, E. A., & Gannot, S. (2007). Generating sensor signals in isotropic noise fields. The Journal of the Acoustical Society of America, 122(6), 3464-3470.    Clarity:  It is well written and generally easy to follow the logic, although many of the details have been compressed to save space, especially in the model description and derivation in section 2.  Since the paper provides a sophisticated combination of graphical modeling with microphone array processing, such details could be useful for readers from each of those communities to better understand the other, but it is understandable given the page limit.  One specific point that could improve the clarity would be describing the details of the kernels used in equations (1) and (2) adjacent to those equations instead of in the experiments section.  The baseline algorithm also performs surprisingly well, for example in figure 2.  It would be useful and interesting to describe it in more detail.   Minor clarity issues/questions: * On line 138, should C have a subscript associated with it, i.e., should there be different C's for different sources? * On line 141, why is only beta normalized? * On line 149, in most of the paper t_1 appears to be a time index at the frame level, but here it appears to be at the sample level, could this be clarified? Is this smoothing happening between frames? What w(.) function was used in the experiments?    Originality:  The paper does a good job of describing related work and directly comparing it to the proposed system.  In fact, some description of the previous work could perhaps be compressed, since it is discussed in the main introduction, the Prior Work section, and the Experiments section.  The proposed approach provides a very nice framework for extending MESSL that can be built upon in the future with additional assumptions and components.    Significance:  The proposed system provides a interesting improvement of the MESSL approach and paves the way for further improvements.  Empirically it seems to show good performance in the setting in which it was evaluated.   Response to author feedback:  I agree with much of what the authors say in their response, but do not fully agree that SIR and SDR only focus on "precision" of source separation.  While there are several incompatible definitions of SIR and SDR in the literature, there are certain existing definitions, for example SNR_ME in [A] that combine both precision and recall by measuring the ratio of the target speech present in the output (precision) to the combination of target speech energy absent from the output (recall) and the noise energy present in the output.  BSS_EVAL, while widely used, does in fact focus on precision to the exclusion of recall.   [A] Mandel, M. I., Bressler, S., Shinn-Cunningham, B., & Ellis, D. P. (2010). Evaluating source separation algorithms with reverberant speech. IEEE Transactions on audio, speech, and language processing, 18(7), 1872-1883.