Low-Rank Time-Frequency Synthesis
Many single-channel signal decomposition techniques rely on a low-rank factorization of a time-frequency transform. In particular, nonnegative matrix factorization (NMF) of the spectrogram -the (power) magnitude of the short-time Fourier transform (STFT) -has been considered in many audio applications. In this setting, NMF with the Itakura-Saito divergence was shown to underly a generative Gaussian composite model (GCM) of the STFT, a step forward from more empirical approaches based on ad-hoc transform and divergence specifications. Still, the GCM is not yet a generative model of the raw signal itself, but only of its STFT. The work presented in this paper fills in this ultimate gap by proposing a novel signal synthesis model with low-rank time-frequency structure. In particular, our new approach opens doors to multi-resolution representations, that were not possible in the traditional NMF setting. We describe two expectation-maximization algorithms for estimation in the new model and report audio signal processing results with music decomposition and speech enhancement.
This paper presents a extension in the family of low-rank spectrogram-factorization models of sound decomposition that generalizes the latent factors over an arbitrary (potentially overcomplete) set of bases, meaning that decomposition can be performed on "multiresolution" (e.g. tonal + transient) bases.  This is argued to help with source separation and denoising.  It starts from Gaussian Composite Model of Fevotte et al, which models the variance of each cell of a spectrogram with a low-rank matrix factorization, but then expands the implicit convolution with the multiple gabor impulse responses corresponding to each row of the spectrogram to give a purely time-domain expression for what is being estimated, that can then be modified to work for different sets of impulse responses, including nonorthogonal ones.  An EM (local optimum) analysis procedure is presented.Quality: I think this is an interesting insight, although I'm still trying to figure out the exact implications.  It seems like, at the outset, it's just another way of understanding Fevotte's GCM, but the ability to include overcomplete dictionaries is nice.  On the other hand, it seems like this ought to cause some complications, and I didn't immediately understand why these didn't cause more trouble.  I assume the estimation procedure is correct (and I'm impressed by this kind of analysis) although I couldn't follow it in detail.Clarity: The work is well explained and motivated.  The evaluation, however, was very superficial - we're given one purely illustrative result, and one SNR improvement measurement: I hardly think we can draw any conclusions from a single example.  I was grateful for the included sound examples, but I wasn't all that impressed.  For one thing, the OMLSA enhanced example seems to be much longer (7 sec c/w 2 sec) than the new result, which makes me wonder if the comparison has been done fairly.  Secondly, although OMLSA includes more background noise (in part because it is longer), the speech sounds much higher quality to me.  The denoising of the music example comes at the expense of very significant artifacts - I would expect conventional wavelet shrinkage or weiner filtering to give much higher perceived quality in this case.Originality: The work seems to rely very heavily on the Fevotte et al foundation, although the time-domain interpretation certainly made me think about it in a new way.Significance: On the face of it, the ability to apply matrix-factorization techniques to overcomplete bases seems like it could have significant impact.  However, the examples and evaluation presented in this work were too thin and superficial to really show any of this, so as it stands it is an act of faith only to rate this work as having substantial significance.