Optimal spectral transportation with application to music transcription
Many spectral unmixing methods rely on the non-negative decomposition of spectral data onto a dictionary of spectral templates. In particular, state-of-the-art music transcription systems decompose the spectrogram of the input signal onto a dictionary of representative note spectra. The typical measures of fit used to quantify the adequacy of the decomposition compare the data and template entries frequency-wise. As such, small displacements of energy from a frequency bin to another as well as variations of timbre can disproportionally harm the fit. We address these issues by means of optimal transportation and propose a new measure of fit that treats the frequency distributions of energy holistically as opposed to frequency-wise. Building on the harmonic nature of sound, the new measure is invariant to shifts of energy to harmonically-related frequencies, as well as to small and local displacements of energy. Equipped with this new measure of fit, the dictionary of note templates can be considerably simplified to a set of Dirac vectors located at the target fundamental frequencies (musical pitch values). This in turns gives ground to a very fast and simple decomposition algorithm that achieves state-of-the-art performance on real musical data.
In this paper, spectral unmixing problem for music transcription is formulated as a matrix factorization problem like NMF. In contrast to existing approaches, e.g., PLCA that minimizes KL-divergence between an original spectrogram and multiplication of decomposed factors, authors take into account a small displacement in frequency domain and modeled this displacement by using a transportation matrix. The spectral unmixing is formulated as an optimization problem that minimizes a total transportation cost for a given dictionary matrix. This can be regarded as a natural extension of EMD-NMF which is special case of the proposed method when a cost matrix C is a symmetric matrix. The proposed cost function can be efficiently minimized by a closed-form solution if a set of Dirac vectors are used for a dictionary matrix. In the experiments, authors demonstrated that OST with regularization outperformed PLCA in terms of both accuracy and efficiency.
-Technical correctness and novelty The ideas of modeling frequency displacement by a transportation matrix and obtaining the optimal displacement by closed-form solution are very interesting. (But I am not sure if these ideas have already been proposed in preceding studies or not.) It is also interesting that a harmonic nature can be naturally modeled by a cost matrix C. As suggested by authors, much further investigation on training C from data is an interesting direction as a future work. From a critical point of view, authors do not show a certain cases where group regularization is beneficial for real data. As long as seeing section 6.2 and Table 1, OST with entropic regularization is sufficient to deal with real data. Much deeper investigation on group regularization for real-data is desired.  From another aspect, I am a little concerned about the fact that the dictionary matrix is completely hand-crafted and the frequency displacement model is also data-independent. Can OST outperform methods that use data-dependent dictionary that trained from labeled data?  -Usefulness It is very interesting that there exists closed form solution for OST. Thanks to this fact, the proposed method achieves significantly faster processing compared to PLCA. Reviewer thinks that this characteristic is very practical for real applications. On the other hand, I am not sure if OST really outperforms any other preceding works in terms of accuracy, because OST was compared with only a single method, PLCA, which was proposed in 2006 and may be outdated.  -Clarity This paper is quite well written. Although I do not confident in acoustic signal processing (instead, I confident in matrix factorization problem including NMF), it was very easy to understand technical details. The proposed procedure is correctly formulated by mathematical notations. I found a minor mistake in line 3-4 in Section 3. The variable 'N' in the following sentence, i,j=1, ..., N, must be 'M'.  %% After Rebuttal %%   The reviewer thanks to the authors for their detailed comments. I agree that this paper places importance on the theory part rather than experiments for the practical usefulness. This is a very good paper which should be published from NIPS.