Unsupervised Transcription of Piano Music
We present a new probabilistic model for transcribing piano music from audio to a symbolic form. Our model reflects the process by which discrete musical events give rise to acoustic signals that are then superimposed to produce the observed data. As a result, the inference procedure for our model naturally resolves the source separation problem introduced by the the piano's polyphony. In order to adapt to the properties of a new instrument or acoustic environment being transcribed, we learn recording-specific spectral profiles and temporal envelopes in an unsupervised fashion. Our system outperforms the best published approaches on a standard piano transcription task, achieving a 10.6% relative gain in note onset F 1 on real piano audio.
The paper focuses on the important problem of polyphonic transcription in music information analysis/retrieval. The paper suggest a novel generative model with inspiration form piano music; however, the model might be generalized to other instruments, or maybe multiple instrument tracks. An (unsupervised) inference algorithm is developed, and the method is compared to state-of-the-art methods on the MAPS corpus data. Superior performance is demonstrated both in onset and frame-level evaluation metrics. Despite issues mentioned in detailed comments below, the paper is a solid and significant contribution that might be of interest to the NIPS community.Quality:The paper is relevant and well-written and describe in good detail a novel generative piano music model and arguments for most of the involved model ingredients (some issues mentioned in detailed comments). The model is sufficiently linked and contrasted to existing state-of-art techniques. The methods benefits from being unsupervised and has some generalizability to new instruments although this has only been tested in a rather limited setting. The performance as regards onset and frame-level metrics is superior to state-of-art techniques and the error patterns is argued to bee musically plausible despite the fact that such issues have not been directly been addressed in the model. Whether this is a unique feature of the proposed framework (over alternative methods) is, however, not discussed. The overall conclusion is that the paper is a significant contribution to the community and might inspire further development. Issues and needed clarifications are addressed in the detailed comments below.Clarity: • The paper is well-written and clearly formulated except issues addressed below. It is not possible to do exact reproduction of the results without more details - in particular as regards the experiments.• The references are relevant and adequate.               Significance:• The technical contribution seems sufficiently interesting to a part of the NIPS community related to audio/music, and to sequential models in general. The modeling framework is not complete as the adaption to novel un-seen instruments is not explicitly addressed (except implicit general nature noise models). Further, musical constraints which might help producing perceptually acceptable errors is not addressed. Finally only a single but feasible inference scheme is suggested, hence, the potential benefits of more advanced inference methods is not elicited.Originality:• The overall modeling framework is original and tries to include more aspects of the generation of the oberserved music spectrum. Detailed comments: • Line 033: test data > training data• Last part of intro lines 087-091 should go to the conclusion section.• Line 104 (and other places): loudness is a subjective/perceptual quality. Better to use amplitude/ intensity / strength.• Line 181: activation will remain low? Do you mean low or zero?• Line 185: Gaussian noise distribution. What is this stochastic variable going to model? How do you handle potential negative activations?• Sec 2.3: What is the noise in the spectrum going to represent?• Sec 2.3: Do you have any evidence for the claim that Poisson is the best noise model? Maybe refer to an available technical report.• How is the priors on alpha and sigma going to be constructed, and how important/sensitive is the method to such choices? Have you considered music genre/style variations?• Iterated conditional mode is the chosen as theinference method. Have you considered alternatives, and what are the arguments for sufficiency in the present case.• Sec 3.1. Some details on inferring M is missing. Provide a complete description.• How are the hyper-parameters beta, gamma and xi specified?• The experimental section is rather brief. For reproducibility reasons it would be relevant to refer to a technical report on the details.• The difference between ENSTDkAm and ENSTDkC1 is ambient or close-in recording of the Yamaha Diskklavier. Hence they might be rather similar! Please make clear that the generalizability to a bigger variety of pianos is unclear.• Why is 23ms (actually 23,22ms) a good window size?• Why is 50ms used in evaluation of onsets. 50ms corresponds to 1200 bpm and notes played above this rate might perceptually sound as a single tone. That could be part of the explanation.• Explain in more detail the music possible errors in line 402. Do the alternative methods have same desirable property?• Lines 418-420: Observed errors are claimed not to affect the re-synthesized audio. What is the evidence? How could you take such tolerable error patterns into consideration in the modeling framework?• The generalizability of the model to other instruments or maybe polyphonic music as such is not addressed. Please include some discussion.• Some references seem to be incomplete and ordering of text e.g. 'In Proceedings' should be reversed.