This work describes a novel recurrent model for music composition, which accounts for the rich statistical structure of polyphonic music. There are many ways to factor the probability distribution over musical scores; we consider the merits of various approaches and propose a new factorization that decomposes a score into a collection of concurrent, coupled time series: "parts." The model we propose borrows ideas from both convolutional neural models and recurrent neural models; we argue that these ideas are natural for capturing music's pitch invariances, temporal structure, and polyphony.
We train generative models for homophonic and polyphonic composition on the KernScores dataset (Sapp, 2005), a collection of 2,300 musical scores comprised of around 2.8 million notes spanning time from the Renaissance to the early 20th century. While evaluation of generative models is known to be hard (Theis et al., 2016), we present careful quantitative results using a unit-adjusted cross entropy metric that is independent of how we factor the distribution over scores. We also present qualitative results using a blind discrimination test.
PROs
-seemingly reasonable approach to polyphonic music generation: figuring out a way to splitting the parts, share parameters appropriately, measuring entropy per time, all make sense
-the resulting outputs tend to have very short-term harmonic coherence (e.g. often a ‘standard chord’ with some resolving suspensions, etc), with individual parts often making very small stepwise motion (i.e. reasonable local voice leading)
-extensive comparison of architectural variations
-positive results from listening experiments
CONs
-musical outputs are *not* clearly better than some of the polyphonic systems described; despite the often small melodic steps, the individual lines are quite random sounding; this is perhaps a direct result of the short history
-I do not hear the rhythmic complexity that is described in the introduction
-the work by Johnson (2015) (ref. provided below) should be looked at and listened to; it too uses coupled networks, albeit in a different way but with a related motivation, and has rhythmic and polyphonic complexity and sounds quite good (better, in my opinion) 
-some unclear sections (fixable, especially with an appendix; more detail below)
-despite the extensive architectural comparisons, I was not always clear about rationale behind certain choices, eg. if using recurrent nets, why not try LSTM or GRU? (more questions below)
-would like to have heard the listening tests; or at least read more about how samples were selected (again, perhaps in an appendix and additional sample files)
 quality, clarity, originality and significance of this work, including a list of its pros and cons (max 200000 characters).
Quality -- In this work, various good/reasonable choices are made. The quality of the actual output is fine. It is comparable to-- and to my ears not better than-- existing polyphonic systems such as the ones below (links to sample audio are provided here):
-Bachbot - https://soundcloud.com/bachbot (Liang et al 2017)
- tied parallel nets - http://www.hexahedria.com/2015/08/03/composing-music-with-recurrent-neural-networks/ (Johnson 2015, ref below)
-performanceRNN - https://magenta.tensorflow.org/performance-rnn - (Simon & Oore 2017)
..others as well..
Clarity -- Some of the writing is "locally" clear, but one large, poorly-organized section makes the whole thing confusing (details below). It is very helpful that the authors subsequently added a comment with a link to some sample scores; without that, it had been utterly impossible to evaluate the quality. There are a few points that could be better clarified:
	-p5”a multi-hot vector of notes N”. It sounds like N will be used to denote note-numbers, but in fact it seems like N is the total number of notes, i.e. the length of the vector, right? What value of N is used?
-p5 “a one-hot vector of durations D”. It sounds like D will be used to denote durations, but actually I think D is the length of the 1-hot vector encoding durations right? What value of D is used, and what durations do the elements of this vector represent?
-similarly, does T represent the size of the history? This should really be clarified.
	-p5 Polyphonic models.
		-Eq (2), (3), (4): Presumably the h’s are the hidden activations layers?
		-the networks here correspond to the blue circles in Fig 1, right? If so, make the relationship clear and explicit 
		-Note that most variables in most equations are left undefined       
		-actually defining the W’s in Eq(2-4)  would allow the authors to refer to the W’s later (e.g. in Section 5.2) when describing weight-sharing ideas. Otherwise, it’s all rather confusing. For example, the authors could write, “Thus, we can set W_p1 = W_p2 = W_p3 = W_p4” (or whatever is appropriate). 
	-Generally, I found that pages 5-7 describe many ideas, and some of them are individually fairly clearly described, but it is not always clear when one idea is beginning, and one idea is ending, and which ideas can be combined or not. On my first readings, I thought that I was basically following it, until I got to Table 5, which then convinced me that I was in fact *not* quite following it. For example, I had been certain that all the networks described are recurrent (perhaps due to Fig1?), but then it turned out that many are in fact *not* recurrent, which made a lot more sense given the continual reference to the history and the length of the model’s Markov window etc. But the reader should not have had to deduce this. For example, one could write, 
	“We will consider 3 types of architectures: convolutional, recurrent, .... In each architecture, we will have [...] modules, and we will try a variety of combinations of these modules. The modules/components are as follows:”. It’s a bit prosaic, but it can really help the reader. 
-Appendices, presented well, could be immensely helpful in clarifying the exact architectures; obviously not all 22 architectures from Table 5 need to be shown, but at least a few of them shown explicitly would help clarify. For example, in Fig1, the purple boxes seem to represent notes (according to the caption), but do they actually represent networks? If they really do represent notes, then how can “notes” receive inputs from both the part-networks and the global network? Also, I was not entirely clear on the relationship of the architecture of the individual nets (for the parts) to that of the global integrating network. E.g. for experiment #20, the part-net is an RNN (with how many layers?? with regular or LSTM cells?) followed by a log-linear predictor (with one hidden layer of 300 units right? or are there multiple layers sometimes?), but then what is the global network? Why does the longest part-history vector appear to have length 10 based on Table 5, but according to Table 3 the best-performing history length was 20? Though, I am not sure the meaning of the “bottom/top” column was explained anywhere, so maybe I am completely misunderstanding that aspect of the table? Etc.
-Many piano scores do not easily deconstruct into clean 4-part polyphony; the example in Appendix A is an exception. It was not clear to me how piano scores were handled during training. 
-Terminology: it is not entirely clear to me why one section is entitled “homophonic models”, instead of just “monophonic models”. Homophonic music usually involves a melody line that is supported by other voices, i.e. a sort of asymmetry in the part-wise structure. Here, the outputs are quite the opposite of that: the voices are independent, they generally function well together harmonically, and there is usually no sense of one voice containing a melody. If there’s some reason to call it homophonic, that would be fine, but otherwise it doesn’t really serve to clarify anything. However, the authors do say that the homophonic composition tasks are a “minor generalization of classic monophonic composition tasks”, so this suggests to me that there is something here that I am not quite understanding.
The last sentence of Section 5.3 is very confusing-- I don’t understand what lin_n is, or 1_n is, or how to read the corresponding entries of the table. The first part of the paragraph is fairly clear. 
Table 4: “The first row” actually seems like it is referring to the second row. I know what the authors mean, but it is unnecessarily confusing to refer to it in this way. One might as well refer to “the zeroth row” as listing the duration of the clip :)
The experimental evaluation: I would like to hear some of the paired samples that were played for subjects. Were classical score excerpts chosen starting at random locations in the score, or at the beginning of the score? It is known that listening to a 10-second excerpt without context can sometimes not make sense. I would be curious to see the false positives versus the false negatives. Nevertheless, I certainly appreciate the authors’ warning to interpret the listening results with caution.

Originality & Significance -- So far, based both on the techniques and the output, I am not entirely convinced of the originality or significance of this particular system. The authors refer to “rhythmically simple polyphonic scores” such as Bachbot, but I cannot see what is rhythmically fundamentally more sophisticated about the scores being generated by the present system. One nice characteristic of the present system is the true and audible independence of the voices.
One of the contributions appears to be the construction of models that explicitly leverage with shared weights some of the patterns that occur in different “places” (pitch-wise and temporally) in music. This is both very reasonable, and also not an entirely novel idea; see for example the excellent work by Daniel Johnson, “Generating Polyphonic Music Using Tied Parallel Networks” (paper published 2017, first shared online, as far as I know, in 2015: links to all materials available at http://www.hexahedria.com/2015/08/03/composing-music-with-recurrent-neural-networks/  )
Another now common (and non exclusive) way to handle some of this is by augmenting the data with transposition. It seems that the authors are not doing this here. Why not? It usually helps. 
Another contribution appears to be the use of a per-time measure of loss. This is reasonable, and I believe others have done this as well. I certainly appreciated the explicit justification for it, however.
Note that the idea of using a vector to indicate metric subdivision was also used in (Johnson 2015).
Playing through some of the scores, it is clear that melodies themselves are often quite unusual (check user studies), but the voices do stay closely connected harmonically, which is what gives the system a certain aural coherence. I would be interested to hear (and look at) what is generated in two-part harmony, and even what is generated-- as a sort of baseline-- with just a single part. 
I encourage the authors to look at and listen to the work by Johnson:
-listening samples: http://www.hexahedria.com/2015/08/03/composing-music-with-recurrent-neural-networks/
-associated publication: http://www.hexahedria.com/files/2017generatingpolyphonic.pdf
Overall, I think that the problem of generating rhythmically and polyphonically complex music is a good one, the approaches seem to generally be reasonable, although they do not appear to be particularly novel, and the musical results are not particularly impressive. The architectural choices are not always clearly presented.