MUSIC TRANSFORMER: GENERATING MUSIC WITH LONG-TERM STRUCTURE
Music relies heavily on repetition to build structure and meaning. Self-reference occurs on multiple timescales, from motifs to phrases to reusing of entire sections of music, such as in pieces with ABA structure. The Transformer (Vaswani et al., 2017), a sequence model based on self-attention, has achieved compelling results in many generation tasks that require maintaining long-range coherence. This suggests that self-attention might also be well-suited to modeling music. In musical composition and performance, however, relative timing is critically important. Existing approaches for representing relative positional information in the Transformer modulate attention based on pairwise distance (Shaw et al., 2018). This is impractical for long sequences such as musical compositions because their memory complexity for intermediate relative information is quadratic in the sequence length. We propose an algorithm that reduces their intermediate memory requirement to linear in the sequence length. This enables us to demonstrate that a Transformer with our modified relative attention mechanism can generate minute-long compositions (thousands of steps) with compelling structure, generate continuations that coherently elaborate on a given motif, and in a seq2seq setup generate accompaniments conditioned on melodies 1 . We evaluate the Transformer with our relative attention mechanism on two datasets, JSB Chorales and Maestro, and obtain state-of-the-art results on the latter.
The authors address the problem raised by applying a fully attentional network (FAN) to model music. 
They argue clearly for the need of relational positional embedding in that problem (instead of absolute positional as in vanilla FAN), and highlight the quadratic memory footprint of the current solution (Shaw et al. 2018).
The main contribution of the paper is a solution to this, consisting in a smart idea (sect 3.4.1 and 3.4.2) which allows them to compute relative embeddings without quadratic overhead.
The model performs indeed better than Shaw et al.'s on the single data-set they compared both. On the other one, the argument is that Shaw et al. 2018 cannot be applied because the sequences are too long.
I have two concerns with the paper:
	1/ it is very hard to read at times. In particular, the main contribution took me several passes the understand. I list below a few recommendations for improvement 
	2/ the main argument is that the model requires less memory and is faster. However, the only empirical evidence in that direction is given in the introduction (Sect 1.1., second paragraph).
		The following points remain unclear to me:
			a) why can't the Relative Transformer be applied to Piano-e composition. What is the maximal length that is possible?
			b) how much faster / less memory is the relative music transformers? The only data-point is in Sect 1.1., which seems indeed impressive (but then one wonders why this is not exploited further). A deeper analysis of the comparative memory footprint would greatly strengthen the paper in my opinion.
			
Why "music" relative transformers? Nothing in the model restrict it to that use case. The use of FAN over audio has been explored with limited success, one of the reasons being that - similarly to this use-case here - audio sequences tend to be longer than text.
		
minor comments:
	- abstract, ln9: there seems to be a verb missing
	- p1,ln-2: "dramatic" improvements seems to be exaggerated
	- p2,ln11: "too long". too long for what?
	- p4,ln15: (Table 1). is one sentence by itself. Also, a clear explanation of that table is missing
	- p5,item 2: an explanation in formula would be helpful for those not familiar with reshaping
	- Fig3: it seems very anecdotical. Similar green bloxes might be placed on the left plot
	- sect4.1.1,ln3. that sentence does not parse
	- Table 2: what is cpsi?
	- $l$ is nicer formatted as $\ell$
	- care should be taken to render the Figures more readable (notably the quality of Fig 4, and labels of Fig 7)
	- footnotes in Figures are not displayed (Table 2 and 4)
	- the description of the human evaluation leaves some open questions. I could not come up with 180 ratings (shouldn't it be 180 * 3 ratings?). Also, at least the values of Relative Transformer vs other 3 models should be shown (or all 6 comparisons). Here you call "relative transformer" your model, previously you used that term to refer to (Shaw et al. 2018).
		when reporting statistical significance, there are some omissions which should be clarified.
	- (Shaw et al. 2018) has been published at NAACL. For such an important citation, you should update the reference from the arxiv version.