Brains on Beats
We developed task-optimized deep neural networks (DNNs) that achieved state-ofthe-art performance in different evaluation scenarios for automatic music tagging. These DNNs were subsequently used to probe the neural representations of music. Representational similarity analysis revealed the existence of a representational gradient across the superior temporal gyrus (STG). Anterior STG was shown to be more sensitive to low-level stimulus features encoded in shallow DNN layers whereas posterior STG was shown to be more sensitive to high-level stimulus features encoded in deep DNN layers.
This paper performs a RSA similarity analysis assessment of the similarity of trained DNN to human fMRI traces, using music samples as stimuli.  This type of approach has been highly successful in the visual domain to improve on the understanding of the visual hierarchy, finding great correlations between DNN trained on ImageNet and human/monkey IT.  The authors replicate this using music datasets (MagnaTagaTune to train the DNN, Studyforrest for the RSA).  Their DNN models trained on MagnaTagaTune obtain SOTA classification accuracy. They then find that DNN layers representations similarity seems to go from anterior to posterior along the STG, providing some hints about the salient features represented in STG.
This paper has 2 main contributions: 1) Improve the SOTA on DNN for music tag classification. 2) Look for correlations with the human STG through a RSA + Searchlight procedure.  The treatment of 1) is good and well explained. In Section 2.3, it would be more helpful to spell out directly the number of filters, along with sizes and strides per layers, instead of referring to AlexNet and providing the set of rules to adapt them...  A small figure would be more space efficient. All the latest tricks and techniques were used, there were no flawed in my opinion.  Compared to the previous state of the art in music classification using DNN, it seems like the main difference is a drastic change in filter sizes and using the latest optimizers. If I parsed the methods properly, they use conv filter sizes {121, 25, 9, 9, 9} (strides similarly adapted), compared to {8, 8} in [32]. This is quite a drastic improvement, which might explain the SOTA classification performance. Still, great accomplishment!  The authors do not comment on the low performance for "no vocals" types of labels, it would be interesting to address it (my guess would be that this is a "negation" label which has huge support and variance and is hard to classify from 6sec clips).  Considering contribution 2), it's a good replication of the work performed in the visual domain by several other groups (cited appropriately). The techniques used are sound and SOTA (RSA + Searchlight), but I do not have the knowledge to assess the specificities of the fMRI analysis. The baseline models used for comparison are sound and a welcome addition.  It would have been helpful to provide the RSA matrices of the fMRI data itself, in order to visually compare to Figure 3, as is usually done in the visual DNN-RSA literature (if I missed it in the published papers I apologize, but couldn't find it in [23]). Given the strong ambient/symphonic vs rest subdivision, regressing that effect out would have been interesting (similar to [13] for animate vs inanimate).  I'm not sure how to interpret the pretty low mean representational similarity in Figure 5A when going posterior along the STG. This doesn't make me want to trust the significance of the (layer 6 ; posterior STG) correlation much. Similarly, the drop in similarity as the layer index increases in Figure 4 is a bit contrary to what people have found when looking at IT correlations... It would seem that these results only show that early layers processing are similar to STG, but not later ones. The paper is a bit lacking in interpretations and discussions on that aspect.  Given what is known about Wernicke's area, I would have liked to see style-conditioned similarities: did music genres with vocals representations significantly differ between hemispheres?  Overall, it seems like part 2) is less discussed and lacks a bit of interpretation and context to be really useful for NIPS-attendees. But overall I think it is a strong paper which will find its audience.