Neural network-based methods have recently demonstrated state-of-the-art results on image synthesis and super-resolution tasks, in particular by using variants of generative adversarial networks (GANs) with supervised feature losses. Nevertheless, previous feature loss formulations rely on the availability of large auxiliary classifier networks, and labeled datasets that enable such classifiers to be trained. Furthermore, there has been comparatively little work to explore the applicability of GAN-based methods to domains other than images and video. In this work we explore a GAN-based method for audio processing, and develop a convolutional neural network architecture to perform audio super-resolution. In addition to several new architectural building blocks for audio processing, a key component of our approach is the use of an autoencoder-based loss that enables training in the GAN framework, with feature losses derived from unlabeled data. We explore the impact of our architectural choices, and demonstrate significant improvements over previous works in terms of both objective and perceptual quality.
PRO’s:
+well-written
+nice overall system: GAN framework for super-sampling audio incorporating features from an autoencoder
+some good-sounding examples
CON’s:
-some confusing/weakly-presented parts (admittedly covering lots of material in short space)
-I am confused about the evaluation; would like additional qualitative/observational understanding of what works, including more on how the results differ from baseline
SUMMARY: The task addressed in this work is: given a low-resolution audio signal, generate corresponding high-quality audio. The approach is a generative neural network that operates on raw audio and train within a GAN framework. 
Working in raw sample-space (e.g. pixels) is known to be challenging, so a stabilizing solution is to incorporate a feature loss. Feature loss, however, usually requires a network trained on a related task, and if such a net one does not already exist, then building one can have its own (possibly significant) challenges. In this work, the authors avoid this auxiliary challenge by using unsupervised feature losses, taking advantage of the fact that any audio signal can be downsampled, and therefore one has the corresponding upsampled signal as well.
The training framework is basically that of a GAN, but where, rather than providing the generator with a low-dimensional noise signal input, they provide the generator with the subsampled audio signal. The architecture includes a generator ( G(lo-fidelity)=high-fidelity ), a discriminator ( D(high-fidelity) = real or by super-sampled ? ), and an autoencoder ( \phi( signal x) = features of signal x at AE’s bottleneck). 
COMMENTS:
The generator network appears to be nearly identical to that of Kuleshov et al (2017)-- which becomes the baseline-- and so the primary contribution differentiating this work is the insertion of that network into a GAN framework along with the additional feature-based loss term. This is overall a nice problem and a nice approach! In that light, I believe that there is a new focus in this work on the perceptual quality of the outputs, as compared to (Kuleshov et al 2017). I would therefore ideally like to see (a) some attempts at perceptually evaluating the resulting output (beyond PESQ, e.g. with human subjects and with the understanding that, e.g. not all AMT workers have the same aural discriminative abilities themselves), and/or (b) more detailed associated qualitative descriptions/visualization of the super-sampled signal, perhaps with a few more samples if that would help. That said, I understand that there are page/space limitations. (more on this next)
Given the similarity of the U-net architectures to (Kuleshov et al 2017), why not move some of those descriptions to the appendix? 
For example, I found the description and figure illustrating the “superpixel layers” to be fairly uninformative: I see that the figure shows interleaving and de-interleaving, resulting in trading-off dimensionalities/ranks/etc, and we are told that this helps with well-known checkerboard artifacts, but I was confused about what the white elements represent, and the caption just reiterated that resolution was being increased and decreased. Overall, I didn’t really understand exactly the role that this plays in the system; I wondered if it either needed a lot more clarification (in an appendix?), or just less space spent on it, but keeping the pointers to the relevant references.  It seems that the subpixel layer was already implemented in Kuleshov 2017, with some explanation, yet in the present work a large table (Table 1(b)) is presented showing that there is no difference in quality metrics, and the text also mentions that there is no significant perceptual difference in audio. If the subpixel layer were explained in detail, and with justification, then I would potentially be OK with the negative results, but in this case it’s not clear why spend this time on it here. It’s possible that there is something simple about it that I am not understanding. I’m open to being convinced. Otherwise, why not just write: “Following (Kuleshov et al 2017), we use subpixel layers (Shi et al) [instead of ...] to speed up training, although we found that they make no significant perceptual effects.” or something along those lines, and leave it at that? 
I did appreciate the descriptions of models’ sensitivity to size/structure of the conv filters, importance of the res connections, etc.
My biggest confusion was with the evaluation & results:
Since the most directly related work was (Kuleshov 2017), I compared the super resolution (U-net) samples on that website (https://kuleshov.github.io/audio-super-res/ ) to the samples provided for the present work ( https://sites.google.com/view/unsupervised-audiosr/home ) and I was a bit confused, because the quality of the U-net samples in (Kuleshov 2017) seemed to be perceptually significantly better than the quality of the Deep CNN (U-net) baseline in the present work. Perhaps I am in error about this, but as far as I can tell, the superresolution in (Kuleshov et al 2017) is significantly better than the Deep CNN examples here. Is this a result of careful selection of examples? I do believe what I hear, e.g. that the MU-GAN8 is clearly better on some examples than the U-net8. But then for non-identical samples, how come U-net4 actually generally sounds better than U-net8? That doesn’t make immediate sense either (assuming no overfitting etc). Is the benefit in moving from U-net4 to U-net8 within a GAN context but then stabilizing  it with the feature-based loss? If so, then how does MU-GAN8 compare to U-net4? Would there be any info for the reader by doing an ablation removing the feature loss from the GAN framework? etc. I guess I would like to get a better understanding of what is actually going on, even if qualitative. Is there any qualitative or anecdotal observation about which “types” of samples one system works better on than another? For example, in the provided examples for the present paper, it seemed to be the case that perhaps the MU-GAN8 was more helpful for supersampling female voices, which might have more high-frequency components that seem to get lost when downsampling, but maybe I’m overgeneralizing from the few examples I heard. 
Some spectrograms might be helpful, since they do after all convey some useful information despite not telling much of the perceptual story. For example, are there visible but inaudible artifacts? Are such artifacts systematic?
Were individual audio samples represented as a one-hot encoding, or as floats? (I assume floats since there was no mention of sampling from a distribution to select the value).
A couple of typos:
descriminator → discriminator 
pg 6 “Impact of superpixel layers” -- last sentence of 2nd par is actually not a sentence. “the reduction in convolutional kernels prior to the superpixel operation.”
Overall, interesting work, and I enjoyed reading it. If some of my questions around evaluation could be addressed-- either in a revision, or in a rebuttal (e.g. if I completely misunderstood something)-- I would gladly consider revising my rating (which is currently somewhere between 6 and 7).