STCN: Stochastic Temporal Convolutional Networks
Convolutional architectures have recently been shown to be competitive on many sequence modelling tasks when compared to the de-facto standard of recurrent neural networks (RNNs) while providing computational and modelling advantages due to inherent parallelism. However, currently, there remains a performance gap to more expressive stochastic RNN variants, especially those with several layers of dependent random variables. In this work, we propose stochastic temporal convolutional networks (STCNs), a novel architecture that combines the computational advantages of temporal convolutional networks (TCN) with the representational power and robustness of stochastic latent spaces. In particular, we propose a hierarchy of stochastic latent variables that captures temporal dependencies at different time-scales. The architecture is modular and flexible due to the decoupling of the deterministic and stochastic layers. We show that the proposed architecture achieves state of the art log-likelihoods across several tasks. Finally, the model is capable of predicting high-quality synthetic samples over a long-range temporal horizon in modelling of handwritten text.
 The focus on novelty (mentioned in both the abstract, and conclusion as a direct claim) in the presentation hurts the paper overall. Without stronger comparison to other closely related work, and lack of citation to several closely related models, the claim of novelty isn*t defined well enough to be useful. Describing what parts of this model are novel compared to e.g. Stochastic WaveNet or the conditional dilated convolutional decoder of "Improved VAE for Text ..." (linked below, among many others) would help strengthen the novelty claim, if the claim of novelty is needed or useful at all. Stochastic WaveNet in particular seems very closely related to this work, as does PixelVAE. In addition, use of autoregressive models conditioned on (non-variational, in some sense) latents have been shown in both VQ-VAE and ADA among others, so a discussion would help clarify the novelty claim. Empirical results are strong, though (related to the novelty issue) there should be greater comparison both quantitatively and qualitatively to further work. In particular, many of the papers linked below show better empirical results on the same datasets. Though the results are not always directly comparable, a discussion of *why* would be useful - similar to how Z-forcing was included. In the qualitative analysis, it would be good to see a more zoomed out view of the text (as in VRNN), since one of the implicit claims of the improvement from dense STCN is improved global coherence by direct connection to the "global latents". As it stands now the text samples are a bit too local to really tell. In addition, the VRNN samples look quite a bit different than what the authors present in their work - what implementation was used for the VRNN samples (they don*t appear to be clips from the original paper)? On the MNIST setting, there are many missing numbers in the table from related references (some included below), and the >= 60.25 number seems so surprising as to be (possibly) incorrect - more in-depth analysis of this particular result is needed. Overall the MNIST result needs more description and relation to other work, for both sequential and non-sequential models. The writing is well-done overall, and the presented method and diagrams are clear. My primary concern is in relation to related work, clarification of the novelty claim, and more comparison to existing methods in the results tables. Variational Bi-LSTM https://arxiv.org/abs/1711.05717 Stochastic WaveNet https://arxiv.org/abs/1806.06116 PixelVAE https://arxiv.org/abs/1611.05013 Filtering Variational Objectives https://github.com/tensorflow/models/tree/master/research/fivo Improved Variational Autoencoders for Text Modeling using Dilated Convolutions https://arxiv.org/abs/1702.08139 Temporal Sigmoid Belief Networks for Sequential Modeling http://papers.nips.cc/paper/5655-deep-temporal-sigmoid-belief-networks-for-sequence-modeling Neural Discrete Representation Learning (VQ-VAE) https://arxiv.org/abs/1711.00937 The challenge of realistic music generation: modelling raw audio at scale (ADA) https://arxiv.org/abs/1806.10474 Learning hierarchical features from Generative Models https://arxiv.org/abs/1702.08396 Avoiding Latent Variable Collapse with Generative Skip Models https://arxiv.org/abs/1807.04863 EDIT: Updated score after second revisions and author responses
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
STCN: Stochastic Temporal Convolutional Networks
Convolutional architectures have recently been shown to be competitive on many sequence modelling tasks when compared to the de-facto standard of recurrent neural networks (RNNs) while providing computational and modelling advantages due to inherent parallelism. However, currently, there remains a performance gap to more expressive stochastic RNN variants, especially those with several layers of dependent random variables. In this work, we propose stochastic temporal convolutional networks (STCNs), a novel architecture that combines the computational advantages of temporal convolutional networks (TCN) with the representational power and robustness of stochastic latent spaces. In particular, we propose a hierarchy of stochastic latent variables that captures temporal dependencies at different time-scales. The architecture is modular and flexible due to the decoupling of the deterministic and stochastic layers. We show that the proposed architecture achieves state of the art log-likelihoods across several tasks. Finally, the model is capable of predicting high-quality synthetic samples over a long-range temporal horizon in modelling of handwritten text.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 The focus on novelty (mentioned in both the abstract, and conclusion as a direct claim) in the presentation hurts the paper overall. Without stronger comparison to other closely related work, and lack of citation to several closely related models, the claim of novelty isn*t defined well enough to be useful. Describing what parts of this model are novel compared to e.g. Stochastic WaveNet or the conditional dilated convolutional decoder of "Improved VAE for Text ..." (linked below, among many others) would help strengthen the novelty claim, if the claim of novelty is needed or useful at all. Stochastic WaveNet in particular seems very closely related to this work, as does PixelVAE. In addition, use of autoregressive models conditioned on (non-variational, in some sense) latents have been shown in both VQ-VAE and ADA among others, so a discussion would help clarify the novelty claim. Empirical results are strong, though (related to the novelty issue) there should be greater comparison both quantitatively and qualitatively to further work. In particular, many of the papers linked below show better empirical results on the same datasets. Though the results are not always directly comparable, a discussion of *why* would be useful - similar to how Z-forcing was included. In the qualitative analysis, it would be good to see a more zoomed out view of the text (as in VRNN), since one of the implicit claims of the improvement from dense STCN is improved global coherence by direct connection to the "global latents". As it stands now the text samples are a bit too local to really tell. In addition, the VRNN samples look quite a bit different than what the authors present in their work - what implementation was used for the VRNN samples (they don*t appear to be clips from the original paper)? On the MNIST setting, there are many missing numbers in the table from related references (some included below), and the >= 60.25 number seems so surprising as to be (possibly) incorrect - more in-depth analysis of this particular result is needed. Overall the MNIST result needs more description and relation to other work, for both sequential and non-sequential models. The writing is well-done overall, and the presented method and diagrams are clear. My primary concern is in relation to related work, clarification of the novelty claim, and more comparison to existing methods in the results tables. Variational Bi-LSTM https://arxiv.org/abs/1711.05717 Stochastic WaveNet https://arxiv.org/abs/1806.06116 PixelVAE https://arxiv.org/abs/1611.05013 Filtering Variational Objectives https://github.com/tensorflow/models/tree/master/research/fivo Improved Variational Autoencoders for Text Modeling using Dilated Convolutions https://arxiv.org/abs/1702.08139 Temporal Sigmoid Belief Networks for Sequential Modeling http://papers.nips.cc/paper/5655-deep-temporal-sigmoid-belief-networks-for-sequence-modeling Neural Discrete Representation Learning (VQ-VAE) https://arxiv.org/abs/1711.00937 The challenge of realistic music generation: modelling raw audio at scale (ADA) https://arxiv.org/abs/1806.10474 Learning hierarchical features from Generative Models https://arxiv.org/abs/1702.08396 Avoiding Latent Variable Collapse with Generative Skip Models https://arxiv.org/abs/1807.04863 EDIT: Updated score after second revisions and author responses
assistant
 The focus on novelty (mentioned in both the abstract, and conclusion as a direct claim) in the presentation hurts the paper overall. Without stronger comparison to other closely related work, and lack of citation to several closely related models, the claim of novelty isn*t defined well enough to be useful. Describing what parts of this model are novel compared to e.g. Stochastic WaveNet or the conditional dilated convolutional decoder of "Improved VAE for Text ..." (linked below, among many others) would help strengthen the novelty claim, if the claim of novelty is needed or useful at all. Stochastic WaveNet in particular seems very closely related to this work, as does PixelVAE. In addition, use of autoregressive models conditioned on (non-variational, in some sense) latents have been shown in both VQ-VAE and ADA among others, so a discussion would help clarify the novelty claim. Empirical results are strong, though (related to the novelty issue) there should be greater comparison both quantitatively and qualitatively to further work. In particular, many of the papers linked below show better empirical results on the same datasets. Though the results are not always directly comparable, a discussion of *why* would be useful - similar to how Z-forcing was included. In the qualitative analysis, it would be good to see a more zoomed out view of the text (as in VRNN), since one of the implicit claims of the improvement from dense STCN is improved global coherence by direct connection to the "global latents". As it stands now the text samples are a bit too local to really tell. In addition, the VRNN samples look quite a bit different than what the authors present in their work - what implementation was used for the VRNN samples (they don*t appear to be clips from the original paper)? On the MNIST setting, there are many missing numbers in the table from related references (some included below), and the >= 60.25 number seems so surprising as to be (possibly) incorrect - more in-depth analysis of this particular result is needed. Overall the MNIST result needs more description and relation to other work, for both sequential and non-sequential models. The writing is well-done overall, and the presented method and diagrams are clear. My primary concern is in relation to related work, clarification of the novelty claim, and more comparison to existing methods in the results tables. Variational Bi-LSTM https://arxiv.org/abs/1711.05717 Stochastic WaveNet https://arxiv.org/abs/1806.06116 PixelVAE https://arxiv.org/abs/1611.05013 Filtering Variational Objectives https://github.com/tensorflow/models/tree/master/research/fivo Improved Variational Autoencoders for Text Modeling using Dilated Convolutions https://arxiv.org/abs/1702.08139 Temporal Sigmoid Belief Networks for Sequential Modeling http://papers.nips.cc/paper/5655-deep-temporal-sigmoid-belief-networks-for-sequence-modeling Neural Discrete Representation Learning (VQ-VAE) https://arxiv.org/abs/1711.00937 The challenge of realistic music generation: modelling raw audio at scale (ADA) https://arxiv.org/abs/1806.10474 Learning hierarchical features from Generative Models https://arxiv.org/abs/1702.08396 Avoiding Latent Variable Collapse with Generative Skip Models https://arxiv.org/abs/1807.04863 EDIT: Updated score after second revisions and author responses