Recurrent models for sequences have been recently successful at many tasks, especially for language modeling and machine translation. Nevertheless, it remains challenging to extract good representations from these models. For instance, even though language has a clear hierarchical structure going from characters through words to sentences, it is not apparent in current language models. We propose to improve the representation in sequence models by augmenting current approaches with an autoencoder that is forced to compress the sequence through an intermediate discrete latent space. In order to propagate gradients though this discrete representation we introduce an improved semantic hashing technique. We show that this technique performs well on a newly proposed quantitative efficiency measure. We also analyze latent codes produced by the model showing how they correspond to words and phrases. Finally, we present an application of the autoencoder-augmented model to generating diverse translations.
This is an interesting paper focusing on building discrete reprentations of sequence by autoencoder. 
However, the experiments are too weak to demonstrate the effectiveness of using discrete representations.
The design of the experiments on language model is problematic.
There are a few interesting points about discretizing the represenations by saturating sigmoid and gumbel-softmax, but the lack of comparisons to benchmarks is a critical defect of this paper. 
Generally, continuous vector representations are more powerful than discrete ones, but discreteness corresponds to some inductive biases that might help the learning of deep neural networks, which is the appealing part of discrete representations, especially the stochastic discrete representations. 
However, I didn't see the intuitions behind the model that would result in its superiority to the continuous counterpart. 
The proposal of DSAE might help evaluate the usage of the 'autoencoding function' c(s), but it is certainly not enough to convince people. 
How is the performance if c(s) is replaced with the representations achieved from autoencoder, variational autoencoder or simply the sentence vectors produced by language model?
The qualitative evaluation on 'Deciperhing the Latent Code' is not enough either. 
In addition, the language model part doesn't sound correct, because the model cheated on seeing the further before predicting the words autoregressively.
One suggestion is to change the framework to variational auto-encoder, otherwise anything related to perplexity is not correct in this case.
Overall, this paper is more suitable for the workshop track. It also needs a lot of more studies on related work.