MULTIPLE-ATTRIBUTE TEXT REWRITING
The dominant approach to unsupervised "style transfer" in text is based on the idea of learning a latent representation, which is independent of the attributes specifying its "style". In this paper, we show that this condition is not necessary and is not always met in practice, even with domain adversarial training that explicitly aims at learning such disentangled representations. We thus propose a new model that controls several factors of variation in textual data where this condition on disentanglement is replaced with a simpler mechanism based on back-translation. Our method allows control over multiple attributes, like gender, sentiment, product type, etc., and a more fine-grained control on the trade-off between content preservation and change of style with a pooling operator in the latent space. Our experiments demonstrate that the fully entangled model produces better generations, even when tested on new and more challenging benchmarks comprising reviews with multiple sentences and multiple attributes.
This work proposes a new model that controls several factors of variation in textual data where the condition on disentanglement is replaced with a simpler mechanism based on back-translation. It allows control over multiple attributes, and a more fine-grained control on the trade-off between content preservation and change of style with a pooling operator in the latent space.
One of the major arguments is it is unnecessary to have attribute-disentangled latent representations in order to have good style-transferring rewriting. In Table 2, the authors showed that "a classifier that is separately trained on the resulting encoder representations has an easy time recovering the sentiment" when the discriminator during training has been fooled. Is there any difference between the two discriminators/classifiers? If the post-fit classifier on top of the encoder representation can easily predict the correct sentiment, there should be enough signal from the discriminator to adapt the encoder in order to learn a more disentangled representation. On the other hand, this does not answer the question if a "true" disentangled representation would give better performance. The inferior performance from the adversarially learned models could be because of the "entangled" representations.
As the author pointed out, the technical contributions are the pooling operator and the support for multiple attributes since the loss function is the same as that in (Lample et. al 2018). These deserve more elaborated explanation and quantitative comparisons. After all, the title of this work is "multiple-attribute text rewriting". For example, the performance comparison between the proposed how averaged attribute  embeddings and simple concatenation, and the effect of the introduced trade-off using temporal max-pooling.
How important is the denoising autoencoder loss in the loss function (1)? From the training details in the supplementary material, it seems like the autoencoder loss is used as "initialization" to some degree. As pointed out by the authors, the main task is to get fluent, attribute-targeted, and content-preserving rewriting. As long as the "back-translation" gives expected result, it seems not necessary to have "meaningful" or hard "content-preserving" latent representations when the generator is powerful enough.
I think the last and most critical question is what the expected style-transferred rewriting look like. What level or kind of "content-preserving" do we look for? In Table 4, it shows that the BLEU between the input and the referenced human rewriting is only 30.6 which suggest many contents have been modified besides the positive/negative attribute. This can also be seen from the transferred examples. In Table 8, one of the Male example: "good food. my wife and i always enjoy coming here for dinner. i recommend india garden." and the Female transferred rewriting goes as "good food. my husband and i always stop by here for lunch. i recommend the veggie burrito". It's understandable that men and women prefer different types of food even though it is imagination without providing context. But the transfer from "dinner" to "lunch" is kind of questionable. Is it necessary to change the content which is irrelevant to the attributes?
Other issues:
- Towards the end of Section 3, it says that "without back-propagating through the back-translation generation process". Can you elaborate on this and the reason behind this choice?
- What does it mean by "unknown words" in "... with 60k BPE codes, eliminating the presence of unknown words" from Section 4?
- There is no comparison with (Zhang et. al. 2018), which is the "most relevant work".
- In Table 4, what is the difference among the three "Ours" model?
- In Table 4, the perplexity of "Input Copy" is very high compared with generated sentences.
- In Table 7, what does the "attention" refer to?
- In the supplementary material, there are lambda_BT and lambda_AE. But there is only one lambda in the loss function (1).
- Please unify the citation style.