Style Transfer from Non-Parallel Text by Cross-Alignment
This paper focuses on style transfer on the basis of non-parallel text. This is an instance of a broad family of problems including machine translation, decipherment, and sentiment modification. The key challenge is to separate the content from other aspects such as style. We assume a shared latent content distribution across different text corpora, and propose a method that leverages refined alignment of latent representations to perform style transfer. The transferred sentences from one style should match example sentences from the other style as a population. We demonstrate the effectiveness of this cross-alignment method on three tasks: sentiment modification, decipherment of word substitution ciphers, and recovery of word order.
This paper presents a method for learning style transfer models based on non-parallel corpora. The premise of the work is that it is possible to disentangle the style from the content and that when there are two different corpora on the same content but in distinctly different styles, then it is possible to induce the content and the style components. While part of me is somewhat skeptical whether it is truly possible to separate out the style from the content of natural language text, and that I tend to think sentiment and word-reordering presented in this work as applications correspond more to the content of an article than the style, I do believe that this paper presents a very creative and interesting exploration that makes both theoretical and empirical contributions. 
— (relatively minor) questions:
- In the aligned auto-encoder setting, the discriminator is based on a simple feedforward NN, while in the cross-aligned auto-encoder setting, the discriminators are based on convolution NNs. I imagine ConvNets make stronger discriminators, thus it’d be helpful if the paper can shed lights on how much the quality of the discriminators influence the overall performance of the generators.
- Some details on the implementation and experiments are missing, which might matter for reproducibility. For example, what kind of RNNs are used for the encoder and the generator? What kind of convolution networks are used for the discriminators? What are the vocabulary sizes (and the corresponding UNK scheme) for different datasets? The description for the word reordering dataset is missing. If the authors will share their code and the dataset, it’d help resolving most these questions, but still it’d be good to have some of these detailed specified in the final manuscript.