Existing methods for AI-generated artworks still struggle with generating high-quality stylized content, where high-level semantics are preserved, or separating fine-grained styles from various artists. We propose a novel Generative Adversarial Disentanglement Network which can disentangle two complementary factors of variations when only one of them is labelled in general, and fully decompose complex anime illustrations into style and content in particular. Training such model is challenging, since given a style, various content data may exist but not the other way round. Our approach is divided into two stages, one that encodes an input image into a style independent content, and one based on a dual-conditional generator. We demonstrate the ability to generate high-fidelity anime portraits with a fixed content and a large variety of styles from over a thousand artists, and vice versa, using a single end-to-end network and with applications in style transfer. We show this unique capability as well as superior output to the current state-of-the-art.
The paper proposes an image generation method with the focus on generating anime faces from various artists. The proposed method which is a combination of conditional GANs and conditional VAEs manages to generate high fidelity anime images with various styles.
The paper is well-written and easy to follow and understand. The goals are clearly stated and the background (which is more of history), as well as related work, is comprehensive. The decision behind every design decision has been mentioned in detail which makes the paper stronger. The main writing flaw of the paper is in the figures where more annotation and caption is required to make them easy to understand. For example:
- The significance of colors in Figure 1 (architecture of the model) is not annotated at all and the letters are not clear either (although they are described in the text itself a figure should be comprehensive by itself).
- Figure 2 and Figure 4 are really hard to understand with very limited annotation and caption. I had to read the text multiple time to Figure out what is what in these figures which is not a good sign for clarity.
In terms of experiments, I think where the paper suffers the most is in comparison with other conditional methods. In Section 5 it has been clearly mentioned that "this result can be expected from a class-conditional GAN and the focus in on Disentanglement" however very little evidence has been provided for superior disentanglement. More experiments are required to demonstrate the capabilities of the model compared to other conditional methods (which is currently only limited to StarGAN) as well as its capability of disentanglement. I agree with the authors that quantitative evaluation of generated anime faces is not easy (although it is possible with a carefully designed human study), however, the disentanglement (which is the focus of the paper) is easy to evaluate quantitatively. This demands for more experiments on disentanglement datasets with known generative factors. Although the current ablation study in the Appendix provides more details for architectural decisions, a more qualitative and quantitative comprehensive ablation study (by actually ablating the final model) can help to demonstrate these decisions.
In conclusion, the paper has great results. We all know a big part of writing this kind of paper is to make the model "work" and authors truly demonstrate that they worked hard. However, the impact of the paper (in the current form) is not clear. With the focus on disentanglement, little evidence has been provided to justify the capability of the proposed method. I believe by addressing my comments on the experiments the paper can be easily pushed above the acceptance bar. Also releasing the code dataset should increase the impact of the paper.