Triangle Generative Adversarial Networks
A Triangle Generative Adversarial Network (∆-GAN) is developed for semisupervised cross-domain joint distribution matching, where the training data consists of samples from each domain, and supervision of domain correspondence is provided by only a few paired samples. ∆-GAN consists of four neural networks, two generators and two discriminators. The generators are designed to learn the two-way conditional distributions between the two domains, while the discriminators implicitly define a ternary discriminative function, which is trained to distinguish real data pairs and two kinds of fake data pairs. The generators and discriminators are trained together using adversarial learning. Under mild assumptions, in theory the joint distributions characterized by the two generators concentrate to the data distribution. In experiments, three different kinds of domain pairs are considered, image-label, image-image and image-attribute pairs. Experiments on semi-supervised image classification, image-to-image translation and attribute-based image generation demonstrate the superiority of the proposed approach.
Triangle GAN is an elegant method to perform translation across domains with limited numbers of paired training samples. Empirical results seem good, particularly the disentanglement of known attributes and noise vectors. I'm worried, though, that the comparison to the closest similar method is somewhat unfair, and in general that the relationship to it is somewhat unclear.
Relation to Triple GAN:
Your statements about the Triple GAN model's requirements on p_y(y | x) are somewhat misleading. Both in lines 53-56 and section 2.5, it sounds like you're saying that p_y(y | x) must be known a priori, which would be a severe limitation to their model. This, though, is not the case: rather, the Triple GAN model learns a classifier which estimates p_y(y | x), and show (their Theorem 3.3) that this still leads to the unique desired equilibrium. In cases where p_y(y | x) does not have a density, it is true that their loss function would be undefined. But their algorithm will still simply estimate a classifier which gives high probability densities, and I think their Theorem 3.3 still applies; the equilibrium will simply be unreachable. Your Propositions 1 and 2 also break when the distributions lack a joint density, and so this is not a real advantage of your method.
The asymmetry in the Triple GAN loss function (8), however, still seems potentially undesirable, and your approach seems preferable in that respect. But depending on the complexity of the domains, it may be easier or harder to define and learn a classifier than a conditional generator, and so this relationship is not totally clear.
In fact, it seems like your image-label mappings (lines 164-173) use essentially this approach of a classifier rather than a conditional generator as you described earlier. This model appears sufficiently different from your main Triangle GAN approach that burying its description in a subsection titled Applications is strange. More discussion about its relationship to Triangle GAN, and why the default approach didn't work for you here, 
Similarly, the claim that Triple GAN cannot be implemented on your toy data experiment (section 4.1) seems completely wrong: it perhaps cannot with exactly the modeling choice you make of using G_x and G_y, but p_x(x | y) and p_y(y | x) are simple Gaussians and can certainly be modeled with e.g. a network outputting a predictive mean and variance, which is presumably what the Triple GAN approach would be in this case.
Theory:
The claim that "p(x, y) = p_x(x, y) = p_y(x, y) can be achieved in theory" (line 120) is iffy at best. See the seminal paper
Arjovsky and Bottou. Towards Principled Methods for Training Generative Adversarial Networks. ICLR 2017. arXiv: 1701.04862.
The basic arguments of this paper should apply to your model as well, rendering your theoretical analysis kind of moot. This subarea of GANs is fast-moving, but really I think any GAN work at this point, especially papers attempting to show any kind of theoretical results like your section 2.3, needs to grapple with its relationship to the results of that paper and the followup Wasserstein GAN (your [22]) line of work.
As you mention, it seems that many of these issues could be addressed in your framework without too much work, but it would be useful to discuss this in the theory section rather than simply stating results that are known to be of very limited utility.
Section 4.4:
In Figure 6, the edited images are clearly not random samples from the training set: for example, the images where you add eyeglasses only show women. This adds a natural question of, since the images are not random, were they cherry-picked to look for good results? I would strongly recommend ensuring that all figures are based on random samples, or if that is impractical for some reason, clarifying what subset you used (and why) and being random within that subset.
Smaller comments:
Lines 30-33: ALI and BiGAN don't seem to really fit into the paradigm of "unsupervised joint distribution matching" very well, since in their case one of the domains is a simple reference distribution over latent codes, and *any* pairwise mapping is fine as long as it's coherent. You discuss this a little more in section 3, but it's kind of strange at first read; I would maybe put your citations of [9] to [11] first and then mention that ALI/BiGAN can be easily adapted to this case as well.
Section 2.2: It would be helpful to clarify here that Triangle GANs are exclusively a "translation" method: you cannot produce an (x, y) sample de novo. This is fine for the settings considered here, but differs from typical GAN structures enough that it is probably worth re-emphasizing e.g. that when you refer to drawing a sample from p(x), you mean taking a training sample rather than some form of generative model.