We perform completely unsupervised one-sided image to image translation between a source domain X and a target domain Y such that we preserve relevant underlying shared semantics (e.g., class, size, shape, etc). In particular, we are interested in a more difficult case than those typically addressed in the literature, where the source and target are "far" enough that reconstruction-style or pixel-wise approaches fail. We argue that transferring (i.e., translating) said relevant information should involve both discarding source domain-specific information while incorporate target domain-specific information, the latter of which we model with a noisy prior distribution. In order to avoid the degenerate case where the generated samples are only explained by the prior distribution, we propose to minimize an estimate of the mutual information between the generated sample and the sample from the prior distribution. We discover that the architectural choices are an important factor to consider in order to preserve the shared semantic between X and Y . We show state of the art results on the MNIST to SVHN task for unsupervised image to image translation.
==== After rebuttal === 
I thank the authors for responses. I carefully read the response. But it is difficult to find a reason to increase the score. So, I keep my score. 
====================
Unsupervised image-to-image (I2I) translation is an important issue due to various applications and it is still challenging when applied to diverse image data and data where domain gap is large. This paper employs a neural mutual information estimator (MINE) to deal with I2I translation between two domains where there is a large gap. However, this paper contains several issues.
1. Pros. and Cons.
   (+) Mathematical definition of I2I translation
   (+) Application of mutual information for conserving content.
   (-) Lack of comparison with recent I2I models
   (-) Lack of experimental results and ablation studies 
   (-) Unclear novelty
2. Major comments
   - The novelty of this paper is not clear. Excluding the mathematical definition, it seems that the proposed TI simply combines DCGAN and MINE-based statistical networks. For clarifying the novelty, the detailed architecture and final objective functions can be helpful. 
   - Recent works on unsupervised I2I translation are omitted including UNIT [1], MUNIT [2], and DRIP [3]. Also, the authors need to clarify the main difference of TI-GAN from comparing models.
   - It is not clear to relate the mathematical definition of domain transfer to one-to-many translation within large domain gap. 
   - It is not clear how to use mutual information (MINE) for learning. There is no explicit definition of loss function considering MINE term. 
   - It is short of comparing other state-of-the art models such as UNIT, MUNIT, DRIP, and AugCycleGAN. They compared their results with CycleGAN only.
   - Experiments are not enough to support the authors’ insist. There is not any quantitative metric or qualitative result on generating edge-to-shoes. 
   - It is difficult to read due to inconsistent usage of terms (e.g., Figure 3 and 4 (c)s)
   - For better understanding, it requires to compare the patterns of MINE loss and adversarial loss. 
   - Experiments on more datasets such as animal, season, faces or USPS datasets. 
   - What is the main difference in the results between DCGAN-based and UNet-based models?
Minor
   - cicle_times symbol looks the product between distribution. But it should be defined before being used.
   - A reference of CycleGAN is incorrectly cited. 
   - There are some typos in the paper.
   - page 1: dependent → depend
   - page 3: by separate → by separating
   - page 6: S a I → S and I
1. Ming-Yu Liu, Thomas Breuel, and Jan Kautz. Unsupervised image-to-image translation networks, CoRR, abs/1703.00848, 2017
2. Xun Huang, Ming-Yu Liu, Serge Belongie, Jan Kautz, Multimodal Unsupervised Image-to-Image Translation, CoRR. abs/1804.04732
3. Hsin-Ying Lee, Hung-Yu Tseng, Jia-Bin Huang, Maneesh Kumar Singh, Ming-Hsuan Yang, Diverse Image-to-Image Translation via Disentangled Representations, ECCV 2018.