We perform completely unsupervised one-sided image to image translation between a source domain $X$ and a target domain $Y$ such that we preserve relevant underlying shared semantics (e.g., class, size, shape, etc). 
 In particular, we are interested in a more difficult case than those typically addressed in the literature, where the source and target are ``far" enough that reconstruction-style or pixel-wise approaches fail.
 We argue that transferring (i.e., \emph{translating}) said relevant information should involve both discarding source domain-specific information while incorporate target domain-specific information, the latter of which we model with a noisy prior distribution. 
 In order to avoid the degenerate case where the generated samples are only explained by the prior distribution, we propose to minimize an estimate of the mutual information between the generated sample and the sample from the prior distribution. We discover that the architectural choices are an important factor to consider in order to preserve the shared semantic between $X$ and $Y$. 
 We show state of the art results on the MNIST to SVHN task for unsupervised image to image translation. Unsupervised image to image translation is the task of learning a mapping from images in a source distribution X to images in a target distribution Y without the use of any extrinsic information that can match the two distributions (e.g., labels). Some works Zhu et al., 2017; BID3 BID0 have proposed solutions to this problem using intrinsic properties of the transfer. In order to preserve the relevant semantics, a common approach in all of these methods is to use pixels-wise consistency metrics. For example, CycleGAN proposes a cycle consistency-loss between the input image and its reconstruction. Moreover, we assert that using pixel-wise consistency is unreasonably strong for problems where the source and target domains vary in relevant spatial characteristics.The general problem statement and our solution is depicted in Figure 1 and 2, respectively. The translator takes two inputs, one is the source input that we wish to translate and the second is the independent noise meant to model statistical variation of the target not present in the source. In order to ensure the output of this translator resembles the target, we train this whole model as the generator in a generative adversarial networks (GAN, BID6 ).An important aspect of this problem is precisely what is meant by translation. Information content is one way to think of this, but unfortunately this quantity doesn't distinguish between things we care about (e.g., salient qualities such as shape, size, class, color, etc) and things we don't (noise). Furthermore , these semantics can be case-driven and can dependent on the end-goal of designing a translator. We assert in this paper that structural assumptions must be incorporated into our model. This insight is nothing new, recent works on representation learning as well as numerous works from computer vision and generative models BID5 BID15 BID14 all operate on this assumption. Therefore, we perform our transfer in a way that forces the maintenance of spatial characteristics across transfer (i.e., by architecture choice).That said, in order to encourage transfer between the source and target domains, we use mutual information as an additional objective for the translator. While the mutual information is intractable to compute for continuous variable, Mutual Information Neural Estimation (MINE, BID2 showed that this is not only possible, but this same estimator can be used to provide a gradient signal for a generator. We observe that using MINE to either minimize (between the independent Figure 1 : Formulation of the image to image translation problem. Given two domains X and Y , presented as cat and dog images, we postulate that these two domains are in part explained by random variables U, V from the shared semantic space and random variables ψ and ξ independent of U, V that explain features specific to X and Y , respectively. noise and the output) or maximize (between the source and target variables) mutual information performs reasonably well on completely unsupervised tasks that models that rely on pixel-wise consistency performs poorly on, in our case MNSIT to SVHN, obtaining 71% accuracy on the transfer task.The contribution of this paper are the following:• We formalize the problem of unsupervised translation • We propose an augmented GAN framework that takes two input and demonstrate competitive results on the image to image translation tasks • We propose to use the mutual information to avoid the degenerate case where the generated images are only explained by one of the inputs by using the information theory In this paper, we present an unsupervised image to image translation framework. We formalize the problem of domain translation. We show that one to many image translation can be achieved without using any consistency loss. We explore the more complicated translation task where geometric changes are needed with the MNIST to SVHN task. We notice that the SVHN to MNIST task is harder. We hypothetize that this is due to the fact that SVHN has more factors of variation that the network can pick to generate the MNIST digits.
We train an image to image translation network that take as input the source image and a sample from a prior distribution to generate a sample from the target distribution This paper formalizes the problem of unsupervised translation and proposes an augmented GAN framework which uses the mutual information to avoid the degenerate case This paper formulates the problem of unsupervised one-to-many image translation and addresses the problem by minimizing the mutual information. 