Domain adaptation is critical for success in new, unseen environments.
 Adversarial adaptation models applied in feature spaces discover domain invariant representations, but are difficult to visualize and sometimes fail to capture pixel-level and low-level domain shifts.
 Recent work has shown that generative adversarial networks combined with cycle-consistency constraints are surprisingly effective at  mapping images between domains, even without the use of aligned image pairs.
 We propose a novel discriminatively-trained Cycle-Consistent Adversarial Domain Adaptation model.
 CyCADA adapts representations at both the pixel-level and feature-level, enforces cycle-consistency while leveraging a task loss, and does not require aligned pairs.   Our model can be applied in a variety of visual recognition and prediction settings.
 We show new state-of-the-art results across multiple adaptation tasks, including digit classification and semantic segmentation of road scenes demonstrating transfer from synthetic to real world domains. Deep neural networks excel at learning from large amounts of data, but can be poor at generalizing learned knowledge to new datasets or environments. Even a slight departure from a network's training domain can cause it to make spurious predictions and significantly hurt its performance BID41 . The visual domain shift from non-photorealistic synthetic data to real images presents an even more significant challenge. While we would like to train models on large amounts of synthetic data such as data collected from graphics game engines, such models fail to generalize to real-world imagery. For example, a state-of-the-art semantic segmentation model trained on synthetic dashcam data fails to segment the road in real images, and its overall per-pixel label accuracy drops from 93% (if trained on real imagery) to 54% (if trained only on synthetic data, see Table 5 ).Feature-level unsupervised domain adaptation methods address this problem by aligning the features extracted from the network across the source (e.g. synthetic) and target (e.g. real) domains, without any labeled target samples. Alignment typically involves minimizing some measure of distance between the source and target feature distributions, such as maximum mean discrepancy BID23 , correlation distance BID35 , or adversarial discriminator accuracy BID9 BID41 . This class of techniques suffers from two main limitations. First, aligning marginal distributions does not enforce any semantic consistency, e.g. target features of a car may be mapped to source features of a bicycle. Second, alignment at higher levels of a deep representation can fail to model aspects of low-level appearance variance which are crucial for the end visual task.Generative pixel-level domain adaptation models perform similar distribution alignment-not in feature space but rather in raw pixel space-translating source data to the "style" of a target domain. Recent methods can learn to translate images given only unsupervised data from both domains BID2 BID21 BID34 . The results are visually compelling , but such image-space models have only been shown to work for small image sizes and limited domain shifts. A more recent approach BID1 ) was applied to larger (but still not high resolution) images, but in a controlled environment with visually simple images for robotic applications. Furthermore, they also do not necessarily preserve content: while the translated image may "look" like it came from the right domain, crucial semantic information may be lost. For 54.0% 83.6%Figure 1: We propose CyCADA , an adversarial unsupervised adaptation algorithm which uses cycle and semantic consistency to perform adaptation at multiple levels in a deep network. Our model provides significant performance improvements over source model baselines.Pixel Feature Semantic Cycle Loss Loss Loss ConsistentCycleGAN BID48 Feature Adapt BID9 BID41 Pixel Adapt BID36 BID2 ) CyCADA Table 1 : Our model, CyCADA, may use pixel, feature, and semantic information during adaptation while learning an invertible mapping through cycle consistency.example, a model adapting from line-drawings to photos could learn to make a line-drawing of a cat look like a photo of a dog.How can we encourage the model to preserve semantic information in the process of distribution alignment? In this paper, we explore a simple yet powerful idea: give an additional objective to the model to reconstruct the original data from the adapted version. Cycle-consistency was recently proposed in a cross-domain image generation GAN model, CycleGAN BID48 , which showed transformative image-to-image generation results, but was agnostic to any particular task.We propose Cycle-Consistent Adversarial Domain Adaptation (CyCADA), which adapts representations at both the pixel-level and feature-level while enforcing pixel and semantic consistency. We use a reconstruction (cycle-consistency) loss to enforce the cross-domain transformation to preserve pixel information and a semantic labeling loss to enforce semantic consistency. CyCADA unifies prior feature-level BID9 BID41 and image-level BID21 BID2 BID34 adversarial domain adaptation methods together with cycle-consistent image-to-image translation techniques BID48 , as illustrated in Table 1 . It is applicable across a range of deep architectures and/or representation levels, and has several advantages over existing unsupervised domain adaptation methods.We apply our CyCADA model to the task of digit recognition across domains and the task of semantic segmentation of urban scenes across domains. Experiments show that our model achieves state of the art results on digit adaptation, cross-season adaptation in synthetic data, and on the challenging synthetic-to-real scenario. In the latter case, it improves per-pixel accuracy from 54% to 83 %, nearly closing the gap to the target-trained model.Our experiments confirm that domain adaptation can benefit greatly from cycle-consistent pixel transformations, and that this is especially important for pixel-level semantic segmentation with contemporary FCN architectures. We demonstrate that enforcing semantic consistency between input and stylized images prevents label flipping on the large shift between SVHN and MNIST (example, prevents a SVHN 9 from being mapped into an MNIST 2). Interestingly, on our semantic segmentation tasks (GTA to CityScapes ) we did not observe label flipping to be a major source of error, even without the semantic consistency loss. Because of this, and due to memory constraints, we do not include this loss for the segmentation tasks. Further, we show that adaptation at both the pixel and representation level can offer complementary improvements with joint pixel-space and feature adaptation leading to the highest performing model for digit classification tasks. We presented a cycle-consistent adversarial domain adaptation method that unifies cycle-consistent adversarial models with adversarial adaptation methods. CyCADA is able to adapt even in the absence of target labels and is broadly applicable at both the pixel-level and in feature space. An image-space adaptation instantiation of CyCADA also provides additional interpretability and serves as a useful way to verify successful adaptation. Finally, we experimentally validated our model on a variety of adaptation tasks: state-of-the-art results in multiple evaluation settings indicate its effectiveness, even on challenging synthetic-to-real tasks. We begin by pretraining the source task model, f S , using the task loss on the labeled source data. Next, we perform pixel-level adaptation using our image space GAN losses together with semantic consistency and cycle consistency losses. This yeilds learned parameters for the image transformations, G S→T and G T →S , image discriminators, D S and D T , as well as an initial setting of the task model, f T , which is trained using pixel transformed source images and the corresponding source pixel labels. Finally, we perform feature space adpatation in order to update the target semantic model, f T , to have features which are aligned between the source images mapped into target style and the real target images. During this phase, we learn the feature discriminator, D feat and use this to guide the representation update to f T . In general, our method could also perform phases 2 and 3 simultaneously, but this would require more GPU memory then available at the time of these experiments.For all feature space adaptation we equally weight the generator and discriminator losses. We only update the generator when the discriminator accuracy is above 60% over the last batch (digits) or last 100 iterations (semantic segmentation) -this reduces the potential for volatile training. If after an epoch (entire pass over dataset) no suitable discriminator is found, the feature adaptation stops, otherwise it continues until max iterations are reached.
An unsupervised domain adaptation approach which adapts at both the pixel and feature levels This paper proposes a domain adaptation approach by extending the CycleGAN with task specific loss functions and loss imposed over both pixels and features.  This paper proposes the use of CycleGANs for Domain Adaptation This paper makes a novel extension to the previous work on CycleGAN by coupling it with adversarial adaptation approaches, including a new feature and semantic loss in the overall objective of the CycleGAN, with clear benefits.