Recently, Generative Adversarial Network (GAN) and numbers of its variants have been widely used to solve the image-to-image translation problem and achieved extraordinary results in both a supervised and unsupervised manner. However, most GAN-based methods suffer from the imbalance problem between the generator and discriminator in practice. Namely, the relative model capacities of the generator and discriminator do not match, leading to mode collapse and/or diminished gradients. To tackle this problem, we propose a GuideGAN based on attention mechanism. More specifically, we arm the discriminator with an attention mechanism so not only it estimates the probability that its input is real, but also does it create an attention map that highlights the critical features for such prediction. This attention map then assists the generator to produce more plausible and realistic images. We extensively evaluate the proposed GuideGAN framework on a  number of image transfer tasks. Both qualitative results and quantitative comparison demonstrate the superiority of our proposed approach. Generative Adversarial Networks (GANs) have drawn much attention during the past few years, due to their proven ability to generate realistic and sharp looking images. Various computer vision problems are solved using this framework, such as super-resolution (Ledig et al., 2017) , colorization (Cao et al., 2017) , denoising (Yang et al., 2018) and style transfer . All these problems can be considered as an image-to-image translation problem, mapping an image from source domain to target domain, for instance, the super-resolution problem of trying to transfer a low-resolution image (source domain) to a corresponding high-resolution image (target domain). Existing literatures have shows that variants of GAN achieved impressive results in both a supervised and unsupervised setting. Choi et al., 2018; Huang et al., 2018) Even with such great success, most GAN-based approaches are suffering from the imbalance between the generator and discriminator (Arjovsky & Bottou, 2017) . In practice, the discriminator is usually too powerful for its task. Thus, the generator obtains very small gradients from discriminator and is hard to converge. Most state-of-the-art solutions are trying to find a new objective or add some new regularization terms to the cost function, which mainly affect the generator Arjovsky & Bottou, 2017; Mao et al., 2017; Nowozin et al., 2016; Hu et al., 2018) . To address this problem from a different direction, we want to borrow some power from the discriminator by incorporating the attention mechanism to help the generator. In this paper, we propose that the critical locating areas are more significant in the translation. The generator should pay more attention to some particular areas rather than the whole image. Imagine a student is learning how to draw a horse. The standard discriminator, as a painting master, merely grades the student's painting and hopes that can help the student improve his work. On the other hand, another master will provide additional information. For instance, an error canvas circling each incorrect region. That is exactly our idea: we suggest that the student (generator) gains benefit from the second master (attention embedded discriminator). Our main contribution is threefold: • A flexible attention-augmented discriminator: such discriminator provides not only the probability of realness, but an attention map from its perspective. Both trainable attention module and post hoc attention are implemented. • A unified GAN framework using attention information: to improve the translation of the generator, we combine the attention map with raw input via two concatenate methods: 1) convert the input to a RGBA image by adding an alpha channel; 2) compute the residual Hadamard production of the attention map and corresponding original input, based on RAM; • Extensive experiment validation on different benchmarks: we provide extensive experimental validation of GuideGAN on different benchmarks; both the qualitative results and quantitative comparisons against state-of-the-art methods demonstrate the effectiveness of our approach. To the best of our knowledge, we are the first to report image-to-image translation results using the attention information from discriminator. Different with previous approaches, our framework strengthens the communication and guidance between the generator and discriminator. At a high level, the significance of our work is also on discovering that the attention information from auxiliary network affects the result of image-to-image translation, which we think would be influential to other related research in the future. we have proposed a novel method incorporating attention map from discriminator for image-toimage translation. The experiments on different datasets have shown successful translation in both supervised and unsupervised setting. We remark that our idea can apply on any GAN-based model with little modification, such as those baselines in the paper. Nonetheless, the results are sensitive to the selection of attention module and concatenation. Investigating the impact of different attention mechanism and new tasks could be an interesting research direction in the future.
A general method that improves the image translation performance of GAN framework by using an attention embedded discriminator A feedback mechanism in the GAN framework which improves the quality of generated images in image-to-image translation, and whose discriminator outputs a map indicating where the generator should focus to make its results more convincing. Proposal for a GAN with an attention-based discriminator for I2I translation which provides the probability of real/fake and an attention map which reflects salience for image generation.