Existing methods for AI-generated artworks still struggle with generating high-quality stylized content, where high-level semantics are preserved, or separating fine-grained styles from various artists. We propose a novel Generative Adversarial Disentanglement Network which can disentangle two complementary factors of variations when only one of them is labelled in general, and fully decompose complex anime illustrations into style and content in particular. Training such model is challenging, since given a style, various content data may exist but not the other way round. Our approach is divided into two stages, one that encodes an input image into a style independent content, and one based on a dual-conditional generator. We demonstrate the ability to generate high-fidelity anime portraits with a fixed content and a large variety of styles from over a thousand artists, and vice versa, using a single end-to-end network and with applications in style transfer. We show this unique capability as well as superior output to the current state-of-the-art. Computer generated art (Hertzmann, 2018) has become a topic of focus lately, due to revolutionary advancements in deep learning. Neural style transfer (Gatys et al., 2016) is a groundbreaking approach where high-level styles from artwork can be re-targeted to photographs using deep neural networks. While there has been numerous works and extensions on this topic, there are deficiencies in existing methods. For complex artworks, the methods that rely on matching neural network features and feature statistics, do not sufficiently capture the concept of style at the semantic level. Methods based on image-toimage translation (Isola et al., 2017) are able to learn domain specific definitions of style, but do not scale well to a large number of styles. In addressing these challenges, we found that style transfer can be formulated as a particular instance of a general problem, where the dataset has two complementary factors of variation, with one of the factors labelled, and the goal is to train a generative network where the two factors can be fully disentangled and controlled independently. For the style transfer problem, we have labelled style and unlabelled content. Based on various adversarial training techniques, we propose a solution to the problem and call our method Generative Adversarial Disentangling Network. Our approach consists of two main stages. First, we train a style-independent content encoder, then we introduce a dual-conditional generator based on auxiliary classifier GANs. We demonstrate the disentanglement performance of our approach on a large dataset of anime portraits with over a thousand artist-specific styles, where our decomposition approach outperforms existing methods in terms of level of details and visual quality. Our method can faithfully generate portraits with proper style-specific shapes and appearances of facial features, including eyes, mouth, chin, hair, blushes, highlights, contours, as well as overall color saturation and contrast. To show the generality of our method, we also include results on the NIST handwritten digit dataset where we can disentangle between writer identity and digit class when only the writer is labelled, or alternatively when only the digit is labelled. We introduced a Generative Adversarial Disentangling Network which enables true semanticlevel artwork synthesis using a single generator. Our evaluations and ablation study indicate that style and content can be disentangled effectively through our a two-stage framework, where first a style independent content encoder is trained and then, a content and styleconditional GANs is used for synthesis. While we believe that our approach can be extended to a wider range of artistic styles, we have validated our technique on various styles within the context of anime illustrations. In particular, this techniques is applicable, as long as we disentangle two factors of variation in a dataset and only one of the factors is labelled and controlled. Compared to existing methods for style transfer, we show significant improvements in terms of modeling high-level artistic semantics and visual quality. In the future, we hope to extend our method to styles beyond anime artworks, and we are also interested in learning to model entire character bodies, or even entire scenes. In the top two rows, in each column are two samples from the training dataset by the same artist. In each subsequent group of three rows, the leftmost image is from the training dataset. The images to the right are style transfer results generated by three different methods, from the content of the left image in the group and from the style of the top artist in the column. In each group, first row is our method, second row is StarGAN and third row is neural style. For neural style, the style image is the topmost image in the column. As stated in (Gatys et al., 2016) , which is based on an earlier work on neural texture synthesis (Gatys et al., 2015) , the justification for using Gram matrices of neural network features as a representation of style is that it captures statistical texture information. So, in essence, "style" defined as such is a term for "texture statistics", and the style transfer is limited to texture statistics transfer. Admittedly, it does it in smart ways, as in a sense the content features are implicitly used for selecting which part of the style image to copy the texture statistics from. As discussed in section 2 above, we feel that there is more about style than just feature statistics. Consider for example the case of caricatures. The most important aspects of the style would be what facial features of the subjects are exaggerated and how they are exaggerated. Since these deformations could span a long spatial distance, they cannot be captured by local texture statistics alone. Another problem is domain dependency. Consider the problem of transferring or preserving color in style transfer. If we have a landscape photograph taken during the day and want to change it to night by transferring the style from another photo taken during the night, or if we want to change the season from spring to autumn, then color would be part of the style we want to transfer. But if we have a still photograph and want to make it an oil painting, then color is part of the content, we may want only the quality of the strokes of the artwork but keep the colors of our original photo. People are aware of this problem and in (Gatys et al., 2017) , two methods, luminance-only style transfer and color histogram matching, are developed to optionally keep the color of the content image. However, color is only one aspect of the image for which counting it as style vs. content could be an ambiguity. For more complicated aspects, the option to keep or to transfer may not be easily available. We make two observations here. First, style must be more than just feature statistics. Second, the concept of "style" is inherently domain-dependent. In our opinion, "style" means different ways of presenting the same subject. In each different domain, the set of possible subjects is different and so is the set of possible ways to present them. So, we think that any successful style transfer method must be adaptive to the intended domain and the training procedure must actively use labelled style information. Simple feature based methods will never work in the general setting. This includes previous approaches which explicitly claimed to disentangle style and content, such as in (Kazemi et al., 2019) which adopts the method in the original neural style transfer for style and content losses, and also some highly accomplished methods like (Liao et al., 2017) . As a side note, for these reasons we feel that some previous methods made questionable claims about style. In particular, works like (Huang et al., 2018) and StyleGAN (Karras et al., 2018 ) made reference to style while only being experimented on collections of real photographs. By our definition, in such dataset, without a careful definition and justification there is only one possibly style, that is, photorealistic, so the distinction between style and content does not make sense, and calling a certain subset of factors "content" and others "style" could be an arbitrary decision. This is also why we elect to not test our method on more established GAN datasets like CelebA or LSUN, which are mostly collections of real photos.
An adversarial training-based method for disentangling two complementary sets of variations in a dataset where only one of them is labelled, tested on style vs. content in anime illustrations. An image generation method combining conditional GANs and conditional VAEs that generates high fidelity anime images with various styles from various artists.  Proposal for a method to learn disentangled style (artist) and content representations in anime.