# Learning to Generate Artistic Character Line Drawing

Cheng-Yu Fang, Xian-Feng Han1, Qi Zhong, Shi-Jie Sun, and Guo-Qiang Xiao
Cheng-Yu Fang, Xian-Feng Han, Qi-Zhong, Guo-Qiang Xiao with the College of
Computer and Information Science, Southwest University, Beibei District,
Chongqing, China e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Cheng-Yu Fang and Xian-Feng Han have contributed equally to
this work.1 Corresponding Author.Shi-Jie Sun with Changan University, Nan’er
Road, Xi’an, ShananXi, China<EMAIL_ADDRESS>received
April 19, 2005; revised August 26, 2015.

###### Abstract

Character line drawing synthesis can be formulated as a special case of image-
to-image translation problem that automatically manipulates the photo-to-line
drawing style transformation. In this paper, we present the first generative
adversarial network based end-to-end trainable translation architecture,
dubbed P2LDGAN, for automatic generation of high-quality character drawings
from input photos/images. The core component of our approach is the joint
geometric-semantic-driven generator, which uses our well-designed cross-scale
dense skip connections framework to embed learned geometric and semantic
information for generating delicate line drawings. In order to support the
evaluation of our model, we release a new dataset including 1,532 well-matched
pairs of freehand character line drawings as well as corresponding character
images/photos, where these line drawings with diverse styles are manually
drawn by skilled artists. Extensive experiments on our introduced dataset
demonstrate the superior performance of our proposed models against the state-
of-the-art approaches in terms of quantitative, qualitative and human
evaluations. Our code, models and dataset is available at
https://github.com/cnyvfang/P2LDGAN.

###### Index Terms:

Image-to-Image Translation, Character Line Drawing, Generative Adversarial
Network.

##  I Introduction

Character Line Drawing, also named Line Art, aims to translate the information
in photograph/image domain into a simplified representation domain with
fundamental graphic components (e.g., lines or curves) to show the change of
plane. It is an abstract and flexible form of art, whose applications include
diverse scenes, such as entertainment [1], key art [2], caricature [3] and
computer generated animation [4]. As we all know, high-quality line drawing
generation typically requires professional artists or domain experts to spend
considerable effort. And more details in structural lines mean more
difficulties in drawing the line arts [5]. Therefore, manually drawing
character line arts is a labor-intensive, time-consuming and challenging task.
And it is highly desired to development an automatic generation method that
can assist artists or amateurs in drawing character line drawings.

Recently, the great process of convolutional neural network (CNN), especially
generative adversarial network (GAN), has popularized the image
generation/translation tasks due to their powerful ability in synthesizing
impressive, high-visual-quality, realistic and faithful images. Is it possible
to automatically perform character photo-to-line drawing style translation
with the help of these artificial intelligence techniques? In essence, this
question can be considered as an image-to-image translation problem, which
converts the character image/photo to line drawings representation domain. The
GAN-based approaches are actually applicable to tackling this problem.

However, to the best of our knowledge, there is no previous studies
specifically developed for character line drawing creation. In addition,
directly extending the state-of-the-art image translation methods (e.g.,
pix2pix [6], CycleGAN [7], [8]) to this task cannot achieve high-quality
character drawings due to the following issues. (1) Freehand character line
drawings is abstract [9] and sparse [10], totally different from other image
translation applications. (2) Loss of semantic information results in unclear
and imperfect lines.

Therefore, in this paper, we attempt to make these challenges resolvable. We
introduce the first GAN-based end-to-end architecture, called P2LDGAN, aiming
to automatically learn the cross-domain corresponding relations between
images/photos and hand-drawn line drawings. The starting point of our P2LDGAN
is an input real character photo, outputting a realistic hand-drawn character
line drawing. To improve the generation quality with more details and clear
lines, we design a joint geometric-semantic-driven generator, in which the
feature maps with different scales and information flows from encoding stage
are densely concatenated into corresponding layers of decoder using cross
skips to fuse geometric and semantic features for fine-grained drawings
generation. For discriminator, we adopt patch discriminator to improve the
discriminative ability.

In order to train and evaluate our proposed model, we introduce a new dataset,
which consists of more than one thousand of character images/photos and line
drawings pairs, where the line drawings are manually created by skilled
artists we invite. We quantitatively and qualitatively compare our framework
against the state-of-the-arts on this newly collected dataset, and
experimental results show the superiority of proposed P2LDGAN. Finally, we
perform the ablation studies to further validate the effectiveness of our key
components.

Summarily, the main contributions of our study include,

  * •

To the best of our knowledge, this is probably the first work focusing on
character line drawing study.

  * •

We contribute a new photo-to-line drawing dataset including 1,532 pairs of
character photos/images and corresponding well-aligned freehand line drawings
provided by professional artists, to benefit the research on character line
drawing creation.

  * •

We present the joint geometric-semantic-driven generative adversarial
architecture with our well-designed cross-scale dense skip connections
framework as generator for automatic character line drawing generation in an
end-to-end manner.

  * •

Experimental results as well as perceptual study demonstrate the effectiveness
of the proposed P2LDGAN. And compared to state-of-the-art methods, our model
can achieve significantly higher quantitative and qualitative performance.

##  II Related Work

###  II-A Image-to-Image Translation

The image-to-image translation [11][12][13][14] can be formulated as an image
generation function that maps the given source domain image into the desired
target artistic style. It has a wide range of applications, including image
cartoonization [15][16], flat filling [17][18], face sketch synthesis [19][20]
and image inpainting [21][22]. Particularly, the generative adversarial
network based image translation has been receiving substantial interest. Based
on conditional adversarial network, Pix2pix [6] develops a general image
translation framework for different tasks, like colorization, sketch-to-
portrait. However, the training instability makes it difficult to apply
pix2pix to high-resolution image generation issue. Wang et al. [23] introduced
the multi-scale generator and discriminator models together with a new feature
match loss to form pix2pixHD to achieve high-resolution results. Qi et al.
[24] developed a semantic-driven GAN with pix2pix as backbone, and designed an
adaptive re-weighting loss to balance semantic classes’ contributions. In
contrast with learning from paired data, unsupervised image translation has
recently attracted more and more attention. CycleGAN [7] designs a cycle
consistency constraint to conduct learning with unpaired inputs. Similarly,
DualGAN [25] also proposes a general-purpose image translation framework using
a cyclic mapping. While DiscoGAN [26] couples two different GANs for
bidirectional mapping between source and target domains. Based on the
assumption that corresponding images from different domains share the same
intermediate representation in a latent space, Liu et al. [8] integrated the
core ideas of GANs and variational autoencoders (VAEs) forming the UNIT
framework to model each image domain. MUNIT [27] assumes that the
representation of source and target images can be decomposed into a shared
content space but different domain-specific style spaces. To achieve image
translation, it performs a swapping combination of these content and style
codes.

###  II-B Deep Learning in Line Drawing

The deep learning methodology has already pushed significant steps in a wide
variety of visual processing tasks, which makes great contributions to line
drawing related applications, including line drawing colorization [28],
artistic shadow creation [29], manga structural line extraction [30], and face
portrait line drawing [31]. Zhang et al. [17] proposed a split filling
mechanism to perform line art colorization. They first spilt the input user
scribbles into several groups for influence areas estimation, then a data-
driven color generation process is conducted for each group. These outputs are
finally merged to form the high-quality filling results. Zheng et al. [5]
first designed a ShapeNet to learn the 3D geometric information from line
drawings, then a RenderNet is performed to produce 3D shadow. The SmartShadow
[29] application consists of three data-driven tools, a shadow brush used to
determine the areas the user want to create shadow, a shadow boundary brush to
control the shadow boundaries, and a global shadow generator for the entire
image shadow generation based on the estimated global shadow direction. Li et
al. [30] took advantage of the ideas of residual network and symmetric
skipping network to build their CNN based method to solve problem of
structural line extraction from manga image. To generate portrait drawings in
multiple styles and unseen style, Yi et al. [10] designed a novel portrait
drawing generation architecture using a asymmetric cycle structure, where a
regression network is first adopted to calculate the quality score for an
APDrawing. Based on this model, they defined a quality loss to guide the
network to generate high-quality APDrawings. Im2Pencil [32] and [33] treat the
procedure of line drawing generation as an independent subtask to facilitate
manga and Illustration synthesis, respectively.

Different from these methods, we apply the generative adversarial network to a
new problem, i.e. our P2LDGAN mainly concentrates on finding cross-domain
relations between character images/photos and hand-drawn character line
drawings using paired data.

##  III Data Preparation

Our photo/image-to-line drawing converter can be fundamentally posed as
supervised image-to-image translation problem which heavily relies on
significant training samples, therefore, it is necessary to construct a
dataset to help our model learn an accurate mapping to bridge the real
character photo/image and freehand character line drawing domains.

Specifically, we first collect high-resolution character images/photos from
the internet mainly covering five categories, namely male, female,
manga/cartoon male, manga/cartoon female, and others, to enrich or diversify
the data. Then, to meet the demands of different artistic styles, we invite
experienced artists and many skilled students majored in visual communication
design to manually draw the character line drawings at the same scale as the
given source images/photos using specific application and digital devices.
Finally, to standardize the gathered images/photos and hand-drawn line arts,
we preprocess them to the size of 1024 $\times$ 1024 pixels, and fine tune the
structural lines manually to form the strictly aligned image/photo-line
drawing pair (describing the same character) with the help of professional
artists and image processing software.

We carefully select 1,532 pairs of high-quality character line drawings paired
with real images/photos to construct our final dataset. Table 1 illustrates
the distributions of our collected dataset. Figure LABEL:fig_example shows
representative character image/photo-line drawing pairs. In order to
facilitate the learning and evaluation, we choose to split the dataset into
two disjoint subsets, i.e. 70% for training and the remaining 30% as testing
set. The data will be released to the public for research related to line
drawing.

TABLE I: The Composition of our constructed benchmark dataset. The bottom row shows the number of image/photo-line drawing pairs contained in each category. Category | Male | Female | Cartoon Male | Cartoon Female | Others
---|---|---|---|---|---
No. | 388 | 422 | 210 | 426 | 86

##  IV Character Photo-to-Line Drawing Translation Architecture

###  IV-A Overview

We attempt to generalize the application of generative adversarial framework
to the photo-to-line drawing style conversion problem, i.e. automatic artistic
character line drawings generation from real images/photos. Given the well-
aligned source-reference pair $\\{p_{i},l{i}\\}_{i=1}^{N}$, where $p_{i}$ and
$l_{i}$ belong to character photo domain $\mathcal{P}$ and line drawing domain
$\mathcal{I}$, respectively. $N$ denotes the number of image pairs. The goal
of our model is to learn a mapping function $\mathcal{F}$ that automatically
discover the domain relations between line drawing $\hat{l}_{i}$ and
corresponding character photo.

$\hat{l}_{i}=\mathcal{F}(p_{i},l_{i})$ (1)

Figure LABEL:fig_framework illustrates the details of our developed P2LDGAN
for this cross domain corresponding learning, which consists of (1) a joint
geometric-semantic-driven generator G, and (2) a character line drawing
discriminator D. In the following, we give a detailed description of our
method.

###  IV-B Joint Geometric-Semantic-Driven Generator

The character line drawings describe the characters’ clear and critical
features using structural lines representation. Therefore, both geometric and
semantic information play a fairly important role in synthesizing vital
details in drawings. Previous state-of-the-art image translation models mainly
adopt U-net framework with skip connections. However, these methods only
combine the feature maps of the same scale from encoding and decoding stage,
lacking geometric and semantic information fusion, which limits the generation
quality.

Therefore, in order to progressively propagate the geometric and semantic
information into the output line drawing, we development a simple but
effective framework, named cross-scale dense skip connections module, which is
the core of our joint geometric-semantic-driven generator.

Our generator fundamentally is an encoder-decoder architecture, where the
encoding stage compresses the rich information in character photo into latent
representation, while the decoding network constructs the desired line
drawings from encoded representation. As shown in the bottom-left part of
Figure LABEL:fig_framework, we adopt the pre-trained ResNeXt-50 as encoder
because of its simpleness, modularization, efficiency and higher learning
capability. Specifically, we extract the stages conv1-conv5 from ResNeXt as
our encoding layer (i.e. ResNeXt Layer), which can be formulated as,

$\mathcal{F}_{i}=\mathcal{F}_{i-1}+\sum_{j=1}^{C}\mathcal{T}_{j}(\mathcal{F}_{i-1})$
(2)

Where $i$ indicates the $i$th layer of the encoder, $\mathcal{F}_{i-1}$ and
$\mathcal{F}_{i}$ denoted the input and output. $C$ in the number of groups.
$\mathcal{T}$ means transformation function, here we use a sequence of
$1\times 1$, $3\times 3$ and $1\times 1$ convolutions.

We input a character image/photo of size $512\times 512$ into encoder, and
extract feature maps from each ResNeXt layer together with our decoding layer
to build the cross-scale skip connections model to fuse and propagate
information of different levels of abstractness. To be specific, suppose our
generator has $n$ layers in total, the input of the current decoding layer
$n-i+1$ is the combination between the outputs of the previous layer and
corresponding encoding layers having the same or larger resolutions. The
formulation is defined as,

Where $\mathcal{F}_{n-i}$ represents the previous $n-i$th layer’s output,
$\mathcal{F}_{1}$ to $\mathcal{F}_{i}$ denote the outputs of the 1st to $i$th
encoding layers with scales larger than or equal to that of the $n-i+1$ layer.
PDS() refers to progressive downsampling operation to downsample feature maps
using convolutions. $\oplus$ is the element wise sum operation. Subsequently,
the fused multi-scale features flows through the decoding layer to perform
geometric and semantic information propagation into a higher-resolution maps.

$\mathcal{F}_{n-i+1}=UP(\mathcal{I}_{n-i+1})$ (3)

Here, UP() operation is used to upsample the feature maps by a factor of 2
using nearest neighbor algorithm. Through our cross skip connection mechanism,
feature maps from each encoding layer can be embedded into different decoding
layers to strengthen semantic and geometric information integration and
improve feature propagation across encoder and decoder, so that we can
directly train our generator to learn character line drawing modality and
translation the real character image/photo into line drawing with details
preservation.

###  IV-C Discriminator

For discriminator network, its task is to discriminate the generated character
line drawings from ground truth. Here, we adopt the PatchGAN exploited in [6]
to classify whether the $32\times 32$ patches are real or not. The bottom-
middle part of Figure LABEL:fig_framework displays discriminator architecture.
We can observe that each discriminator block is composed of $4\times 4$
Convolution, InstanceNorm and LeakyReLU with a slop of 0.2. Such a
discriminator contributes to high-quality line drawings generation since it
makes our P2LDGAN pay more attention to detailed content in patches, and can
process images of any size with fewer parameters [7].

###  IV-D Objective Functions

Adversarial Loss. The generator $G$ creates character line drawing $\hat{l}$
from image $p$ in character photo domain $\mathcal{P}$, which can not be
distinguished by discriminator $D$, while the goal of discriminator $D$ is to
discriminate between translated drawings and real samples $l$ from
$\mathcal{I}$. Motivated by the significant stability and high-quality
generation capability of relativistic average GANs [34], we introduce the
following Adversarial loss to supervise our P2LDGAN for more realistic line
drawing synthesis.

Where $f_{1}$ is mean square error loss. $f_{2}$ performs mean operation.
$f_{4}$ patches the input photo with the same size as outputs of discriminator
$D$, $f_{3}$ and $f_{5}$ are used to fill tensor with scalar value 1 and 0,
respectively.

Pixel-wise Loss. Since we use paired samples to train our model, we can
introduce the $L_{1}$ loss [35] (compared to $L_{2}$ loss, it introduce less
blurriness [25]) to measure the error between the generated and ground truth
character line drawings, which can make translated drawings follow the domain
$I$ distribution. The function is defined as,

In our experiments, we set the weights $\lambda_{1}=1$, $\lambda_{2}=0.5$, and
$\lambda_{3}=100$, respectively.

##  V Experiments

In this section, we first describe the implementation details and evaluation
metrics. Then we evaluate the performance of our P2LDGAN by making
quantitative and qualitative comparisons against state-of-the-art models as
well as reporting human perceptual study. Finally, we conduct ablation study
to verify the effectiveness of designed component of our proposed approach.

###  V-A Implementation Details

We implement the photo-to-line drawing translator using PyTorch. All
experiments are conducted on a single NVIDIA GeForce RTX 3090 GPU. For both
the generator and discriminator network, we use the Adam optimizer with
$\beta_{1}=0.5$ and $\beta_{2}=0.999$. The initial learning rate is set to
0.0002. We train the model for 200 epochs with a mini batch size of 1. All
input images and ground truth are aligned and resized to 512 $\times$ 512
pixels.

###  V-B Evaluation Metrics

To measure the quality of synthesized line drawing images, three commonly-used
similarity metrics, namely Fr$\acute{e}$chet Inception Distance (FID) [36] and
Structural Similarity Metric (SSIM) [37] and Peak Signal-to-noise Ratio (PSNR)
[2], are adopted to quantitatively evaluate the performance of previous
methods and our proposed P2LDGAN. Where the FID calculates the distribution
distance between the set of line drawing images created from input photos and
the corresponding ground truth drawings (the smaller the FID value is, the
better the drawing quality will be ), while SSIM describes the similarity
between the generated image and ground truth (Higher SSIM value indicates
better result). PSNR computes the intensity difference between predicted and
ground truth images (Larger PSNR scores means smaller difference between
images) [14].

###  V-C Comparison with State-of-the-Art Methods

To demonstrate the superior performance of our P2LDGAN model, we make the
quantitative and qualitative comparisons with five state-of-the-art networks,
including one existing neural style transfer method (i.e. Gatys [38]) as well
as four general image-to-image translation approaches, CycleGAN [7], DiscoGAN
[26], UNIT [8], pix2pix [6], MUNIT [27]. It is worth noting that we also use
paired data to train unsupervised methods.

We qualitatively evaluate the performance of our proposed P2LDGAN by comparing
with state-of-the-art competitors on our testing data. Figure
LABEL:fig:topbanner and Figure LABEL:fig_experiment showcase the visual
examples. From these results, we can conclude the following findings.

The character line drawing is essentially composed of abstract lines without
any texture information. However, the example-based Gatys [38] produces global
gray-like feeling [32] drawings (shown in the third column) with much texture
filled in, which are far apart from the line drawing distribution. It is
mainly because of the usage of Gram matrix [31]. On the contrary, our learning
model can generate clear and natural lines with less texture.

For CycelGAN [7], it heavily blurs the details, such as the boy’s cloth and
shoes in the first row of Figure LABEL:fig:topbanner and introduces great
distortions in various areas (e.g., the girl’s hair and bag in the third row
of Figure LABEL:fig_experiment), which results in visually unappealing line
quality. While pix2pix [6] succeeds in creating somewhat acceptable perceptual
appearance, but it suffers from jagged boundaries, incomplete lines, and
details loss. For example, as can be seen from second row of Figure
LABEL:fig:topbanner, pix2pix can not construct the nose for the man. In
comparison, our P2LDGAN is able to alleviate such unpleasant artifacts,
creating better looking line drawings with more details and structural
information preserved.

We also compare our model to DiscoGAN [26], UNIT [8] and MUNIT [27]. Although
they can capture the underlying character line styles, and produces more
reliable results, the drawings yielded by them are corrupted by messy
boundaries and over-smoothed lines. Our approach can give more precise lines
to improve visual quality, being coherent with ground truth line drawings.

In addition, most of these methods do not take full advantage of semantic
information, they, therefore, fails to learn the remarkable characteristics of
input photo, and generate natural line drawings with less visible artifacts.
For example, from the third rows of Figure LABEL:fig:topbanner and Figure
LABEL:fig_experiment, it can be explicitly stated that these models could not
generate the delicate human face structures.

In summary, (1) the proposed P2LDGAN significantly outperform state-of-the-art
methods in terms of visual quality, details preservation and artifacts
reduction. (2) Our method works well on both simple and complex character line
drawings learning.

####  V-C1 Quantitative Comparisons

To quantify the realism and faithfulness of generated character line drawings
[9], we conduct objective evaluation by computing the average scores of three
measurement methods, FID, SSIM and PNSR, on our testing set. The quantitative
comparison results with state-of-the-art GAN-based methods are summarized in
Table LABEL:tab:quantativeresult. As we can clearly observe, (1) our
architecture achieves the best FID, SSIM and PNSR values, significantly
outperforming the previous competitors from the realism and faithfulness point
of view. (2) Specifically, the lowest FID score suggests that our generated
line drawings are closest to ground truth drawings distribution, while the
highest SSIM and PNSR scores further show the maximum similarity between our
results and ground truths. (3)In summary, the quantitative experiments
demonstrate the effectiveness and superiority of our model in synthesizing the
high-quality character line drawings, which are consistent with visual
results.

####  V-C2 Human Perceptual Study

Actually, character image/photo-line drawing translation is a highly
subjective task, we, therefore, perform human perceptual studies to
perceptually evaluate the ability of our model to generate better-looking
drawings.

Participant. 30 participants jointed our user studies, including 20 students
without any artistic knowledge, and 10 professional students with at least
three years of drawing experience.

Data. This experiment covers 50 pairs of well-aligned character photos and
line drawings we randomly select from the testing set. We use our proposed
models and the state-of-the-art networks described in previous section to
translate the photos into line drawings. And the ground truth drawings are
used as reference images.

Setup. We show an input photo, ground truth and eight generated line drawings
to participants, who are asked to rank the similarity between ground truth and
resulting drawings with 4 meaning highest rank, and 1 indicating lowest one
[31]. The users are also asked to rate the overall quality (whether there are
noise, deformation, unclear or unstructured lines, incomplete areas, as well
as other artifacts in the drawings) [16] on a scale of 1 to 5 scores (Higher
score denotes better quality).

Result and Discussion. We finally collected 1,500 votes in total. The
percentage of line drawings being ranked 4 and the average scores of drawing
quality are computed and summarized in Table LABEL:tab:userstudy. From these
results, we can report that (1)our method receives the highest values in both
evaluation criteria. (2) It performs favorably over these state-of-the-arts
frameworks in terms of user preferences. (2) To conclude, our method can be
able to learn structural line representation, generating high-quality
character line drawings.

We conduct the following ablation experiments to study the contributions of
the key factors.

###  V-D Analysis of fusion strategy

We replace the sum operation in cross-scale skip connections modules with
concatenation. From Figure LABEL:fig_ablation, we can report that the usage of
concatenation achieves acceptable results but having unclear boundaries, some
roughtness and noiseness. The numerical values in Table
LABEL:tab:ablationeresult also demonstrate the influence of fusion strategies
had on drawings quality.

###  V-E Analysis of generator

In order to validate the effectiveness of our cross-scale skip connections
model in keeping fine details, We perform quantitative and qualitative
comparisons between our P2LDGAN with/without using cross-scale skip
connections model (our baseline). From the visual results, it can be reported
that without our connection model, the generated line drawings have many
structural details loss (e.g., hair area in the second row), resulting in poor
visual effects, while our P2LDGAN could restore these delicate structures,
producing robust and better looking results.

We also replace our decoding layer with a ConvTransporse + Upsample +
Convolution block. As can be found in Figure LABEL:fig_ablation and Table
LABEL:tab:ablationeresult, the network using replaced decoder produces the
character line drawings that are similar to or even better than our proposed
models. However, it requires more parameters and computation consumption than
P2LDGAN.

Summarily, our well-designed P2LDGAN shows superior performance in translating
character images/photos into high-quality line drawings with better detailed
structure, clear lines, and fewer artifacts.

In this paper, GANs based image-to-image translation tasks provide a solution
to our character line drawing generation problem. We present a novel joint-
geometric-semantic-driven GAN model, named P2LDGAN, to discover a mapping
function that learns the relations between a character image/photo and its
translated line drawing representation. We construct a dataset consists of
1,532 pairs of character images/photos and hand-drawn line drawings for
training and validating our solution. Quantitative and qualitative comparisons
with state-of-the-art methods, human evaluation as well as ablation studies
show the superiority of our P2LDGAN framework in generation quality.

Limitation. Figure LABEL:fig_failure presents some failure examples. It can be
observed that (1) when the photos are overexposed or underexposed, some areas
that are too bright (e.g., the cloth in the first column) or too dark (e.g.,
the facial area in the second column) would be missing in the generated line
drawings. (2) The low-quality original photos, such as these with blurry
texture, would cause noisy and messy lines, failing to preserve fine details,
like the third column. These problems may be considered in our future study,
and we will also investigate how to improve the realism and faithfulness of
the real human line drawings.

## References

  * [1] L. Yan, W. Zheng, C. Gou, and F.-Y. Wang, “Isgan: Identity-sensitive generative adversarial network for face photo-sketch synthesis,” _Pattern Recognition_ , vol. 119, p. 108077, 2021.
  * [2] Z. Chen, C. Wang, B. Yuan, and D. Tao, “Puppeteergan: Arbitrary portrait animation with semantic-aware appearance transformation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 13 518–13 527.
  * [3] W. Jang, G. Ju, Y. Jung, J. Yang, X. Tong, and S. Lee, “Stylecarigan: caricature generation via stylegan feature map modulation,” _ACM Transactions on Graphics (TOG)_ , vol. 40, no. 4, pp. 1–16, 2021.
  * [4] L. Mourot, L. Hoyet, F. Le Clerc, F. Schnitzler, and P. Hellier, “A survey on deep learning for skeleton-based human animation,” in _Computer Graphics Forum_. Wiley Online Library, 2021\.
  * [5] Q. Zheng, Z. Li, and A. Bargteil, “Learning to shadow hand-drawn sketches,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 7436–7445.
  * [6] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 1125–1134.
  * [7] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 2223–2232.
  * [8] M.-Y. Liu, T. Breuel, and J. Kautz, “Unsupervised image-to-image translation networks,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [9] C. Gao, Q. Liu, Q. Xu, L. Wang, J. Liu, and C. Zou, “Sketchycoco: Image generation from freehand scene sketches,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 5174–5183.
  * [10] R. Yi, Y.-J. Liu, Y.-K. Lai, and P. Rosin, “Quality metric guided portrait line drawing generation from unpaired training data,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2022.
  * [11] Q. Mao, H.-Y. Tseng, H.-Y. Lee, J.-B. Huang, S. Ma, and M.-H. Yang, “Continuous and diverse image-to-image translation via signed attribute vectors,” _International Journal of Computer Vision_ , pp. 1–33, 2022.
  * [12] T. Wang, L. Wu, and C. Sun, “A coarse-to-fine approach for dynamic-to-static image translation,” _Pattern Recognition_ , vol. 123, p. 108373, 2022.
  * [13] E. Richardson, Y. Alaluf, O. Patashnik, Y. Nitzan, Y. Azar, S. Shapiro, and D. Cohen-Or, “Encoding in style: a stylegan encoder for image-to-image translation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 2287–2296.
  * [14] Y. Pang, J. Lin, T. Qin, and Z. Chen, “Image-to-image translation: Methods and applications,” _IEEE Transactions on Multimedia_ , 2021.
  * [15] Y. Dong, W. Tan, D. Tao, L. Zheng, and X. Li, “Cartoonlossgan: Learning surface and coloring of images for cartoonization,” _IEEE Transactions on Image Processing_ , vol. 31, pp. 485–498, 2021.
  * [16] X. Wang and J. Yu, “Learning to cartoonize using white-box cartoon representations,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 8090–8099.
  * [17] L. Zhang, C. Li, E. Simo-Serra, Y. Ji, T.-T. Wong, and C. Liu, “User-guided line art flat filling with split filling mechanism,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 9889–9898.
  * [18] Y. Wu, X. Wang, Y. Li, H. Zhang, X. Zhao, and Y. Shan, “Towards vivid and diverse image colorization with generative color prior,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 14 377–14 386.
  * [19] P. Li, B. Sheng, and C. P. Chen, “Face sketch synthesis using regularized broad learning system,” _IEEE Transactions on Neural Networks and Learning Systems_ , 2021.
  * [20] B. Cao, N. Wang, J. Li, Q. Hu, and X. Gao, “Face photo-sketch synthesis via full-scale identity supervision,” _Pattern Recognition_ , vol. 124, p. 108446, 2022.
  * [21] L. Zhao, Q. Mo, S. Lin, Z. Wang, Z. Zuo, H. Chen, W. Xing, and D. Lu, “Uctgan: Diverse image inpainting based on unsupervised cross-space translation,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 5741–5750.
  * [22] H. Liu, Z. Wan, W. Huang, Y. Song, X. Han, and J. Liao, “Pd-gan: Probabilistic diverse gan for image inpainting,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 9371–9381.
  * [23] T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, A. Tao, J. Kautz, and B. Catanzaro, “High-resolution image synthesis and semantic manipulation with conditional gans,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 8798–8807.
  * [24] X. Qi, M. Sun, W. Wang, X. Dong, Q. Li, and C. Shan, “Face sketch synthesis via semantic-driven generative adversarial network,” in _2021 IEEE International Joint Conference on Biometrics (IJCB)_. IEEE, 2021, pp. 1–8.
  * [25] Z. Yi, H. Zhang, P. Tan, and M. Gong, “Dualgan: Unsupervised dual learning for image-to-image translation,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 2849–2857.
  * [26] T. Kim, M. Cha, H. Kim, J. K. Lee, and J. Kim, “Learning to discover cross-domain relations with generative adversarial networks,” in _International conference on machine learning_. PMLR, 2017, pp. 1857–1865.
  * [27] X. Huang, M.-Y. Liu, S. Belongie, and J. Kautz, “Multimodal unsupervised image-to-image translation,” in _Proceedings of the European conference on computer vision (ECCV)_ , 2018, pp. 172–189.
  * [28] M. Yuan and E. Simo-Serra, “Line art colorization with concatenated spatial attention,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 3946–3950.
  * [29] L. Zhang, J. Jiang, Y. Ji, and C. Liu, “Smartshadow: Artistic shadow drawing tool for line drawings,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 5391–5400.
  * [30] C. Li, X. Liu, and T.-T. Wong, “Deep extraction of manga structural lines,” _ACM Transactions on Graphics (TOG)_ , vol. 36, no. 4, pp. 1–12, 2017.
  * [31] R. Yi, Y.-J. Liu, Y.-K. Lai, and P. L. Rosin, “Unpaired portrait drawing generation via asymmetric cycle mapping,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 8217–8225.
  * [32] Y. Li, C. Fang, A. Hertzmann, E. Shechtman, and M.-H. Yang, “Im2pencil: Controllable pencil illustration from photographs,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 1525–1534.
  * [33] L. Zhang, X. Wang, Q. Fan, Y. Ji, and C. Liu, “Generating manga from illustrations via mimicking manga creation workflow,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 5642–5651.
  * [34] A. Jolicoeur-Martineau, “The relativistic discriminator: a key element missing from standard gan,” in _International Conference on Learning Representations_ , 2018.
  * [35] W. Chen and J. Hays, “Sketchygan: Towards diverse and realistic sketch to image synthesis,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 9416–9425.
  * [36] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [37] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” _IEEE transactions on image processing_ , vol. 13, no. 4, pp. 600–612, 2004.
  * [38] L. A. Gatys, A. S. Ecker, and M. Bethge, “Image style transfer using convolutional neural networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 2414–2423.