PGT-Net: Progressive Guided Multi-task Neural Network for Small-area Wet
  Fingerprint Denoising and Recognition
Fingerprint recognition on mobile devices is an important method for identity
verification. However, real fingerprints usually contain sweat and moisture
which leads to poor recognition performance. In addition, for rolling out
slimmer and thinner phones, technology companies reduce the size of recognition
sensors by embedding them with the power button. Therefore, the limited size of
fingerprint data also increases the difficulty of recognition. Denoising the
small-area wet fingerprint images to clean ones becomes crucial to improve
recognition performance. In this paper, we propose an end-to-end trainable
progressive guided multi-task neural network (PGT-Net). The PGT-Net includes a
shared stage and specific multi-task stages, enabling the network to train
binary and non-binary fingerprints sequentially. The binary information is
regarded as guidance for output enhancement which is enriched with the ridge
and valley details. Moreover, a novel residual scaling mechanism is introduced
to stabilize the training process. Experiment results on the FW9395 and
FT-lightnoised dataset provided by FocalTech shows that PGT-Net has promising
performance on the wet-fingerprint denoising and significantly improves the
fingerprint recognition rate (FRR). On the FT-lightnoised dataset, the FRR of
fingerprint recognition can be declined from 17.75% to 4.47%. On the FW9395
dataset, the FRR of fingerprint recognition can be declined from 9.45% to
1.09%.
Yu-Ting Li, Ching-Te Chiu, An-Ting Hsieh, Mao-Hsiu Hsu, Long Wenyong, Jui-Min Hsu
PGT-Net: Progressive Guided Multi-task Neural Network for Small-area Wet Fingerprint Denoising and Recognition

###### Abstract

Fingerprint recognition on mobile devices is an important method for identity verification. However, real fingerprints usually contain sweat and moisture which leads to poor recognition performance. In addition, for rolling out slimmer and thinner phones, technology companies reduce the size of recognition sensors by embedding them with the power button. Therefore, the limited size of fingerprint data also increases the difficulty of recognition. Denoising the small-area wet fingerprint images to clean ones becomes crucial to improve recognition performance. In this paper, we propose an end-to-end trainable progressive guided multi-task neural network (PGT-Net). The PGT-Net includes a shared stage and specific multi-task stages, enabling the network to train binary and non-binary fingerprints sequentially. The binary information is regarded as guidance for output enhancement which is enriched with the ridge and valley details. Moreover, a novel residual scaling mechanism is introduced to stabilize the training process. Experiment results on the FW395 and FT-lightnoised dataset provided by FocalTech shows that PGT-Net has promising performance on the wet-fingerprint denoising and significantly improves the fingerprint recognition rate (FRR). On the FT-lightnoised dataset, the false recognition rate (FRR) of fingerprint recognition can be declined from 17.75% to 4.47%. On the FW395 dataset, the FRR of fingerprint recognition can be declined from 9.45% to 1.09%.

 Wet fingerprint denoising, Real noise, Synthetic noise, Multi-task neural network.

## I Introduction

Biometrics-based security, such as fingerprint authentication, is proven to be both more secure and convenient than passwords and is widely used in our daily life. The uniqueness of fingerprints allows applications, including electronic payment authentication, background checks, mass disaster identification, criminal identity verification, and mobile phone unlocking. However, fingerprint images are degraded due to the noise caused by sweat, grease, or water. Nowadays, mobile phones become thinner and the screen becomes larger, and the space for the fingerprint sensor has been compressed. Fig. 1 shows an example of the sensor on the side of a mobile phone that collects tiny fingerprints of size 176 x 36. The size of the captured fingerprints' area is small which increases the difficulty of fingerprint recovery due to the limited information available from the input. If the fingerprint images are both small and blurry (particularly in wet conditions), the task of recovering clear fingerprints becomes more challenging.

Recently, deep learning neural networks have achieved great success in image processing. Works have been studied to apply deep learning network models to fingerprint denoising tasks. Some studies focus on latent fingerprints [1, 2], which are accidentally captured at the crime scene. In [3], denoising is applied first then followed by Fingerprint pore matching. Some studies focus on fingerprints with synthetic background noises, such as [4]. Unfortunately, to our best knowledge, few studies focus on daily applications to our lives-wet fingerprints denoising, especially tiny fingerprints with real noises caused by water, sweat, or grease. This situation may be caused by related datasets that are not easy to obtain since fingerprints are private information. Also, constructing such datasets that contain clean & noisy fingerprint pairs requires a lot of time and effort. Also, due to the security issue, fingerprint recognition needs a very high quality of denoising performance. Recognition algorithms need to have a meager false rejection rate (FRR) to ensure that the identity can be recognized successfully. Consequently, even small changes

Fig. 1: (a) Sensor on the side of a mobile phone that collects tiny fingerprints. (b) A fingerprint example collected by the sensor.
in the fingerprint image may lead to failure in fingerprint identification.

In this work, we developed a network, progressive guided multi-task network (PGT-Net) for wet fingerprint denoising. We also attempted several techniques to improve the model's performance. First, in the multi-task architecture, we use binary fingerprints as a supported task (guided task) to guide the main task toward more precise results. Second, our well-designed data flow makes outstanding performance, as we can get multiple helpful information through it, such as basic feature maps or the shared features between binary and non-binary fingerprints, and use the information for fingerprint denoising. Third, the novel residual scaling can help stabilize the training loss. Fourth, the binary progressive guided multi-task can produce precise binary fingerprint contour, which allows the main task to know how to restore the fingerprint better. Last but not least, the optimized loss function adds structural similarity index measure (SSIM) loss and Laplacian loss, which significantly improves the denoising results. In this work, we present the PGT-Net-block-84 model and PGT-Net-block-edge for heavy and light weight applications.

Fig. 2 shows an example of the denoised fingerprints that fail to be recognized with their corresponding input noisy image and output ground truth image. A red cross means a fingerprint fails to be identified, while a green circle represents a fingerprint that can be identified successfully. As shown in The noisy fingerprints in the first column fail to be recognized since the finger is covered with water, grease, or sweat. The images in the second and third columns are the denoised results of the FPD-M-Net [4] and single-task PGT-Net-block-84 model, the denoised quality is quite well that the denoised fingerprint is very similar to the ground truth in the fourth column, but it still cannot be recognized. While our proposed multi-task PGT-Net-block-84 neural network not only removes real-world noise but keeps the detail of the contour to succeed in passing the recognition, as shown in the fourth column.

To further show the different results between models, we enlarged the critical parts of the fingerprints that affect the recognition results, as shown in Fig. 3. The proposed multi-task PGT-Net-block-84 has the capability of not only denoising the fingerprint, and also making it precise and correct.

To be summarized, the contributions of our work are listed below:

* The proposed model can be used the wet fingerprints with tiny image sizes and significantly reduce the false rejection rate (FRR).
* The convenience of fingerprint recognition has been greatly improved.
* The FRR of the FT-lightnoised dataset (real noise) can be improved from 17.75 % to 4.47 %
* The FRR of the FW9395 dataset (synthetic noise) can be improved from 9.45% to 1.09 %
* Outperform state-of-the-art network models on fingerprint denoising.

## II Related Work

Fingerprints enhancements related technologies have been developed for decades, and many traditional methods or algorithms have been introduced. Such as Gabor-filtering-based fingerprint enhancements [5, 6, 7], orientation field estimation methods [8, 9, 10, 11, 6, 12, 13], total variation (TV) model [14, 13]. In general, these conventional methods may not perform well when heavy noise exists, which leads to poor fingerprint recovery or recognition performance. Consequently, it's still beneficial for deep learning training data pre-processing [2].

As described previously, small-area wet fingerprint denoising is difficult since there is limited input information, and it needs high precision so that the denoised fingerprints can be recognized. Most of current researches either have a poor denoising performance, or the denoised fingerprint contour is not precise enough for real wet fingerprint images. Below are the description of deep learning with different neural network models for fingerprint denoising.

### _Residual Neural Networks_

Residual learning and residual neural networks (ResNet) were invented by He et al. [15] to solve the problem of performance degradation as a network model's depth increases. This learning mechanism adds the extracted features and the output from previously convolution layers. It achieves remarkable success in the computer vision field.

The PFE-Net [3] makes use of the residual architecture to improve its performance. To reconstruct the input images, they make use of the residual structure to learn the local features. Also, the scaling is decreasing layer by layer between the residual blocks, which makes the PFE-Net able to learn the fingerprint features robustly.

Fig. 3: Enlarged critical parts of the fingerprints that affect the recognition results.

Fig. 2: Real-world Fingerprint denoising and recognition results with different model on FT-lightnoised dataset.
### _U-Net_

The U-Net [16] is an autoencoder architecture that has been used widely. It was proposed for the semantic segmentation task. The architecture has two main components, an encoder path, and a symmetric decoder path. The encoder is used to capture the context, and the decoder is used to estimate the segmentation. The symmetric architecture concatenates feature maps at the same level, thus improving the feature map's reusability and reducing the information loss during the encoding or decoding process.

FusionNet [17] is a U-Net-based architecture, which focuses on extracting cellular membrane segmentation in electron microscopy (EM) images, and had achieved great success in the EM segmentation tasks. Jung Yoon Bae et al. [18] proposed a network model for fingerprint denoising that is based on FusionNet. DenseUNet [1] is also based on U-Net and focuses on latent fingerprint enhancement. Also, it uses dense blocks to improve the information flow between layers.

### _M-Net_

The M-Net [19] is a U-net based model which is modified for better segmentation. The main difference is that two side paths are added to the two main encoding and decoding paths. The model preserves the input details with these two legs in case the downsampling operator drops image details. Besides, skip connections between corresponding encoding and decoding layers provide sufficient information and better features.

The FPD-M-Net [4] is a model that focuses on latent fingerprint denoising. It is modified from M-Net to fit the fingerprint denoising and inpainting tasks. The segmentation-based architecture demonstrated the capability to handle both denoising and inpainting tasks for fingerprint images simultaneously. The proposed model outperformed both the U-net and the baseline model provided in the competition. The model exhibited robustness to challenges such as strong background clutter and weak signals and effectively performed automatic filling. The findings also emphasized the importance of sensor-specific training for improved results when dealing with images acquired from diverse sensors.

### _Multi-tasks Learning_

Multi-tasks learning is a technique that trains multiple tasks together to improve the accuracy or reduce the parameters. Such a technique has been applied to different domains in deep learning [20, 21, 22, 23, 24, 25, 26, 21, 22, 23, 20, 24, 25, 26], such as text classification [21, 23], or image processing [20, 21, 22, 24, 25, 26].

Conventionally, deep learning models reach their target directly. For example, for fingerprint denoising, most of the studies get the denoised output fingerprint directly. While in our work, the denoised binary output fingerprints will be obtained first, and then the denoised non-binary output fingerprints, which are our main target.

FingerNet [2] introduced a multi-task deep learning fingerprint denoising network model. Some significant differences exist between our works and FingerNet [2].

First of all, although both our work and [2] are multi-task, the weights of the tasks are different in our work. In [2], two tasks exist in the model, one is the orientation task, and another is the enhancement task; these two tasks are equally important and produce equally important outputs. While in our work, we also have two tasks, one task is the "progressive guided multi-task," which has a lighter weight during training, and the other one is the "main task," which has a heavier weight during training. We focus more on the output of the main task, the output of the supported task is only for guiding the main task to achieve better denoising performance.

Second, the output of the supported task in the PGT-Net has a concatenate path to the input of the main task branch. Such concatenation further helps the main task improve its denoising performance since the denoised binary fingerprint can provide some helpful information such as fingerprints orientation or the contour of ridges and valleys in the fingerprints. While in [2], the task output wasn't utilized for further improvements.

## III Network Model

The PGT-Net is a multi-task network model. It takes noisy non-binary inputs and then separates them into two different paths to produce two outputs: denoised binary outputs and denoised non-binary contour outputs. Fig. 4 shows the data flow and the critical ideas of our proposed method. Besides, Fig. 5 displays the complete architecture of our proposed model. There are four main data flow blocks in the PGT-Net.

#### Iii-D1 Produce the Basic Feature Map (BFM)

The basic feature map is the important feature map that contains extracted features of the input image. It can also provide information to reduce feature loss during the following denoising process. BFM will be concatenated to the output of data flow block C and the output of data flow block D.

#### Iii-D2 Processing Shared Features

There are some shared features between the non-binary fingerprints and binary fingerprint contour. For example, denoised binary and non-binary fingerprints must have clear ridges and valleys to achieve a higher identification rate after denoising. Consequently, this step, "Processing shared features", is necessary.

#### Iii-D3 Processing Progressive Guided Features

As described previously, the denoised binary and non-binary fingerprints must have a precise contour to have a higher identification rate. The denoised binary fingerprint image contains accurate information about the ridges and valleys of the fingerprints, which is very helpful for the non-binary fingerprint image denoising.

#### Iii-D4 Processing Main Task's Features

At this step, the data flow block D receives the output of data flow block B,and the denoised binary output as its input. Those feature maps contain valuable information that can be used to denoise the non-binary fingerprints, which is same with our main target.

Our dataflow uses concatenation operations, as shown in Fig. 4. The concatenate operations are used to increase the number of feature maps. And the concatenate operation can be described precisely and mathematically with the concept of "set".
### _PGT-Net-block-84_

To further improve the performance of fingerprint recognition after denoising, PGT-Net-block-84 is proposed. Fig. 5 shows the detailed architecture of the proposed PGT-Net-block-84.

* The first two convolution layers are used to produce the BFM.
* Residual blocks stage 1-24 are used to the shared features, and Fig. 6 shows the architecture of a residual block. Further information about residual blocks will be discussed in section III-A1.
* Residual blocks stage 25-60 of the binary branch are used to process the progressive guided features.
* Residual block stages 25-48 of the non-binary branch are used to process the main task's features.
* PGT-Net-block-84 has 84 residual blocks in total.

And detailed information about the architecture will be discussed in sections III-A1 to III-A3.

#### Iii-A1 Residual Blocks

The architecture of residual blocks is shown in Fig. 6. Each residual block contains two convolution layers, and the choice between 2 activation functions, Sigmoid and Relu, is based on the residual block's task. For those residual blocks that handle the non-binary information, Relu will be chosen. While for those residual blocks that handle binary information, Sigmoid will be selected, as it could help the pixel value converge to either one or zero. For example, In PGT-Net, a binary task guides the non-binary task to produce better results. After processing through the shared features layers, the left side of the fork handles the binary task, and the right side deals with the non-binary task.

Also, there is a scaling factor \(\varepsilon\) in each residual block, \(\varepsilon\) is set to \(0.01\times\alpha\) initially, and it will decay 0.01 as the stage increase by one, as shown in Eq. 1. The residual scaling is proven helpful when training a deeper network, and such a technique makes the model more robust [27, 28, 29].

Note that the residual scaling factor \(\varepsilon\) could be either positive or negative. For example, the residual block at stage 15 will have \(\varepsilon\) = 0.01 \(\times\) (24 - 15) = 0.09, the residual block at stage 30 will have \(\varepsilon\) = 0.01 \(\times\) (24 - 30) = -0.06. We use positive scaling to extract fingerprint features and use negative scaling for denoising. Take PGT-Net-block-84 as an example, stage 1 to 23 are responsible for extracting shared features, so its scaling factor \(\varepsilon\) is set to be positive. Stage 24 to 60 are responsible for the binary or non-binary fingerprint image denoising, so they have a negative scaling factor \(\varepsilon\). Fig. 7 shows the residual blocks with positive scaling factor \(\varepsilon\) or negative scaling factor \(\varepsilon\). To avoid the scaling factor being zero, the residual scaling after stage 24 is subtracted 0.01 explicitly. Experiments have shown that such a residual scaling technique can achieve better performance, further analytic data can be referred to in section VI-C3.

\[\varepsilon=\begin{cases}0.01(\alpha-\text{current stage}),&if\text{current stage}<24\\ 0.01(\alpha-\text{current stage})-0.01,&otherwise\end{cases} \tag{1}\]

Fig. 4: The data flow of the proposed PGT-Net model.

Fig. 5: The network architecture of multi-task PGT-Net-block-84.

Fig. 6: The architecture of a residual block.

Fig. 7: Residual blocks with positive or negative scaling factor\(\varepsilon\).
#### Iii-B2 Binary Progressive Guided Task

We chose binary fingerprints as our guided task because the textures in binary fingerprints are more precise. For example, the textures are more evident in binary fingerprints than in non-binary ones. Thus binary fingerprints can provide a better quality of fingerprint contour information, which can guide the main task to find the critical information, such as the ridges and valleys of fingerprints, thus reaching a better denoising performance.

#### Iii-B3 Optimized Loss Function

Laplacian Operator is a differential operator widely used in image edge detection. We add Laplacian loss to our loss function because we need to emphasize the edge information between edges and valleys of a fingerprint so that the model learns how to restore the orientation of a noisy fingerprint.

Eq. 2 shows the definition of the Laplacian operator, the Laplacian of f is the sum of all unmixed second partial derivatives in the Cartesian coordinates \(x_{i}\).

\[\Delta f=\sum_{i=1}^{n}\frac{\partial^{2}f}{\partial x_{i}^{2}} \tag{2}\]

In practice, we usually use the Laplacian filter to convolve with the image to get the gradient of the image. Eq. 3 shows the discrete form of the Laplacian operator, namely, the Laplacian filter [3].

\[Laplacian\ filter=\begin{bmatrix}-1&-1&-1\\ -1&8&-1\\ -1&-1&-1\end{bmatrix} \tag{3}\]

Fig. 8a, b shows the result of a non-binary fingerprint convolves with the Laplacian filter. Fig. 8c, d shows the result of a binary fingerprint convolves with the Laplacian filter. As you can see, compared to the non-binary result, binary fingerprints emphasize the gradient, which makes the model easier to learn how to restore the texture of fingerprints.

Eq. 4, 5, 6, 7, 8 shows the loss function of a single-task used in PGT-Net. In addition to the mean squared error (MSE), we also add SSIM & Laplacian loss. SSIM in loss function can help to improve the performance in terms of SSIM in our experiments. And Laplacian loss can calculate the gradient of fingerprints, which allows the model to identify the orientations of fingerprints [3]. Eq. 9 shows the weighting of the loss function are empirically set as 0.7 for non-binary task (main task) and 0.3 for binary task (guided/supported task).

\[L_{MSE}=\frac{1}{N}\sum_{ij}(x_{ij}-y_{ij})^{2} \tag{4}\]

\[L_{Laplacian}=\sum_{ij}(\textit{Lap\ filter}(x_{ij})-\textit{Lap\ filter}(y_{ij})) ^{2} \tag{5}\]

\[SSIM(x,y)=\frac{(2\mu_{x}\mu_{y}+C_{1})(2\sigma_{xy}\mu_{y}+C_{2})}{(\mu_{x}^{ 2}+\mu_{y}^{2}+C_{1})(\sigma_{x}^{2}+\sigma_{y}^{2}+C_{2})} \tag{6}\]

\[L_{SSIM}=1-SSIM(x,y) \tag{7}\]

\[Single\ task\ loss=0.1L_{MSE}+0.2L_{Lap}+0.7L_{SSIM} \tag{8}\]

\[Total\ Loss=0.3\timesbinary\ task\ loss \tag{9}\] \[+0.7\times\textit{non\_binary\ task\ loss}\]

## IV Network Model on Edge Devices

The fingerprint-related applications are usually on edge devices with limited computing resources. As a result, the denoising algorithm can not be very complicated or needs heavy computations.

PGT-Net is also a neural network model with high scalability. One can adjust the number of the residual blocks or the architecture easily. As long as one follows the data flow described in section III, they can also get a model with excellent performance with acceptable downgrade but lower complexity.

PGT-Net-Edge provides an example of simplifying the model to reduce the computation resources on edge devices. One can follow similar steps to streamline their model according to their demands. Fig. 15(a) shows the architecture of the PGT-Net-Edge model. The PGT-Net-Edge reduces the complexity be three ways described below.

#### Iv-1 Residual Blocks Quantity Reduction

The number of residual blocks can be easily reduced, and the reduction can depend on the application. For example, in PGT-Net-Edge, the number of sigmoid residual blocks is reduced to zero, which handles the binary supported task initially.

#### Iv-2 Residual Blocks Output Channel Reduction

The output channels of residual blocks can also be adjusted according to the application. And reduce the output channel can significantly reduce the number of parameters. For example, in the original PGT-Net, all residual blocks have 64 output channels. While in PGT-Net-Edge, stage 1 to 28 has 32 output channels, and stage 29 to 32 has only 16 output channels.

#### Iv-3 Dynamic Fixed Point Quantization

[30] has proposed an algorithm that quantifies the floating-point weights or feature maps to dynamic-fixed-point. Compared to static-fixed-point quantization, dynamic-fixed-point quantization is more precise under the same precision and thus performs better. The experimental result of PGT-Net-Edge after quantization will be shown in section VI-A.

Fig. 8: Fingerprint convolves with the Laplacian filter. (a) normal fingerprint (b) normal fingerprint with Laplacian filter (c) binary fingerprint (d) binary fingerprint with Laplacian filter
## V Datasets

### _Denoising Datasets_

We use the FW9395 and FT-lightnoised two datasets for training. Both of the datasets are divided into training, validation, and testing and provided by FocalTech. It is described in Table I. Both of the datasets consist of a pair of blurry wet and ground-truth fingerprint images.

#### V-A1 FT-lightnoised Dataset

The FT-lightnoised dataset is collected by the Focaltech optical image sensor. The sensor captures the entire clean and wet fingerprints. After aligning the clean and wet pair, we obtain the small-area fingerprint images by cropping the entire images to size 176 x 36. Fig. 10a shows an example image pair.

#### V-A2 FW9395-synthetic Dataset

The dataset, FW9395-synthetic dataset, we used in the research is collected by the Focaltech capacitive image sensor. Because of the limitation of the hardware, the capacitive image sensor is hard to produce aligned wet & clean fingerprint pairs. So we used Gaussian noise to simulate the wet fingerprints noise. Fig. 10b shows an example pair of images in the FW9395-synthetic dataset. And the synthetic process was introduced in Section V-B2.

### _Data Preprocessing_

#### V-B1 Generate Binary Fingerprint Images

Binary fingerprints are used as supporting information to improve the wet fingerprints' denoising performance in our work. Friction ridges of a finger contain important features of fingerprint, therefore, recovering correct ridges plays a crucial role in the task. Thus, we generated reliable binary data from the original ground truth by fingerprint enhancer 1 which was based on Hong's research [5], so that these binary data will lead the model to learn better ridges features. Fig. 10c shows an example of a non-binary fingerprint and its corresponding binary fingerprint. The ridges of the fingerprints will set to 1, while the valleys of which will be 0.

Footnote 1: Fingerprint enhancer,[https://github.com/Utkarsh-Deshmukh/Fingerprint-Enhancement-Python](https://github.com/Utkarsh-Deshmukh/Fingerprint-Enhancement-Python).

#### V-B2 Synthesize Blurry Fingerprint Images

In FW9395, images are collected by a capacitive sensor, which has the difficulty to get aligned wet blurry and clean normal fingerprint data. To have correspondence between noised fingerprint and normal fingerprint for neural network training, we need to synthesize blurry fingerprint images from clean normal fingerprint images through the following steps.

1. Binarize the fingerprint. 1. Ridges in the original image will have a pixel value of 255 in the binarized image (white) 2. Valleys in the original image will have a pixel value of 0 in the binarized image (black)
2. Create Gaussian kernel with size 13, 15, 17, 19, 21, std_dev = 1
3. Set the following parameter 1. Appearance probability of noise = \(0.2\) 2. Darkness of noise = \(-0.2\) (the smaller the value, the darker the noise) 3. Darkness range = \((-0.01\,\ 0.01)\)
4. Select a pixel on the ridge (x, y), apply Gaussian kernel.
5. Set darkness_value = \((darkness\ of\ noise)+random(darkness\ range)\)
6. Update Gaussian kernel with the multiplication of Gaussian kernel and darkness_value
7. Apply Gaussian kernel to the chosen area.

The synthetic results are displayed in Fig. 9.

### _Recognition Datasets_

The fingerprint recognition tool is provided by FocalTech whose algorithm is based on the referred to SIFT [31], which is the same as the fingerprint recognition software applied to smartphones. The tool requires two inputs: the enrolled and the identified fingerprint images. The enrolled images are clean fingerprints and will be registered on each finger into the FocalTech recognition tool. The identified images are noisy fingerprints with water, grease, or sweat. After the identified fingerprint images are restored by the model, they will be put into the recognition tool to examine if the restored outputs are able to be recognized. The FT-lightnoised recognition dataset contains fingerprints from four people with different fingers. The FW9395 recognition dataset contains fingerprints from six people with different fingers. Table II summarizes the detailed information of the datasets and their corresponding FRR when the denoising algorithm is not applied.

## VI Experimental Results

### _Performance Summarization_

Table III summarizes the denoising performance of different models. Table IV summarizes the recognition performance of different models. Note that because the PGT-Net-Edge is optimized for the FW9395 dataset, it may perform poorly on the FT-lightnoised dataset, which has more complicated noise. Also, since the FW9395 dataset has simpler noise synthesized

Fig. 9: FW9395 synthetic data examples.
artificially, the differences between single-task and multi-task are not very much. For the more complicated FT-lightnoised dataset, we can see the value of the proposed multi-task model.

To summarize, Fig. 11 & Fig. 12 show the results for the FT-lightnoised dataset. Fig. 13 shows the results for the FW9395 dataset. The three figures mentioned above are all based on PGT-Net-block-84, the model with the best performance. As you can see from those figures, the proposed algorithm has an outstanding result, with different noise levels or sensors.

As mentioned in section IV, the PGT-Net-Edge reduces the complexity be three ways described below.

* Residual blocks quantity reduction.
* Residual blocks output channel reduction.
* Dynamic fixed point quantization.

Table V shows the experimental results after quantization. There are few performance downgrades, but the model size has been quartered after quantization from float32 to the 8-bit dynamic-fixed point. As you can see from the tables, there are only a few performance downgrades compared to the

Fig. 11: PGT-Net-block-84 denoised and recognition results on the FT-lightnoised dataset (light and medium noise).

Fig. 14: The denoised images and recognition results with different residual scaling settings.

Fig. 12: PGT-Net-block-84 denoised and recognition results on the FT-lightnoised dataset (heavy noise).

Fig. 10: Fingerprint pairs on different datasets. (a) FT-lightnoised wet and clean pair (b) FW9395 wet and clean pair (c) clean fingerprint and corresponding binary fingerprint
PGT-Net-block-84, which have similar dataflow but a higher quantity of parameters.

### _Performance Comparison_

Table VI summarizes the experiment results of the fingerprint denoising-related works mentioned above. Table VII summarizes the experiment results of fingerprint recognition compared with related works. According to these two table, we observe that PGT-Net performs well on denoising and recognition tasks. From Table VII, PGT-Net yields substantial improvement in fingerprint recognition tasks, with a minor increase in the FAR. The FAR slightly elevate from 0.02% to 0.07%. One potential explanation is that during the restoration process of our proposed model, features belonging to fingerprints from other classes are inadvertently restored, as PGT-Net did not incorporate fingerprint categories in its training process. We consider this to be an acceptable trade-off when weighed against the enhancement in FRR.

### _Ablation Studies_

Here we present ablation experiments to analyze the contribution of each component of our model. Evaluation is performed on the FW9395 and FT-lightnoised datasets.

#### Iv-C1 Single-task versus Multi-task Model

Our model yields better performance as Multi-task model(Fig. 5). Fig. 15 shows an architecture of the PGT-Net-single-task; it is similar to the original PGT-Net model but in the single-task version, and the number of the residual blocks are the same, so they have a similar quantity of parameters. Also, both single-task and multi-task have the same residual scaling setting with \(\alpha\) = 24, as described in Eq. 1, and thus a fair comparison. We demonstrate the evaluation results of single-task and Multi-task model training with FT-lightnoised dataset.
fingerprint from the first column. Still, in some details, multi-task has a more accurate fingerprint contour (marked in red boxes), and such information makes the denoised fingerprint of multi-task recognized successfully. In conclusion, the binary branch did help in fingerprint restoration.

#### V-B2 Binary Progressive Guided Task

Here we use some techniques to improve the performance of our Multi-task model.

* _Progressive Guided Concatenation Path:_ The model with the "binary guided concatenate path" is shown in Fig. 5, and the model without the "Progressive guided concatenation path" is modified from the original PGT-Net with the "Progressive guided concatenation path" begin deleted. Experiments shows that the concatenate path can improve the performance.
* _Separate the Training Process:_
* Phase 1: Train the binary-related parameters in our model first.
* Phase 2: Do the regular training. Such a technique can improve the performance because the binary path in the model is pre-trained and can produce a more precise binary fingerprint contour to guide the model toward better performance more accurately. Fig. 15c shows the partial model that can be trained during the 2-Phase training process. Yellow blocks are trainable both in Phases 1 and 2. Gray blocks are only trainable in Phase 2.

Table VIII summarizes the denoising performance with different settings. As you can observe from the table, the "progressive guided concatenation path" and "progressive 2-stage training" can make the model performs better.

#### V-B3 Residual scaling factor \(\varepsilon\)

As described previously in section III-A1

* Each residual block has a constant \(\varepsilon\) for residual scaling, as shown in Fig. 6
* The scaling factor \(\varepsilon\) can be either positive or negative, as shown in Fig. 7.
* In our work, we set \(\alpha\) to 24 in Eq. 1

If we set the \(\varepsilon\) as the FENet [3], such that all the residual blocks have positive \(\varepsilon\), namely \(\varepsilon=0.01\times\alpha-current\)\(stage)\), with \(\alpha=61\) or \(85\), the training loss will significantly increase, and thus a poor recognition performance.

There are two different settings with positive \(\varepsilon\), as described below:

* \(\alpha=61\) because the maximum stage of PGT-Net-block-84 is 60. And the model architecture of this setting is exactly the same as PGT-Net-block-84 (Fig. 5).
* \(\alpha=85\) because there are 84 residual blocks in PGT-Net-block-84. Although the model architecture is similar to PGT-Net-block-84, it still exists some differences. The stage of the non-binary branch starts at 61 instead of 25, ends at stage 84 instead of 48, as shown in Fig. 15c. The non-binary branch starts at 61 because it has the input comes from the binary output, which ends at stage 60.

The training loss of the residual scaling setting mentioned above is shown in Fig. 16. The proposed residual scaling can further reduce the training loss and thus achieve better denoising and recognition performance.

Table IX summarizes the denoising performance for the F-lightnoised dataset, and Table X summarizes the recognition performance for the FT-lightnoised dataset. The proposed residual scaling can indeed perform better in our experiments.

Fig. 14 shows the denoised output of different residual scaling settings. Those images denoised with all positive residual scaling settings can't be recognized successfully, but the output fingerprint of proposed residual scaling can. Although all of them can produce denoised clear fingerprint output, on some details (marked in red boxes), proposed residual scaling makes

Fig. 16: Training loss of different residual scaling setting.

Fig. 15: PGT-Net variants. (a) The architecture of PGT-Net-Edge (b) Single-task version of PGT-Net-block-84 (c) Two-phase training process of PGT-Net
the denoised fingerprint texture more precise and thus have lower FRR.

## VII Conclusion

In this work, we presented PGT-Net for wet fingerprint denoising. The proposed methodologies have proven to be effective in multiple types of sensors. And can recover the noisy fingerprints covered with real and synthetic noise. PGT-Net uses residual blocks as the fundamental structures. And considering that the fingerprints get by the sensor would be small and thin, multi-task architecture is added in PGT-Net to use the results of supported tasks to guide the main task to reach better performance. Also, the proposed residual scaling did a great job at reducing the training loss.

The proposed data flow is simple and scalable. One can easily modify the model according to the proposed data flow and will get a result with an excellent denoising performance.

The convenience of fingerprint recognition in our daily usage has also been greatly improved. The proposed methodologies have proven effective in both optical and capacitive sensors. The FRR has been reduced from 17.75 % to 4.47 % for the FT-lightnoised dataset. FRR has been reduced from 9.45 % to 1.09 % for the FW9395 dataset.
