**Deep Binary Classification via Multi-Resolution Network and Stochastic Orthogonality for Subcompact Vehicle Recognition**

#### **Joongchol Shin † , Bonseok Koo † , Yeongbin Kim and Joonki Paik \***

Department of Image, Chung-Ang University, Seoul 06974, Korea<EMAIL_ADDRESS>(J.S.);

<EMAIL_ADDRESS>(B.K<EMAIL_ADDRESS>(Y.K.)

**\*** Correspondence<EMAIL_ADDRESS>Tel.: +82-10-7123-6846

† These authors contributed equally to this work.

Received: 17 March 2020; Accepted: 7 May 2020; Published: 9 May 2020

**Abstract:** To encourage people to save energy, subcompact cars have several benefits of discount on parking or toll road charge. However, manual classification of the subcompact car is highly labor intensive. To solve this problem, automatic vehicle classification systems are good candidates. Since a general pattern-based classification technique can not successfully recognize the ambiguous features of a vehicle, we present a new multi-resolution convolutional neural network (CNN) and stochastic orthogonal learning method to train the network. We first extract the region of a bonnet in the vehicle image. Next, both extracted and input image are engaged to low and high resolution layers in the CNN model. The proposed network is then optimized based on stochastic orthogonality. We also built a novel subcompact vehicle dataset that will be open for a public use. Experimental results show that the proposed model outperforms state-of-the-art approaches in term of accuracy, which means that the proposed method can efficiently classify the ambiguous features between subcompact and non-subcompact vehicles.

**Keywords:** vehicle recognition; multi resolution network; optimization

### **1. Introduction**

Typically, subcompact cars are defined by the engine displacement, width, and height under 1000 cc, 1.6 m, and 2.0 m, respectively. To satisfy these specifications, the subcompact car has a unique shape such as shorter-bonnet and hatchback. In addition, there are various environmental benefits because the subcompact cars have a small displacement engine and a light weight. To encourage people to drive subcompact cars, many countries provide several benefits—discounts on tall road charge and parking fee. Since classification of subcompact cars from other requires labor-intensive human investigation, an automatic vehicle classification system is needed. In general, vehicle classification methods can be classified into two approaches: one uses infrared sensors to measure physical dimensions of a vehicle such as length, height, and width. The other uses a single camera and image processing algorithms to recognize the visual characteristics of vehicles [1,2]. Despite of accuracy and robustness, the infrared sensor-based system is too expensive to be installed in many places. Thus, we propose an image recognition system to reduce the installation and maintenance cost. To classify the visual feature in images, Dalal et al. extracted the histogram of oriented gradients (HOG) and classify the HOG using support vector machine (SVM) [3]. To the best of authors' knowledge, the HOG-based SVM is the most popular approach to recognize objects before deep learning has become popular. To enhance HOG features that are affected by rotation, or distance, and occlusion, various approaches were proposed. Llorca et al. proposed vehicle manufacturer recognition by detecting the vehicle-logo, but a subcompact car can not be completely classified using only manufacturer information [4]. Clady et al. recognized the vehicle type by separating objects and the background in interactively selected regions [5]. This method is robust to the variance in the distance. However, the region should be passively selected. Mohottala et al. created vehicle images using computer graphics (CG), and then classify the type of vehicle using eigenvalues [6].

Although this approach can easily obtain the vehicle data, it cannot avoid error in real vehicle data. Michael and Daniel classified the eigenvalues of vehicle classes using neural networks [7]. Since they used an artificial neural network, classification accuracy was acceptable only without occlusion. Huttunen et al. adaptively recognized the vehicle classes using a deep neural network [8]. This method can recognize the multi-class vehicles such as sedan, truck, and bus. However, in subcompact car classification, it has overffiting while learning the subcompact vehicle class because it is difficult to discriminate the subcompact vehicle from others. Simonyan et al. proposed the deep convolutional neural networks called VGG16 and VGG19 [9]. Since the VGG networks can be pre-trained via a large-scale image dataset [10], it can have a very deep hidden-layer to recognize the vehicle. However, it cannot robustly classify the subcompact vehicles because of both obscure features and environmental variables as shown in Figure 1. He et al. proposed the more deep residual networks [11]. This network can be designed more deeply such as 50, 101, 151 layers because of the residual learning. However, in the binary classification, the VGG networks are also deep enough. Xie et al. applied the split-transform-merge strategy to deep residual networks [12]. This strategy can effectively recognize various features, but it can not adaptively crop the image region. Karpathy et al. proposed the multiple convolutional neural networks with center clipping and image fusion for video classification [13]. It can recognize the obscure objects and actions in video, but it cannot localize objects that are not in the center of the image.

**Figure 1.** (**a**) Subcompact vehicles and (**b**) sedans. It is not easy to differentiate two classes using small features such as head lamp or rear-view mirror. On the other hand, there are differences in bigger features such as bonnet and overall shape of vehicles.

To solve this isolated problem, we proposed a novel multi-resolution network and stochastic orthogonal learning method. More specifically, the proposed method include three functional steps: (i) we emphasize the features using retinex model-based image-enhancement [14], (ii) we track the bonnet region using an optimized correlation filters [15], and (iii) we engage this region and image using the proposed multi-resolution network. In addition, we learned our muti-resolution network by considering stochastic orthogonality of probabilities between subcompact and general vehicles. We also build a subcompact vehicle dataset including 1500 training data and 2000 test-images. Experimental results show that the proposed method outperforms state-of-the-arts approaches in terms of accuracy by over 12.25%. This paper is organized as follows. In Section 2, we describe the related works. The proposed multi-resolution network is presented in Section 3 followed by experimental results in Section 4, and Section 5 concludes this paper.

#### **2. Related Works**

#### *2.1. Support Vector Machine*

To classify the features in images, the SVM can be applied by minimizing as following Equation [16]

$$\text{1: } \underset{\vec{w}, b}{\text{arg min}} \frac{1}{2} \|\vec{w}\|^2 - \sum\_{i=1} a\_i \left( y\_i \left( \vec{w} \bullet \vec{x}\_i + b \right) - 1 \right) \tag{1}$$

where *w*~ and *b* represent weight and bias of hyper-plane to classify the features, *α* is an operator to find the support vectors, and *y* denotes the label such as positive or negative. The optimized hyper-plane of the support vector machine works well, but it should be estimated low-dimensional features such as histograms of gradients and scale-invariant features [3,17] to apply imaging systems.

#### *2.2. Neural Network*

The neural networks can classify the non-linear features because the each node in hidden-layer discriminates the complicated patterns as shown Figure 2. Each node includes the weight, bias, and activate function such as sigmoid, and relu. These parameters can be easily estimated by simple cost function and chain rule as

$$E\_{\text{total}} = \sum \frac{1}{2} (y - f(\mathbf{x}))^2,\tag{2}$$

and

$$\begin{array}{lcl}\frac{\partial E\_{\text{total}}}{\partial w\_{\ast}} = \frac{\partial E\_{\text{total}}}{\partial out\_{o1}} \* \frac{\partial out\_{o1}}{\partial net\_{o1}} \* \frac{\partial net\_{o1}}{\partial w\_{\ast}},\\ \frac{\partial E\_{\text{total}}}{\partial b\_{\ast}} = \frac{\partial E\_{\text{total}}}{\partial out\_{o1}} \* \frac{\partial out\_{o1}}{\partial net\_{o1}} \* \frac{\partial net\_{o1}}{\partial b\_{\ast}},\end{array} \tag{3}$$

where *f* returns the results of neural networks, *w*∗ and *b*∗ are weight and bias in ∗-th node. Therefore, each parameter can be estimated as

$$\begin{cases} w\_\*(t+1) = w\_\*(t) - \frac{\partial E\_{\text{total}}}{\partial w\_\*}, \\ b\_\*(t+1) = b\_\*(t) - \frac{\partial E\_{\text{total}}}{\partial b\_\*}. \end{cases} \tag{4}$$

However, the neural net also has limitation to apply large-scale image classification.

**Figure 2.** The architecture of neural network.

#### *2.3. Convolutional Neural Network (CNN)*

Since an image has various features such as gradients, color, and intensity information, the convolution operators in the hidden layer are effective to extract the image features [9]. Furthermore, these convolution operator can also be optimized by chain-rule. For example, we visualize the extracted

image features in convolution layer using CNN feature simulator [18]. Note that the convolution operator can extract the large scale features and textures as shown Figure 3. In other words, the CNN can not only classify the multi-class image but also recognize the detail textures. Therefore, the CNN can be applied to various field using the transfer learning method such as medical imaging [19], intelligent transportation system [20,21], and remote sensing [22]

**Figure 3.** The convolutional neural network and convolution features.

### **3. Proposed Method**

### *3.1. Subcompact Vehicle Dataset*

To train and test the proposed network, we collected vehicle images using a digital camera (gray-scale) at a parking gate in Seoul, South Korea. Since the camera was installed under the charge machine, images were captured from an angle viewed from below as shown in Figure 4. Furthermore, we collected vehicle images for 1 year and 6 months to reflect various environmental variables such as day light, back light, dust, and night. The collected images were classified into five types including subcompact sedan, subcompact van, subcompact truck, sedan and sport utility vehicle (SUV), and truck and van according to the design and shape. The dataset was split into training and test sets including 1500 and 2000 images, respectively. The goal of this work is the binary classification (subcompact vehicle or not). To this end, each set of the proposed dataset is divided into subcompact and non-subcompact vehicles as shown in Figure 5.

**Figure 4.** (**a**) Camera installation and (**b**) four different illumination conditions.

**Figure 5.** An example of five vehicle classes in the proposed dataset.

#### *3.2. Pre-Convolution Layer*

The proposed network consists of pre-convolution and multi-resolution network layers. In the pre-convolution layer, we resize the original 1920 × 1080 px images to 400 × 300 px, and we amplify the intensity and increase the local-contrast using a simple retinex-based image enhancement algorithm as [14]

$$H(\mathbf{x}) = \frac{I\left(\mathbf{x}\right)}{\max\left(l\left(\mathbf{x}\right)\varepsilon\right)},\tag{5}$$

where *H* represents high-resolution image, *I* the input gray-image in the dataset, and *ε* is a very small positive number to avoid division by zero. The illuminance map *l* can be estimated as a smoothing term [23]

$$l = \text{med}\left(I\right) - \text{med}\left(\left|\text{med}\left(I\right) - I\right|\right),\tag{6}$$

where *Med* represents the local-median filter [24]. Figure 6a,c show that the environmental variables can be normalized, and at the same time local-contrast in the shadow region is also enhanced. To process a low-resolution image, the vehicle region should be localized. In this paper, we detect the region using a correlation filter which has low-computational complexity and efficient localization performance [25]. To reflect a feature of texture, we applied multi-channel correlation filter (MCCF) with the histogram of oriented gradients, which can be defined as ridge regression

$$E\left(w\right) = \frac{1}{2} \sum\_{i}^{N} \sum\_{j}^{D} \left( y\_i\left(j\right) - \sum\_{k=1}^{K} w^{(k)T} x\_i^{(k)} \left[\Delta \tau\_j\right] \right)^2 + \frac{\lambda}{2} \sum\_{k=1}^{K} \left( w^{(k)} \right)^2,\tag{7}$$

where *yi*(*j*) is the desired and shifted response in i-th sample *yi* = [*yi*(1), ...., *yi*(*D*)]*T* , *xi* [∆*τj* ] is a set of cyclically shifted vehicle images in the training dataset. *N* represents the number of training images, *K* is the channels of feature map including HOG 34-channels, and *w* represents the correlation filter. The response map *y* for a vehicle coordinate in the frequency domain has a Gaussian-shaped distribution centering on a pre-annotated region. Since both input patch and response map are circular matrices for cyclic convolution, the correlation filter *w* can be simply expressed in the frequency domain as

$$
\hat{w}^\* = \left(\lambda I + \sum\_{i}^{N} \hat{X}\_i^T \hat{X}\_i\right)^{-1} \sum\_{i=1}^{N} \hat{X}^T \hat{y}\_{i\prime} \tag{8}
$$

where, *w*ˆ represents a variable *w* in the frequency domain, ∗ and *T* respectively represent the complex conjugate and transpose of a matrix. The optimized correlation filter can estimate the coordinates of the vehicle region via maximum response-region and distribution as shown in Figure 6d–f. We cropped high-resolution images, based on this picking coordinates to obtain low-resolution images *L* (280 × 200). Note that the proposed pre-convolution layer should be processed in Central processing unit (CPU) for efficient memory allocation.

(**d**) Feature extracting (**e**) Result of the correlation filter (**f**) Low-resolution image

**Figure 6.** Step-by-step results in the pre-convolution layer (**a**–**f**).

#### *3.3. Multi-Resolution Network*

We changed the size of both high- and low-resolution images to 224 × 224 to recognize vehicle type. Note that the scale of the proposed subcompact dataset is gray. Therefore, we generate the 3 zero-min channels with the average color of ImageNet [10], and concatenate 3 zero-min channels to create a pseudo color as

$$H^R = H - 0.4850,\ H^G = H - 0.4580,\ H^B = H - 0.4076,\tag{9}$$

and

$$L^R = L - 0.4850,\\ L^G = L - 0.4580,\\ L^B = L - 0.4076,\tag{10}$$

where *H* and *L* are single gray-scale (224 × 224 × 1), and *H*∗ and *L* ∗ have pseudo RGB channel (224 × 224 × 3) as shown as high- and low-resolution in Figure 7. We correspondingly defined both high- and low-resolution network with 13 convolution layers, 5 max-pooling layers, and 3 fully connected layers as shown in Figure 7. A 3 × 3 filter is used in each convolution layer, ReLU is used for an activation function, and 2 × 2 max-pooling filters are used to maximize the receptive field [9]. Each fully connected layer has 4096 perceptrons, except for the last five layers. Finally, the soft-max operator returns the probability using returned five values. In this paper, the proposed multi-resolution network is combined to our pre-convolution layer described in Section 3.2.

**Figure 7.** The architecture of the proposed multi-resolution network. Blue, black, and red cubes represent the resized input, convolution, and max-pool layers, respectively. Black and purple boxes are fully-connected and softmax layers, respectively.

#### *3.4. Orthogonal Learning*

To combine the results of low- and high-resolution networks, we define the average least square loss as

$$L\_{least} = \frac{1}{2B} \sum\_{n=0}^{B} \sum\_{i=1}^{N} \left( g\_i^n - P\_i \left( H\_n^\* \right) \right)^2 + \left( g\_i - P\_i \left( L\_n^\* \right) \right)^2 \tag{11}$$

where *Hn* and *Ln* respectively represent the *n*−th high and low resolution image, *B* denotes the size of batch, and *N* is the vehicle type between C1 to C5. *g* represents the one-hot vector of size *B* × 5, which is pre-labeled in our dataset described in Section 3.1. To reduce the correlation between two groups, we also define the orthogonal loss as [26]

$$L\_o = \frac{1}{B} \sum\_{n=0}^{B} \sum\_{i=1}^{2} \left( P\_i \left( H\_{n'}^\* L\_n^\* \right) P\_3 \left( H\_{n'}^\* L\_n^\* \right) + P\_i \left( H\_{n'}^\* L\_n^\* \right) P\_4 \left( H\_{n'}^\* L\_n^\* \right) + + P\_i \left( H\_{n'}^\* L\_n^\* \right) P\_5 \left( H\_{n'}^\* L\_n^\* \right) \right). \tag{12}$$

In binary classification, the error can be reduced when sum of multiplication between subcompact and other group is closed to zero as shown in Figure 8. Therefore, the proposed total loss can be defined as

$$L\_{\text{total}} = L\_{\text{least}} + L\_0. \tag{13}$$

To reduce the total loss *Ltotal*, the set of parameters including convolution kernel, bias, and perceptron weight are updated via stochastic gradient decent optimization [9]. The learning and dropout rates are set to 0.0001 and 0.5, respectively. For supervised learning, we train the model using 1500 labeled training data given in Section 3.1. For transfer learning, all of convolution layers are pretrained by ImageNet data [10]. We trained the proposed model using 4500 iterations, and 15 batches are engaged to the proposed multi-resolution network for each learning. Figure 9 shows the proposed learning procedure. Finally, to distinguish subcompact vehicle, the probability values of the optimized model are estimated using the thresholding operator as

$$D\_n = \begin{cases} \begin{array}{l} \text{False} \\ \begin{array}{l} \text{True} \end{array} \text{arg}\max \frac{1}{2} \left( P\left(H\_n^\*\right) + P\left(L\_n^\*\right) \right) \in \{\text{C1}, \text{C2}\} \\ \text{True} \end{array} \\ \begin{array}{l} \text{arg}\max \frac{1}{2} \left( P\left(H\_n^\*\right) + P\left(L\_n^\*\right) \right) \in \{\text{C3}, \text{C4}, \text{C5}\} \end{array} \end{cases} \tag{14}$$

**Figure 8.** An example of orthogonal learning.

**Figure 9.** Dual procedure of the proposed method using both CPU and GPU.

#### **4. Experimental Results**

#### *4.1. Quantitative Evaluation*

To evaluate the proposed method, we compared experimental results with 2000 test data and state-of-the-arts classification models including HOG based recognition model (*HOG* + *SVM*) proposed by Dalal et al. [3], MCCF combined HOG recognition (*MCCF* + *HOG* + *SVM*) [15], multi-resolution image based HOG recognition (*Retinex* + *MCCF* + *HOG* + *SVM*), deep neural network based Huttunen's method (*DNN*) [8], convolutional neural network (*CNN*) with 16 layers proposed by Simonyan et al. [9], retinex CNN *Retinex* + *CNN*, based on proposed pre-convolution layer (*Retinex* + *MCCF* + *CNN*), and the proposed multi-resolution network without orthogonal learning (*Retinex* + *MCCF* + *MRN*). All the algorithms were implemented in visual studio 2015 and Python 3.5 using on a desktop PC with i7 CPU, 64 GB RAM, and NVIDIA RTX 2080ti graphics processing unit (GPU). We also quantitatively measured accuracy (Acc.), precision, recall, and false-positive rate (FPR) as

$$precision = \frac{TP}{TP + FP'} \tag{15}$$

$$Recall = \frac{TP}{TP + FN} \,\tag{16}$$

$$FPR = \frac{FP}{TP + FP'} \tag{17}$$

and

$$Acc. = 100 \times \frac{TP + TN}{2000} \,\text{.}\tag{18}$$

where TP, TN, FP, and FN respectively represent the true-positive, true-negative, false-positive, and false negative. Table 1 shows several evaluation results using state-of-the arts and the proposed

methods. Since *HOG* uses handcraft-based features, its recognition performance is limited for ambiguous features. *HOG* + *SVM* method results in many mis-classification cases represented by *FN* and *FP* as shown in Table 1.

**Table 1.** Quantitative comparison with state-of-the art approaches.

*MCCF* + *HOG* + *SVM* method can improve the false-positive case because MCCF based localization effectively removes unnecessary information such as background, but FN case can be increased. Since retinex-based image enhancement enhance too much textures, *MCCF* + *HOG* + *SVM* outperforms *Retinex* + *MCCF* + *HOG* + *SVM* in every sense. The deep neural network (*DNN*) can effectively increase the TP compared with SVM-based methods, but false-positive rate was slightly higher than *MCCF* + *HOG* + *SVM*. Simonyan's convolutional neural network model can improve both TP and TN, so accuracy was highly increased over 4% than both *DNN* and *SVM* based methods, Especially, false-positive rate rapidly decreases compared with DNN and SVM based method, but accuracy is not enough because of the imbalance between true positive and false negative. Since the enhanced textures in a shadow region can compensate for the imbalance, the retinex-based CNN model (*Retinex* + *CNN*) outperforms the vanilla *CNN* in terms of the recall, accuracy. As a result of the localization error of *MCCF*, *Retinex* + *MCCF* + *CNN* can generate errors such as FN and FP, but the combined version *Retinex* + *MCCF* + *MRN* outperforms *CNN* models in all of evaluation terms. This is mean that the proposed *MRN* can adaptively reflect between localized information and enhanced textures to recognize the sub-compact vehicle. Furthermore, the proposed orthogonal learning method has TP values higher than *Retinex* + *MCCF* + *MRN* because it can generate the uncorrelated group-probability vectors. Note that the precision, recall, fpr, and accuracy are better by 0.1268, 0.45, 1.268, and 12.25% than convolutional neural network (*CNN*). In conclusion, the proposed approach can effectively classify the ambiguous objects because it designed and optimized with consideration of the group-error and ambiguous features. In addition, the multi-class recognition performance is compared with the *CNN* model as shown in Figure 10. The *CNN* method misclassifies between sedan and subcompact vehicles many times. Furthermore, subcompact truck and van are sometimes mis-recognized by the *CNN* method. However, the proposed method can not only reduce the mis-recognized case but also improve the accuracy by 10.85%. In addition, we conducted ablation study using validation check as shown in Figure 11. When we train the MRN without pseudo color (9), the performance can be degraded as shown black line in Figure 11 because the MRN was pre-trained by true color image dataset. The center crop-based MRN(Center crop) means that the MCCF operator in proposed pre-convolution layer was replaced to center cropping method [13]. Since the center cropping method can not adaptively localize the object, it can not outperform than proposed MRN. Furthermore, if MRN was optimized by only least square loss (11), the extracted features are not suitable for binary classification. Thus, when MRN was learned without proposed orthogonal loss (12), the performance of the MRN can not reach the orthogonal learning-based MRN as shown in Figure 11.

Convolutional Neural Network [9] Multi-Resolution Network

**Figure 10.** Multi-classification comparison.

**Figure 11.** Ablation study: MRN (*Lleast* + *Lo*) is the proposed orthogonal learning based multi-resolution network (MRN), MRN (*Lleast*) is the least square loss based MRN, MRN (Center Crop) is the center cropping method to generate the low-resolution image, and MRN (Gray) is a single channel based MRN.

#### *4.2. Baseline Comparison*

To compare the efficient baseline networks, we evaluate the accuracy with several efficient networks such as VGG16 [9], residual network50(resnet50) [11], and resinext50 [12]. Table 2 shows the maximum binary, and multi class accuracy values. Since general networks do not consider ambiguous in binary classification problem, the MRN outperforms than several based line networks in terms of both binary and multi-class accuracy. The residual network based MRN slightly lower than the VGG16 based MRN because the VGG16 have already deep layers in binary classification. However, resnext based MRN outperforms the VGG16 based MRN in terms of binary accuracy because the split-transform-merge strategy can effectively apply to recognize the ambiguous binary objects. Figure 12 shows the accuracy for each training epoch. Note that the proposed networks outperform than state-of-the-arts baseline-networks in most epoch. In addition, we recorded computational complexity and allocated GPU-memory on average to verify the computational efficiency.

**Table 2.** Effects on the accuracy for different baseline networks.

**Figure 12.** Accuracy evaluation according to each epoch.

#### *4.3. Visualization*

To verify the performance of the proposed orthogonal learning, we visualized the output values of last layer in the both CNN [9] and the proposed MRN by projecting to two-dimensional space using t-stochastic neighbor embedding (t-SNE) [27]. In Figure 13a, the visualization is not easy because of the correlation between subcompact vehicle and other groups. However, Figure 13b shows that the points are clustered to easily classify, which means that the proposed orthogonal learning can remove the group-correlation. As a result, proposed orthogonal learning can improve the performance of deep binary classification.

In Figure 14, we visualized the classified label and localized regions, where the localized bonnet region is the input to the low-resolution network, and the entire image is engaged to the high-resolution network. The resulting label (subcompact and non-subcompact vehicle) based on the average value of the two probabilities is reflected at the end of red-box. The proposed MRN can not only classify in various illuminations but also distinguish the ambiguous vehicle types.

(**a**) Convolutional Neural Network (**b**) Proposed Method

**Figure 13.** Probability Visualization using t-Stochastic Neighbor Embedding (t-SNE) [27].

Classification and Localization Result **Figure 14.** Classification and localization result.

#### **5. Conclusions**

To recognize ambiguous features between the subcompact and other vehicles, we collected a novel set of subcompact vehicle images, and proposed a pre-convolution layer that is combined with the multi-resolution network with an orthogonal learning method. The proposed method can not only enhance the textures using retinex-based enhancement but also adaptively cropped the bonnet region using correlation computation. As a result, our MRN can avoid over-fitting by ambiguous features between vehicle types, and outperforms the existing CNN(VGG16) method by 12.25%. Therefore, the proposed method can be applied to various traffic management systems such as toll and parking gates for automatic charging system. In the future work, we will expand the proposed MRN by combining the license plate detection system. The source code is available at https://github.com/JoongcholShin.

**Author Contributions:** Conceptualization, J.S. and Y.K.; methodology, J.S., B.K.; software, J.S., B.K., and Y.K.; validation, J.S. B.K., and Y.K.; formal analysis, J.S., B.K., and Y.K.; investigation, J.S., B.K., and Y.K.; resources, J.P.; data curation, J.S., B.K., and Y.K.; writing—original draft preparation, J.S., B.K., and Y.K.; writing—review and editing, J.P.; visualization, J.S and B.K; supervision, B.K., Y.K., and J.P.; project administration, J.P.; funding acquisition, J.P. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was partly supported by Institute for Information & communications Technology Promotion( IITP) grant funded by the Korea government(MSIT) (2017-0-00250, and by Intelligent Defense Boundary Surveillance Technology Using Collaborative Reinforced Learning of Embedded Edge Camera and Image Analysis) and supported by the Chung-Ang University Research Scholarship Grants in 2020.

**Acknowledgments:** We acknowledge support given by Nexpa System Co., Ltd.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).


**7.2 Dental enamel**

162 Atomic Force Microscopy – Imaging, Measuring and Manipulating Surfaces at the Atomic Scale

Fig. 19. Three individual spaces in a surface profile. The mean spacing is the average of the

Similar to the Rq and Ra, RMS Average Wavelength (λq) takes as reference the root mean square of the spacing between peaks and valleys weighted by their individual frequencies

q

q R

Δ (12)

three individual spaces on the evaluation length (Zygo Corporation, 2011).

and amplitudes (Park, 2011). RMS Average Wavelength can be calculated by:

Where Rq is the RMS roughness and Δq the RMS slope of profile (Park, 2011).

spectroscopy). This process is known as electrochemical intercalation.

q

λ 2π

The roughness is a very significant parameter for various applications. The characterization of materials through its roughness allows one to obtain information on the efficiency of

Some materials, usually transition metals oxides, are classified as intercalation materials. These materials are able to receive short radius ions (H+, Li+...) in its network structure via electrochemical techniques (such as cyclic voltammetry and electrochemical impedance

The electrochemical intercalation is a reversible process, making these materials very interesting in applications where the control over the ability of insertion/extraction of an ion

Intercalation materials are commonly used in micro-batteries, smart windows, smart mirrors, displays, gas sensors and other applications. These materials have potential applications due to the possibility of control of their electronic and optical properties.

The electro-physical-chemical properties of intercalation materials are strongly dependent

**6.2.6 RMS average wavelength**

**7. Applications (materials)**

materials in various application areas.

**7.1 Electrochemical intercalation**

in a structure is essential.

on surface roughness.

Where Ra is the roughness average and Δa the mean slope of profile (Park, 2011).

The bleaching process can cause variation in the surface roughness of human tooth enamel. These changes are responsible for color changes, glare reduction, opacity.

The surface roughness is responsible for diffuse scattering of light incident on tooth enamel. Thus the surface roughness is an important variable in the bleaching process and must necessarily be considered.

The search for materials that do not significantly increase the roughness of the tooth surface is a challenge in dentistry. Bleaching agents based on nanoparticles of hidroxiapatatia have been shown to be effective by reducing the surface roughness and increasing the brightness of the tooth enamel (Takikawa et al., 2006).

The brightness and surface roughness are closely associated by an inverse correlation. As shown in the figure 20, the gloss increases with decreasing roughness of tooth enamel (Heintze et al., 2006).

#### **7.3 X-ray anode**

The anode is the positive electrode in an X-ray tube. It receives the impact of electrons accelerated by the potential difference due to the high voltage applied. The anode is generally made of materials that have high thermal dissipation, such as copper, molybdenum or rhenium. Depending on the x-ray application (i.e. energy of x-ray) a metal coating such as tungsten (W) or molybdenum (Mo) is placed over these thermally dissipative metals at the impact area of the accelerated electrons.

A change in the spectral distribution of X-rays in a cathode ray tube with increasing roughness of the anode has been observed. The increased surface roughness implies an increase of characteristic peaks and a decrease corresponds to the lower energies of the bremsstrahlung spectrum, and an increase in the average energy of beams of X-rays (Nagel, 1988; Stears et al., 1986). The increased surface roughness implies an increase of characteristic peaks and a decreasing in the part corresponding to the lower energies of the bremsstrahlung spectrum. Unlike the filtration process for tungsten (W) where a dip occurs at lower energies (Yoriyaz et al., 2009).

#### **7.4 Polymeric membranes**

In the area of environmental protection, a very significant technology is the process of separation by polymeric membranes. Polymeric membranes, such as Polysulfone / Blend Membrane PLURONIC F127, are used to separate the undesired solute in solution. Thus, the active area of the polymer membrane to carry out the process is the surface. The properties related to the surface are important for performing the separation process. Properties such as the pores size distribution, long-range electrostatic interactions and surface roughness are factors that determine the efficiency of polymer membrane for this application.

Measurement of the Nanoscale Roughness by

**8. Effect of RMS roughness on adhesion**

binding of atoms and molecules.

contact, thus reducing adhesion.

of the tip used in AFM.

the effects that surface roughness plays on adherence.

Atomic Force Microscopy: Basic Principles and Applications 165

The surface roughness of the polymer membrane is a factor proportional to the bond strength of the membrane. The higher roughness leads to greater adhesive strength of the

In the study of surfaces, related applications to adherence are extremely important. The surface morphology plays a significant effect on adherence. In this section we will discuss

Adherence is a chemical-physical phenomenon responsible for the union of two surfaces when they come into contact. This union has force of high magnitude in conditions where there is a chemical bond with sharing of electrons, or Coulomb attraction. In some cases the bond strength has relatively low magnitude to occurring by attractive forces of VdW type. The origin of the adhesion force is the same fundamental force of nature responsible for the

This phenomenon of interest is multidisciplinary, for example, the effect of adherence on civil engineering projects, cell adhesion in different microorganisms, adhesion of bacteria on the surface of dental enamel, adhesion of polymeric membranes in separation processes for

When contact occurs between two solid bodies, adhesion is not observed. This is due to the fact that much of the surface has a roughness at the microscale. This roughness decreases the area of active interaction between two solid bodies, as only regions with peaks come into

Liu, D. -L. et al (2007) conducted a study concerning the effect of RMS roughness on the adhesion using AFM. This study provided a better understanding of the effect of roughness on the adhesion when working in the nanoscale. On this scale the effects of adhesion are

The total adhesion force in this case, the contribution of all molecules involved in the

R h 2πωR

q c q cq

RR h R

Where: R = tip radius; Rq= RMS of roughness; hc = distance separating the tip/sample, and 2πωR represents the strength of the AFM system. The total force is normalized by the

The adhesion force falls with increasing surface roughness and also with increasing radius

**9. Complementary analysis: Fractal dimension and power spectral density**  Two powerful techniques for further analysis in the study of surfaces are the fractal dimension and Power Spectral Density. These analysis are based on surface roughness. This

2

(13)

solutes, adhesion of nanoparticles, among others (Bowen et al., 1998).

significant in applications of microelectromechanical systems.

process can be described by the equation (Bowen et al., 1998).

*F*

surface energy so that ω is the work of adhesion force.

membrane and greater efficiency in the separation process (Bowen et al., 1998).

Fig. 20. Relationship between surface roughness and brightness of the teeth treated with five different materials (Heintze & Russon, 2006)*.*

Fig. 20. Relationship between surface roughness and brightness of the teeth treated with five

different materials (Heintze & Russon, 2006)*.*

The surface roughness of the polymer membrane is a factor proportional to the bond strength of the membrane. The higher roughness leads to greater adhesive strength of the membrane and greater efficiency in the separation process (Bowen et al., 1998).