4.1.1. Garlic

It was observed that garlic reduces total cholesterol by almost 5% and LDL-C by 6% in subjects with elevated total cholesterol levels (mean TC > 200 mg/dL), and has a statistically non-significant effect on HDL-C and triglycerides. This observed reduction in LDL and total cholesterol is clinically relevant as it is associated with a reduction in the risk of adverse coronary and vascular events [69,70].

The results also suggested that garlic was highly tolerable and does not pose any side effects, which are more common with conventional therapies. In a majority of the studies, bad smell or odor was the only major side effect of garlic. A small proportion of the population did experience some gastrointestinal issues such as belching and acid reflux, but in comparison to statins, these side effects are not serious. Statins, on the other hand, do pose some serious adverse effects including a high risk of diabetes, cognitive and muscular impairment, sexual dysfunction, mood swings, anxiety, and irritability [73].

These findings corroborate the results of previous meta-analyses and systematic reviews [74–76]. In previous literature, the effect of garlic on cholesterol levels has been a debatable topic. Time and again, many individual trials of garlic have reported diverse therapeutic effect size of garlic on cholesterol levels. The study by Stevinson et al. [76] was one of the earliest meta-analysis of garlic on hypercholesterolemia and included 13 trials, whereas the recent and most updated study, by Ried et al. [74], included 39 studies. The studies by Stevinson et al. and Ried et al. both suggest that garlic reduces TC levels by approximately 15 mg/dL, whereas the study by Reinhart et al. [75] suggests its effect size is nearly half of what was observed by previous studies, i.e., −7.34 mg/dL. They attribute this smaller effect size to including newer studies, which exhibit more modest effects than older studies. However, the recent and most updated meta-analysis on garlic by Ried et al. [74] seems to be the most comprehensive one.

Meanwhile, observed results from this study have suggested that garlic has a modest effect size of −12.45 mg/dL. One reason behind this finding may have to do with the type of included studies in this review. Because Ayurveda does not entertain the use of aged garlic, extracted garlic oil, or extracted compound of allicin from garlic, and uses whole garlic as a preparation, this work did not include any of the studies that used those various forms of garlic as an intervention. There is a common trend of using aged garlic and a lot of studies had to be excluded for using aged garlic as an intervention. However, the studies that used whole garlic or dried garlic powder were included, as it is the common practice to use whole garlic cloves in Ayurvedic pharmaceutical science [71]. From a pharmacological point of view, it is believed that the active compound called allicin, a garlic derivative, is responsible for reducing cholesterol levels and for the distinctive smell of garlic [77]. Allicin is a volatile compound and is responsible transiently for cardiovascular effects [77]. However, the exact ingredients and their mechanism of action still remains unknown. On the other hand, Ayurveda has been using garlic for treating Hridroga, an Ayurvedic term for cardiovascular diseases [71]. In Ayurveda, garlic, which possesses all five tastes except sour taste, is capable for clearing channels and all the coverings (avarana) because of its pungent and piercing qualities [71,78].

In conclusion, the findings of this review suggest that garlic is superior to placebos in reducing elevated total cholesterol and LDL-C levels. Garlic has also been shown to have additional cardiovascular benefits such as reducing high blood pressure [79]. When taken together, garlic preparations can be considered as a general heart tonic with cholesterol regulating properties. It can also be used as a preventive agent in borderline cholesterol levels, with a higher safety and tolerability profile than statins.

#### 4.1.2. Nigella Sativa

Another intervention that showed some efficacy on cholesterol levels was *Nigella sativa*. Commonly known as black cumin and Upakunchika in Ayurveda, it is more famous for its digestive effect. Ideally, *Nigella sativa* seed powder is used in Ayurveda, so studies of its seed oil or other extracts were not included. Based on the analysis of three studies on *Nigella sativa*, it was observed that it reduces total cholesterol by 9.28 mg/dL (95% C.I.: 17.36 to 1.19, *p* = 0.02). However, it does not seem to have a statistically significant effect on other parameters such as LDL-C 2.12 mg/dL (95% C.I.: 7.85 to 3.6; *p* = 0.47), triglycerides 21.09 mg/dL (95% C.I.: to −2.77; *p* = 0.08), and HDL-C 1.92 mg/dL (95% C.I.:1.62 to 5.45; *p* = 0.29). This observation is comparable to the results of a recently published systematic review by Sahebkar et al. [80]. The difference between the current study and [80] is the inclusion criteria; the present study only included studies with whole seed powder, whereas Sahebkar et al. [80] included other versions of *Nigella sativa* such as seed oil and also had no restriction on baseline total cholesterol levels for the inclusion criteria, which limited the number of studies in this meta-analysis.

As an addition to the herbs mentioned above, we also conducted a systematic review to examine the effects of *Terminalia arjuna* on lipid parameters of hypercholesterolemic patients. Due to a lack of strict randomized controlled designs and inconsistencies in Arjuna preparations being used, the meta-analytic results are not included in our main results section. Nonetheless, the findings from such quasi-randomized studies are still worth mentioning. After analyzing data from 14 arms of 10 different studies enrolling 547 participants, it was observed that Ayurvedic herbal preparations with *Terminalia arjuna* as a main ingredient reduces total cholesterol by 19.47 mg/dL (95% C.I.:30.73, 8.20, *p* = 0.0007), LDL-C by 16.33 mg/dL (95% C.I.:23.21, 9.45, *p* < 0.00001), triglycerides by 11.24 mg/dL (95% C.I.:22.02, 0.46, *p* = 0.04), and raises HDL-C by 5.16 mg/dL (95% C.I.:2.62, 7.69, *p* < 0.00001). Overall, Arjuna may also be considered as a viable herb that can be quite beneficial for patients with hypercholesterolemia.

#### 4.1.3. Adverse Effects

No serious adverse events were reported in the majority of the studies, except Szapary (2003) [21] reported that one patient each from experimental group and control group had serious side effects and Joseph (2012) reported that one participant in the active control group withdrew due to a five-fold increase in serum creatinine level. Other minor adverse events such as gastrointestinal upset, skin rashes, nausea, and bad odor of breath and body were also reported in few other studies [47,56].

Limitations: Although every effort was made to discover all eligible studies published through December 2020, there is a possibility of some studies still being left behind. Most of the studies that were seemingly eligible did not qualify given our inclusion criteria, but some of the studies did show consistent findings with our study [55–57]. A Pakistani study by Zeb et al. [81] examined the impact of garlic powder, coriander powder, and a mixture of the two on lipid profile as compared to a placebo. Garlic powder was found to be the most effective in reducing TC and LDL and increasing HDL, as compared to all other groups. The latest study by Iskander et al. [82] showed that a nutraceutical combination including g gugguluipid showed a significant reduction in TC and LDL levels after 8 weeks of consumption in a randomized, placebo-controlled double-blind trial. Another study by Kuchewar et al. [83] found beneficial effects of Triphala on the lipid profile of patients with dyslipidemia. Many other herbs such as Amla [84,85] have also shown promising

results for controlling lipid parameters among patients with hyperlipidemia/dyslipidemia. However, these studies failed to meet the inclusion criteria of the current review.


The high operational cost of aerial images makes it difficult to acquire periodic observation of the region of interest. Satellite imagery is an alternative for the problem, but lowresolution is an obstacle. In this study, we proposed a context-based approach to simulate the 10 m resolution of Sentinel-2 imagery to produce 2.5 and 5.0 m prediction images using the aerial orthoimage acquired over the same period. The proposed model was compared with an enhanced deep super-resolution network (EDSR), which has excellent performance among the existing superresolution (SR) deep learning algorithms, using the peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and root-mean-squared error (RMSE). Our context-based ResU-Net outperformed the EDSR in all three metrics. The inclusion of the 60 m resolution of Sentinel-2 imagery performs better through fine-tuning. When 60 m images were included, RMSE decreased, and PSNR and SSIM increased. The result also validated that the denser the neural network, the higher the quality. Moreover, the accuracy is much higher when both denser feature dimensions and the 60 m images were used.

**Keywords:** aerial orthoimage; Sentinel-2; super-resolution; image simulation; residual U-Net

#### **1. Introduction**

Aerial imagery has been widely used for monitoring the surrounding environment due to its long history. Orthoimages created from aerial images can provide high-quality geospatial information taken at lower altitudes than satellite images. Continuously monitoring a rapidly changing environment requires reducing the observation period for a site. However, the tradeoff between spatial resolution and ground coverage prevents aerial images from covering a wide area. The role of aerial imagery has been gradually replaced by satellite imagery with its wide area coverage and regular repeat pass capabilities. Moreover, satellites equipped with multispectral sensors have enabled multiple applications such as resource management, urban research, facility mapping, and disaster monitoring.

The resolution of most of the current satellite images is still lower than that of aerial images. The price of commercially available high-resolution (HR) satellites has frequently hindered many researchers' progress in their projects. In most countries, including Korea, HR aerial orthoimages are provided to the public for free [1]. Furthermore, in the United States and the European Union, low- and medium-resolution satellite images are provided free of charge to users around the world. Research is needed to increase the resolution of mid- and low-resolution satellite images using freely available HR aerial images.

In the field of remote sensing, a visible improvement of image resolution primarily implies pan-sharpening. This method improves the resolution of low-resolution multispectral images using an HR panchromatic image. There are two typical approaches, one using

**Citation:** Yoo, S.; Lee, J.; Bae, J.; Jang, H.; Sohn, H.-G. Automatic Generation of Aerial Orthoimages Using Sentinel-2 Satellite Imagery with a Context-Based Deep Learning Approach. *Appl. Sci.* **2021**, *11*, 1089. https://doi.org/10.3390/app11031089

Academic Editor: Yang Dam Eo Received: 1 January 2021 Accepted: 22 January 2021 Published: 25 January 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Intensity-Hue-Saturation (IHS) information [2] and one using principal component analysis (PCA) [3]. The primary concern for pan-sharpening is that it is applicable only when an HR panchromatic image is available. Consequently, the resolution of the pan-sharpened image cannot be higher than that of the input panchromatic image. With the recent development of deep learning techniques, studies to produce images with higher resolution than the input image have been conducted. Several studies using deep learning techniques have been published in the remote sensing community. Related studies can be largely divided into two usage categories: multiple sensors from one platform and multiple sensors from multiple platforms [4–10].

Improving the resolution of multispectral sensors from one (same) platform is usually performed by merging lower and higher multispectral images. Gargiulo et al. [5] enhanced a 20 m shortwave infrared (SWIR) image acquired by Sentinel-2 into a 10 m SWIR image. Similar to the pan-sharpening approach, the four-channel 10 m visible and NIR resolution images of Sentinel-2 were regarded as panchromatic. A shallow convolutional neural network (CNN) was constructed to improve the resolution of the SWIR image. The limitation of this study is that only the resolution of an SWIR image can be improved. Lanaras et al. [6] presented research results that can address this limitation. By constructing deep and dense neural network models, DSen2 and VDSen2, they improved the 20 m resolution of three red-edge and three SWIR images, two 60 m resolution images of water vapor, and 60 m SWIR mages of Sentinel-2 images into 10 m. They asserted that the model could be extended and improved from 20 m and 60 m to a 10 m resolution. However, the first category cannot produce images with higher resolution than the maximum resolution provided by the platform.

Another category is improving the resolution of multispectral sensors from multiple (different) platforms. Few studies have improved 30 m Landsat-8 satellite images to 10 m using Sentinel-2 images. Shao et al. [7] proposed the extended super-resolution convolutional neural network (ESRCNN) by blending Landsat-8 and Sentinel-2 data. They demonstrated the effectiveness of the deep learning-based fusion method for improving the resolution of Landsat-8 imagery. In their study, a performance comparison was performed using area-to-point regression kriging rather than other deep learning-based algorithms. Pouliot et al. [9] tested shallow and deep CNNs and confirmed that the deep CNN performed the same or better than the shallow CNN. The suggested algorithm demonstrated high-performance, but computational complexity and memory requirements could be problematic because the model is trained for each band.

After analyzing the previous studies, we found three common points. The first is that the use of deep neural networks is superior [6,8,9]. Tai et al. [8] analyzed the performance of each neural network by constructing shallow, deep, and very deep networks. They confirmed that the deeper the neural network, the higher the performance. Second, most neural networks have residual blocks and skip connections [6,8,10,11]. Consequently, the vanishing gradient problem can be alleviated, and the learning speed improved, even though the neural network is deeper. Third, the size of the input image inside the neural network is maintained until the last stage of the output, in contrast to neural networks for object detection and segmentation. Accordingly, the enlargement function to create the HR is only located in the final stage of neural networks using upsampling convolution layers or pixel shuffle algorithms [11]. Galar et al. [10] applied an enhanced deep super-resolution network (EDSR) to produce a 5 m resolution RapidEye RGB image with a 10 m resolution Sentinel-2 RGB image. They confirmed superior performance among super-resolution (SR) neural networks [11,12].

Studies so far have used neural networks of increasing resolution between satellite images. In this study, we propose a context-based ResU-Net to increase the resolution of Sentinel-2 imagery using 2.5 and 5.0 m downsampled aerial orthoimage acquired during the same period. For completing the tasks, the aerial orthoimages were first simulated by reconstructing a residual U-Net, which has advantages not only in constructing a deep and dense neural network but also in identifying adjacent contexts and the position of objects. As a result of the experiments, we found that our neural network can express the aerial orthoimages' features and contexts well.

Training datasets were newly generated by using Sentinel-2 and aerial orthoimage. Sentinel-2 images, providing 10 m, 20 m, and 60 m resolution of multispectral bands, were utilized in this research. Sentinel-2 has the highest resolution and shortest revisit date among free satellite images. Since the advantage of obtaining many repeat pass images is a factor that can satisfy the objectives of this study, it was selected as input data. SR research is key to securing a high-resolution ground truth (GT), and aerial orthoimages are one of the most reliable and high-quality data. Therefore, aerial orthoimages with a similar acquisition date were utilized as the GT. Two types of aerial orthoimages were produced, 2.5 m and 5.0 m, as ground truth data downsampled from the original aerial orthoimagery. The data were used for testing two-times magnification (5.0 m based on 10 m) and challenged four-time magnification (2.5 m based on 10 m).

We tested the effect of using the lowest resolution 60 m image on the model and analyzed the model's influence when the feature dimensions are changed. In addition, the quality of our approach was investigated through the peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and root-mean-squared error (RMSE), a common approach in many SR studies.

Finally, we found that our model's performance in most metrics turned out to be better than that of EDSR. We also identified that incorporating 60 m resolution with 10 m resolution Sentinel-2 images outperforms the combination 10 m and 20 m resolution images. In addition, we confirmed that the denser feature dimensions have better performance. In particular, it could be a useful reference for related research as it predicts well even narrow roads that are difficult to identify with low-resolution satellite images.

#### **2. Materials and Methods**

#### *2.1. Training Datasets Generation and Site Selection*

#### 2.1.1. Study Area

Daejeon City, located in the central part of the Korean peninsula, was selected as the study area. The city has an area of approximately 539 km2 and is a transportation hub connecting the southern and northern regions. As depicted in Figure 1, most of the areas illustrate urban landscapes, where large and small buildings are clustered. Rice paddies/fields and mountainous areas are distributed in minimal areas. Middle areas, primarily covered with many complex environments such as urban buildings and roads, are the areas where the SR approach is challenging to apply.

**Figure 1.** Location of Daejeon and Sejong depicted with Google map and Sentinel-2 images (band 2 image acquired on 2018/04/18).

Sejong City, Korea's administrative capital, is being developed into a city since 2012. The area of Sejong City is approximately 465 km2, and most of the regions are still mountains and rice fields. However, due to construction, the impermeable layer is increasing rapidly every year. Daejeon City was selected to produce training datasets, and Sejong City was selected as a test site to analyze the generalization capabilities. Even if training samples and test samples are not overlapped, spatial autocorrelation within the same area cannot be avoided. Therefore, it was necessary to select an independent region with different characteristics.

#### 2.1.2. Aerial Orthoimages

Aerial orthoimages were acquired in 2018, distributed free of charge under the leadership of the Korean government's aerial image acquisition and map production policy. Due to national security reasons, only 51 cm resolution images are provided to the public [1], and internally up to a 25 cm resolution is produced and used. We meticulously inspected the acquisition date of aerial images through the government orthoimage production manual and identified that aerial images were acquired over approximately one month, 21 April, 29 April, 5 May, and 26 May 2018, to cover the entire study area. The 51 cm orthoimages using the aerial triangulation method were provided through the government website. The final orthoimages downloaded are depicted in Figure 2.

**Figure 2.** Aerial orthoimages over the study area with the acquisition date.

2.1.3. Sentinel-2A/B Satellite Imagery

Sentinel-2 is one of the satellites operated by the European Space Agency (ESA) and provides 13 multispectral bands with several different resolutions (10 m, 20 m, and 60 m). The imagery of Sentinel-2 has the highest resolution of 10 m among the current freely available for the general public. Accordingly, Sentinel-2 was selected for the study because it can provide much richer information than any other free satellite images. A short revisit period of five days is another strength of the Sentinel-2 imagery. Initially, the revisit period was ten days, but two satellites named Sentinel-2A and Sentinel-2B take images alternately, which reduces the revisit period to 5 days.

Sentinel-2 provides two types of images: (1) the L1C product, a top of atmosphere (TOA) reflectance image and (2) the L2A product, a bottom of atmosphere (BOA) reflectance image. The L2A product can overcome a significant difference in reflectivity, which varies for different acquisition times. Because aerial images are acquired at a much lower altitude than satellite images, it is better to use images with atmospheric correction. Because ESA provides the L1C product for images over the study area from 2018, all experimental images were converted to L2A through the Sen2cor tool of the Sentinel application platform (SNAP) software [9,13]. Twelve images (four 10 m, six 20 m, and two 60 m) with different spectral bands ranging from visible wavelength to SWIR were acquired. In some land classification studies, 60 m resolution images are not used because they are primarily for atmospheric

correction [14,15]. However, we tested our approach with and without 60 m resolution imagery to consider whether additional atmospheric information is useful for training input images.

For matching the Sentinel-2 images acquired at the same time interval as the aerial orthoimages, data were searched through the Copernicus website, where the Sentinel series took all provided images [16]. We obtained both Sentinel-2A and 2B sensor images, which contain less cloud coverage, from the website. Searched images used in this research are listed in Table 1, and only band 2 images are depicted in Figure 3. All 10 m and 20 m images were used in training as defaults, with 60 m as optional. Because the datasets are acquired simultaneously with the aerial orthoimages, it was assumed that there were no significant topographic changes during the short period. Accordingly, listed datasets are used for all the following experiments.

**Table 1.** Sentinel-2 sensing start times used for the research.

**Figure 3.** Acquired Sentinel-2 band number 2 (10 m) images: (**a**) Sentinel-2A 2018/04/18, (**b**) Setntinel-2A 2018/04/28, (**c**) Setntinel-2A 2018/05/28, (**d**) Setntinel-2B 2018/05/23.

#### 2.1.4. Training Datasets Generation

The training datasets were preprocessed based on 2.5 m downsampled aerial orthoimages and 10 m Sentinel-2 satellite images. The first step was to transform both image sets into the same map projection system. All image sets in this study were projected into the Korea 2000 coordinate system (EPSG: 5186), corresponding to transverse mercator (TM) projection. The second step was to determine the size of training datasets based on 60 m Sentinel-2 images. After considering the computational efficiency of training processes, the 4 × 4 pixels image size was used, corresponding to 240 × 240 m2 on the ground. For this configuration, the image size for 2.5 m and 5.0 m aerial orthoimages were 96 × 96 pixels and 48 × 48 pixels, respectively. For the same reason, the training image sizes of 10 m and 20 m resolution for Sentinel-2 were 24 × 24 pixels and 12 × 12 pixels, respectively.

Training samples and test samples were selected randomly within the study area but did not overlap for the Daejeon area. Through this process, 32,632 training samples (6527 for validation samples, 20% of the training samples) and 8156 test samples were produced. In addition, 39,204 test samples were generated for the Sejong area. Each set consisted of twelve Sentinel-2 images (4 for 10 m, 6 for 20 m, and 2 for 60 m) and two aerial photographs (1 for 2.5 m and 1 for 5.0 m), as depicted in Figure 4. A 5.0 m aerial orthoimage was used as the GT for 2× magnification of 10 m Sentinel-2 images and 2.5 m for 4× magnification of 10 m Sentinel-2 images.

**Figure 4.** Example of training sets (Input: {B2 B3 B4 B8}: 10 m, {B5 B6 B7 B8A B11 B12}: 20 m, and {B1 B9}: 60 m, output: aerial orthoimages 2× for 5.0 m, 4× for 2.5 m).

#### *2.2. Methodology*

#### 2.2.1. Context-Based ResU-Net

The latest research results indicate that the quality of SR increases as more convolution layers or deeper neural networks are assigned [6,8,9]. Most recent deep learning-based SR neural networks adopt this trend by maintaining the size of the input image until the output stage. The enlargement function to create HR is applied to the final stage [8,11,12]. The existing methodology was applied to our datasets with unsatisfactory results. It is speculated that different imaging geometry between aerial and space-borne sensors may lead to unsatisfactory results even with similar research methods. Because the aerial orthoimage contains more context information than the space-borne Sentinel-2 image, we determined that it would be critical to arrange context-preserving and deep and dense neural networks in the initial stage. The proposed architecture of the context-based ResU-Net for our study is depicted in Figure 5.

In our study, the residual U-Net proposed by Zhang et al. [17] was modified to maintain the context information and build deep neural networks. Batch normalization (BN) and ReLU activation functions are included in most of the steps. BN helps to solve gradient vanishing/exploding and overfitting caused by the deep neural network; it also improves accuracy [6,11]. The ReLU is used to remove the values below zero [6]. The encoder's role is to make the input image compact, and the decoder recovers the information to generate the final image. There is a path connecting the encoder and the decoder, and all convolution layers have a filter size of 3 × 3. The encoding path has three

conv-depth blocks. Each block's stride was set to 2 instead of using downsampling layers to reduce the feature map's size in half. The decoding path has three conv-depth blocks to correspond to the encoder, and the size is increased through upsampling layers. End of the decoding path, a convolution layer is inserted to make feature dimensions as 3 with ReLU activation function for generating desired resolution similar to that of aerial orthoimage.

**Figure 5.** Context-based residual U-net for aerial images simulation (*S*: scale, *f*: feature dimension)**.**

There are three major differences between the existing Residual U-Net and our network. First, the conv-depth block was included to reduce computation resources. It is known that depth-wise separable convolution (DepthConv) maintains performance while reducing the number of parameters [18]. As shown in Table 2, if a convolution layer is used instead of a DepthConv layer in our architecture, the number of parameters to be learned becomes larger. In addition, the difference in the number of parameters increased as the size of the feature dimensions increased. Moreover, we had encountered that the validation loss was jagged when only the convolution layer was used. On the contrary, the loss converges evenly with a lower value when using the DepthConv layer, as shown in Figure 6. When only the convolution layer was used, the loss at epoch 1 was 45,328.06, but the value was too large to be displayed on the graph, so only the corresponding value was clipped.

**Figure 6.** Loss convergence comparison: conv-depth block vs. conv only for context-based ResU-Net when using 60 m images with *fc* feature dimensions (2×).

Second, upscaling was applied in the initial stage of the neural networks. The reason for changing the order like this is that the final prediction image becomes smoother or darker than that of the GT image when the image size is enlarged in the final stage as most of the other SR networks are arranged. In our networks, the scale (*S*) indicates an increasing factor of the original image. For example, the scale was set to 2*S* at the beginning and halved at the end, achieving a double improvement effect. Finally, the stride parameter was set to 2 to halve the image resolution.

Third, the feature dimension (*f* = {*f1, f2, f3, f4, f5*}) was configured to increase gradually as the image size decreases, and the experiment was conducted in three groups: *fa* = {16, 32, 64, 128, 256}, *fb* = {32, 64, 128, 256, 512}, and *fc* = {64, 128, 256, 512, 1024}. It was designed to analyze the predictive ability according to the size of feature dimensions.

#### 2.2.2. Hyperparameter Optimization

The following hyperparameters were chosen to control the learning process. The related parameters were the optimizer, loss function, learning rate, batch size, and epoch. For an optimizer Adam optimizer for gradient descent was used in this study, reflecting many previous studies that this optimizer produced the best performance and had lower memory requirements than others [6,9,10,19]. The L1 loss function was used to minimize the error, which is the sum of all the absolute differences between the true value and the predicted value; it has been widely applied to SR neural networks [11,12,19]. For our study, we adopted the mean squared error (MSE) loss function instead of L1 because MSE had better results than L1. Finally, the mini-batch size was set to 32.

The learning rate gradually decreased as the epoch increased through a rate decay scheduler [18]. Consequently, the learning rate functions as an essential hyperparameter because it is dependent on the epoch parameter. Therefore, the epoch was adjusted between 30 and 180 to find the minimum loss; the experiment was repeated for each model. The initial learning rate was set to 5 × 10-4. Early stopping criteria using validation datasets were also applied to avoid overfitting, and learning was stopped if the accuracy was not improved within 10 epochs. All programming was performed with Python-based Tensorflow nightly (2.5.0) GPU version, and learning was conducted using three graphics cards: two GeForce RTX-2080 Ti (11 GB VDRAM) and one RTX-3090 (24 GB VDRAM).

#### **3. Results**

Two experiments were conducted to evaluate our results: (1) whether to use 60 m images and (2) the effects of the feature dimension sizes. The EDSR neural network was also trained for comparison with our results. EDSR was selected as a comparison due to its excellent performance among the currently developed SR neural networks [12]. Lim et al. [11] designed both baseline and EDSR models. The difference between the two models is the number of residual blocks and the feature dimension. The baseline model is organized with 16 residual blocks and 64 feature dimensions, and EDSR is formed with 32 residual blocks and 256 feature dimensions. Both models are utilized for comparison, and all related training parameters were set as the author suggested. After training both neural networks with the same datasets, the results were evaluated with three metrics. PSNR and SSIM were used to evaluate the outcome—they are most frequently used as an evaluation index of SR deep learning research [11,12,20]. The RMSE used in some studies [6,9] was also included. The comparison of the final three metrics is summarized in Table 3 for Daejeon City and Table 4 for Sejong City, respectively. The scale parameters 2 and 4 refer to generating 5.0 m and 2.5 m aerial orthoimages, respectively.

In the case of Daejeon City, our context-based ResU-Net outperformed the baseline and EDSR models for all three metrics. For EDSR, even if the residual blocks and feature dimensions increased comparing with the baseline model, it is difficult to find performance improvement. In the case of Sejong City, in which independent testing was performed, our models performed better in two metrics except for RMSE.

**Table 3.** Quality evaluation results with set parameters for Daejeon City. Blue indicates the best results, and red indicates the second best.

**Table 4.** Quality evaluation results for Sejong City. Blue indicates the best results, and red indicates the second best.

The image quality deteriorated as the magnification was enlarged, and the value of the evaluation metrics gradually deteriorated. Through fine-tuning, the inclusion of 60 m images performs better in two networks. When 60 m images were included, RMSE decreased, and PSNR and SSIM increased. This result demonstrates that the 60 m images have a positive impact on both networks.

The loss converged to a lower value if feature dimensions increased, as depicted in Figure 7. The result also validated that the denser the neural network, the higher the quality. Moreover, we found the accuracy is much higher when both denser feature dimensions and the 60 m images were used.

**Figure 7.** Convergence results according to each feature dimension.

For a visual comparison between EDSR and context-based ResU-Net, the prediction images are listed in Tables 5–9. Each table shows the predicted image of one representative input Sentinel-2 image per resolution (10 m, 20 m, and 60 m) for two scales, 2 and 4. The 2.5 m and 5.0 m aerial orthoimages are GT. The use of the 60 m Sentinel-2 images is shown in the second column. The predicted images of the baseline and EDSR model are shown in the third column. There is not much difference between the baseline and EDSR model, and only EDSR will be compared in the following. The predicted images of our context-based ResU-Net for three feature dimensions (*fa*, *fb*, *fc*) are shown in the fourth column of each table. Generally, the prediction images between EDSR and context-based ResU-Net are visually similar when the feature dimension of context-based ResU-Net is *fa*. For EDSR, even if the residual blocks and feature dimensions increase, no further improvement can be found. However, in our model, as networks become denser from *fa* to *fc*, it can be seen that the prediction images are getting close to GT.

**Table 5.** Predicted images of Sentinel-2 and corresponding GT image (paddy/road area).

**Table 5.** *Cont.*

**Table 6.** Predicted images of Sentinel-2 and corresponding GT image (urban area).

**Table 6.** *Cont.*

**Table 7.** Predicted images of Sentinel-2 and corresponding GT image (forest area).

**Table 7.** *Cont.*

**Table 8.** Predicted images of Sentinel-2 and corresponding GT image (urban/forest area).

**Table 8.** *Cont.*

**Table 9.** Predicted images of Sentinel-2 and corresponding GT image (urban/road area).

**Table 9.** *Cont.*

Observing the boundaries of an object reveals the difference between the two methods. For EDSR, when the image was enlarged four times, the overall boundary of each object remained similar or smoother than that of two-time enlargement-causing the prediction images to look blurry. For context-based ResU-Net, boundaries of each object became more distinct as the feature dimensions increased in density regardless of the enlargement scale. When the feature dimension reached its maximum size, the boundaries of the object became sharpest. Consequently, the visibility of all images improved.

Some differences were found between the two models. EDSR predicts a darker image, especially in forest areas (Table 7), and it produced a blurry image compared to ours, as shown in Tables 5–9. Interestingly, the result of context-based ResU-Net predicts even urban shadows well in the densest feature dimension *fc*, which are not even expressed in EDSR as shown in Table 6. It was also identified that our model generally trained the boundaries of objects better. In particular, road boundaries are well preserved even the width is narrower than the 10 m resolution Sentinel-2 image. The result implies that the recognition of the object of concern, such as the road, can be possible by using predicted Sentinel-2 imagery, as shown in Tables 5 and 7. The road boundaries become clear as feature dimension becomes denser, but it seems that some attention needs to be paid to the shape of the road for the inclusion of 60 m Sentinel-2 imagery. Some of the road boundaries were visually curved when the 60 m Sentinel-2 images were included, but they were straight when the 60 m image was not included. It can be said that there exists a tradeoff between the value of metrics and visualization when the road boundaries are concerned.

#### **4. Discussion**

A study was conducted to produce 2.5 m and 5.0 m resolution imagery with 10 m Sentinel-2 satellite images using aerial orthoimage as a ground truth. For this, training samples were produced by acquiring Sentinel-2 satellite images and aerial orthoimages over the same area and period. The training samples were used to simulate 2.5 m and 5.0 m aerial orthoimages. For quality check and general applicability of our neural network, additional test samples in an independent region were utilized. For producing bettersimulated images, a new context-based neural network was proposed and compared with the existing neural network. Our context-based ResU-Net generally outperformed the baseline and EDSR for all three metrics, both in training samples and test samples. We believe that this is because conv-depth blocks helped the stability of our model. In any case, the utility of our model for successfully predicting narrow roads will be very high. Meanwhile, in order to improve the performance compared to the present, the obstacles to be solved were speculated as follows:

First, the effect of shadows in HR aerial images was significant. The Sentinel-2 images were acquired with a low-resolution at high altitude, whereas aerial images were acquired with HR at low altitude. Even in the same area, when images were acquired at a low altitude, the effect of shadows was much more prominent than at a high altitude. Because most of our study area included urban landscapes, the effect of shadows on HR images was much greater than for high altitude images. The original 51 cm aerial orthoimage was resampled to obtain GT using bilinear interpolation. During the bilinear interpolation process to create GT from the original 51 cm aerial orthoimage, the effect shadow smeared into other features and worsened the SSIM metric.

Second, there existed the effects of color correction during the composition of aerial orthoimages. The primary purpose of aerial orthoimages distributed by the Korean government is to produce a visually attractive map for the general public. We speculated that the original reflectance information had been corrected to make the orthoimage more pleasing, leading to potentially difficult and inaccurate training due to the use of aerial orthoimages.

For this study, it is essential that both aerial orthoimages and satellite images must be taken at a similar period of time. Recently, some countries have provided aerial orthoimages, so if researchers can check the acquisition date of aerial orthoimages, we expect that our research results can be utilized.

In future research, steps for shadow identification and shadow removal must be included based on deep learning, especially when using the HR aerial images as training sets. In the remote sensing community, CNN-based SR research is ongoing. However, several studies have tried to combine images obtained from multiple sensors to produce new images. We believe that the method and results presented in this study can contribute new insights for researchers performing similar studies.

**Author Contributions:** S.Y. and H.-G.S. were the leading directors of this research. S.Y. designed the overall research plan and programmed them for experiments, while J.L. programmed training data generation and augmentation. J.B. and H.J. performed the preprocessing of aerial and satellite images. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was supported by a grant (no. 20009742) from the Disaster-Safety Industry Promotion Program funded by the Ministry of Interior and Safety (MOIS, Korea).

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


**Table 1.**

*Demographics of study subjects.*

*Autism Spectrum Disorder - Profile, Heterogeneity, Neurobiology and Intervention*

conditions, with GI symptoms being the most common [3, 4].

diagnostic measures universally applicable for ASD children. This assumption is supported by the fact that ASD is characterized by multiple co-morbid medical

Many co-morbid conditions reported in ASD subjects are associated with immune mediated inflammation in pathogenesis; GI symptoms found in ASD subjects have been in part implicated with chronic GI inflammation due to dysregulated gut immune responses to microbiota [5, 6]. Mounting evidence also indicates that many, but not all the ASD subjects show some immune abnormalities that affect almost every arm of the immune system [7, 8]. Moreover, given the fact that the immune system and the CNS interact closely interact [9], a role of neuroinflamma-

tion in ASD pathogenesis is highly suspected in a subset of ASD subjects.

In our previous studies, we focused on abnormalities of innate immunity, which plays a major role in the neuro-immune network including stress responses [10–12]. One of the reasons that we focused on innate immune abnormalities is based on findings from one of the most thoroughly studied animal models of ASD, maternal immune activation (MIA). MIA is induced by sterile stimulants of innate immunity in pregnant rodents [13]. MIA generates lasting effects on behavioral symptoms and the immune functions in offspring [13, 14]. It has been puzzling that how innate immunity, which lacks memory for specific antigens, can cause such lasting effects. However, discovery of innate immune memory (IIM) caused by initial stimuli through epigenetic regulations [15, 16], has helped us understand the lasting effects of dysregulated IIM in various inflammatory conditions. In fact, dysregulated IIM is now implicated in the pathogenesis of common neuropsychiatric diseases such as schizophrenia and depression [17, 18]. We found abnormalities of innate immunity in many, but not all the ASD subjects we studied by assessing cytokine profiles from

Given both the considerable amount of abnormalities found in the monocyte cytokine profiles and the high frequency of co-morbid medical conditions in ASD subjects, questions understandably arise concerning the association of co-morbid conditions with changes in monocyte cytokine production. Many ASD subjects are treated with neurotropic medications including selective serotonin re-uptake inhibitors (SSRIs), anti-seizure medications used as mood stabilizers, neuroleptics, and medications for ADHD. These medications may also affect monocyte cytokine profiles. Therefore, this study addressed whether monocyte cytokine profiles differ depending on co-morbid conditions, ASD severity, and other clinical co-variables. The results indicate that co-morbid medical conditions are associated with changes in production of specific cytokines and such associated are not affected by other

**Study subjects**: Study subjects were recruited following the study protocols (#17:53 and #19:53) approved by the institutional review. Signed consent forms

*ASD* subjects: ASD subjects (N = 109) were recruited from the Pediatric Allergy/Immunology Clinic at SPUH. Diagnosis of ASD was made at various autism diagnostic centers, including ours, based on the Autism Diagnostic Observation Scale (ADOS) and/or Autism Diagnostic Interview-Revisited (ADI-R), as well as other standard measures. ASD subjects were also evaluated for their behavioral symptoms and sleep habits with the Aberrant Behavior Checklist (ABC) [19] and the Children's Sleep Habits Questionnaires (CSHQ ) [20], respectively. Information regarding cognitive ability and adaptive skills were obtained from previous school

**84**

purified monocytes [11].

clinical co-variables.

**2. Materials and methods**

were obtained prior to entering the study.

or study subjects preferred, we applied a topical lidocaine/prilocaine cream (Emla cream®) to the site of venipuncture prior to blood sampling.

**Cell cultures:** Ficoll–Hypaque density gradient centrifugation was used for separating PBMCs. From PBMCs, PBMo were further purified using a column of magnetic beads labeled with anti-CD3, CD7, CD16, CD19, CD56, CD123, and glycophorin A (monocyte separation kit II – human, MILTENYI BIOTEC, Cambridge, MA, United States). Combination of these antibodies depletes T, B, natural killer, and dendritic cells from PBMCs.

Cytokine production by purified PBMo was induced by incubating cells overnight (2.5 × 105 cells/ml) with a panel of agonists of TLRs. This assay system was designed to reflect the effects of microbial byproducts commonly encountered in real life. Lipopolysaccharide (LPS), a TLR4 agonist, represents a signaling pathway activated in response to a gram negative [G (−)] bacteria. Zymosan, a TLR2/6 agonist, mimics an innate activation signal in response to G (+) bacteria and fungi. CL097, a TLR7/8 agonist, activates innate signaling pathways in response to ssRNA viruses that cause common respiratory infection. Candida heat extract as a source of ß-glucan, a dectin-1 agonist, was used as well as a C-lectin receptor agonist. PBMos were incubated overnight with LPS (0.1 μg/ml, GIBCO-BRL, Gaithersburg, MD, USA), zymosan (50 μg/ml, Sigma-Aldrich, St. Luis, Mo), C097 (water-soluble derivative of imidazoquinoline, 20 μM, InvivoGen, San Diego, CA, USA), and candida heat extract (HCKA, heat killed *Candida albicans* (107 cells/ml, InVivogen, San Diego, CA) in RPMI 1640 with additives as previously described [28]. Overnight incubation (16-20 h) was adequate to induce the optimal responses in this setting in previous studies [11]. Cytokine levels in the culture supernatant were then measured.

Levels of CCL2, IL-1β, IL-6, IL-10, IL-12p40, transforming growth factor-ß (TGF-ß), tumor TNF-α, and sTNFRII cytokines were measured by enzyme-linked immuno-sorbent assay (ELISA); 10–100 μl/well supernatants were used for ELISA. The OptEIA™ Reagent Sets (BD Biosciences, San Jose, CA, USA) were used for ELISA of IL-1ß, IL-6, IL-10, IL-12p40, and TNF-α. For CCL2, sTNFRII, and TGF-ß ELISA, reagents were obtained from BD Biosciences and R & D (Minneapolis, MN, USA). IL-23 ELISA kit was purchased from eBiosciences, San Diego, CA. Intra- and inter-variations of cytokine levels were less than 5%.

**Statistical analysis**: We used a two tailed Mann–Whitney test for comparison of two sets of numerical data. Kruskal-Wallis test was used for comparison of more than 2 sets of numerical data. When assessing differences in frequency between two groups, we used the Fisher exact test. For assessing differences in frequency among multiple groups, we used the Chi-square test and the Likelihood ratio. P value of less than 0.05 was considered nominally significant. Co-variance analysis was done with the use of analysis of variance (ANOVA) for a fixed factor or for a variable factor. NCSS2020 (NCSS, LLC. Kaysville, UT) was used for such statistical analysis.

#### **3. Results**

**Clinical characteristics:** Frequencies of co-morbid conditions among the recruited ASD subjects are summarized in **Table 2**. These results are consistent with the results of our previous studies [12, 29]. Age and gender were not associated with ASD severity (data now shown). Frequencies of co-morbid conditions and the use of neurotropic medications did not differ due to ASD severity in 108 ASD subjects who were verified ASD severity (**Table 3**).

**Changes in monocyte cytokine production depending on ASD severity**: We then examined whether monocyte cytokine profiles differed with ASD severity

**87**

(**Table 4**).

*Associations between Monocyte Cytokine Profiles and Co-Morbid Conditions in Autism…*

**Comorbid conditions ASD subjects (N = 109) Controls (N = 26)**

 symptoms 71/109 (65.1%) 0 history of NFA 68/109 (63.4%) 2/26 (7.7%) Seizure disorders 18/109 (16.5%) 0 Asthma 5/109 (4.6%) 0 Allergic rhinitis 17/109 (15.6%) 0 Specific antibody deficiency 26/109 (23.9%) 0 PANS like symptoms 62/109 (56.9%) 0 Disturbed Sleep 48/109 (44.0%) 0

*Abbreviations used: ASD; autism spectrum disorder, GI; gastrointestinal, NFA; non-IgE mediated food allergy,*

 **severity Level 1 Level 2 Level 3 Chi-Square**

GI symptoms 14/272 20/32 36/49 p > 0.1 Seizure disorder 1/27 6/32 10/49 p = 0.081 Antibody deficiency 6/27 10/32 10/49 p > 0.1 PANS like symptoms 12/27 19/32 30/49 p > 0.1 Disturbed sleep 8/27 16/32 23/49 p > 0.1

SSRIs 6/27 8/32 4/49 p > 0.1 Anti-seizure medications 3/27 9/32 8/49 p > 0.1 ADHD medications 3/27 3/32 3/49 p > 0.1 Neuroleptics 4/27 1/32 9/49 p = 0.079

*Abbreviations used: ASD; autism spectrum disorder, GI; gastrointestinal, NFA; non-IgE mediated food allergy, ADHD: attention deficiency hyperactivity disorder, PANS; pediatric acute-onset neuropsychiatric syndrome, SSRI:*

*One ASD subject was excluded from this analysis due to lack of validation of ASD severity.*

*Frequencies of Co-morbid conditions and medication use did not differ due to ASD severity.*

and if such changes were affected by other clinical co-variables. ASD severity was shown to be associated with changes in production of TNF-α, IL-1ß, IL-10, and CCL2, and TNF-α/sTNFRII ratios (**Table 4**). However, production of inflammatory monocyte cytokines (TNF-α and IL-1ß) under several culture conditions is affected by presence of co-morbid conditions and the use of ADHD medications

**Changes in monocyte cytokine production depending on co-morbid conditions:** Since associations between ASD severity and monocyte cytokine profiles were often affected by other co-morbid conditions and medication use (**Table 4**), we also evaluated whether changes in monocyte cytokine profiles in ASD

*Co-morbid conditions with objective diagnostic measures:* We evaluated changes in monocyte cytokine profiles in association with co-morbid conditions which were

subjects were affected by the presence of co-morbid conditions.

*DOI: http://dx.doi.org/10.5772/intechopen.95548*

*PANS; pediatric acute-onset neuropsychiatric syndrome.*

*Frequency of comorbid conditions in the study subjects.*

GI1

*1*

**Table 2.**

**ASD1**

**Medications**

*selective serotonin reuptake inhibitor.*

*1*

*2*

**Table 3.**

**Co-morbid conditions**

*Associations between Monocyte Cytokine Profiles and Co-Morbid Conditions in Autism… DOI: http://dx.doi.org/10.5772/intechopen.95548*

*1 Abbreviations used: ASD; autism spectrum disorder, GI; gastrointestinal, NFA; non-IgE mediated food allergy, PANS; pediatric acute-onset neuropsychiatric syndrome.*

#### **Table 2.**

*Autism Spectrum Disorder - Profile, Heterogeneity, Neurobiology and Intervention*

cream®) to the site of venipuncture prior to blood sampling.

dida heat extract (HCKA, heat killed *Candida albicans* (107

inter-variations of cytokine levels were less than 5%.

who were verified ASD severity (**Table 3**).

and dendritic cells from PBMCs.

night (2.5 × 105

measured.

or study subjects preferred, we applied a topical lidocaine/prilocaine cream (Emla

**Cell cultures:** Ficoll–Hypaque density gradient centrifugation was used for separating PBMCs. From PBMCs, PBMo were further purified using a column of magnetic beads labeled with anti-CD3, CD7, CD16, CD19, CD56, CD123, and glycophorin A (monocyte separation kit II – human, MILTENYI BIOTEC, Cambridge, MA, United States). Combination of these antibodies depletes T, B, natural killer,

Cytokine production by purified PBMo was induced by incubating cells over-

designed to reflect the effects of microbial byproducts commonly encountered in real life. Lipopolysaccharide (LPS), a TLR4 agonist, represents a signaling pathway activated in response to a gram negative [G (−)] bacteria. Zymosan, a TLR2/6 agonist, mimics an innate activation signal in response to G (+) bacteria and fungi. CL097, a TLR7/8 agonist, activates innate signaling pathways in response to ssRNA viruses that cause common respiratory infection. Candida heat extract as a source of ß-glucan, a dectin-1 agonist, was used as well as a C-lectin receptor agonist. PBMos were incubated overnight with LPS (0.1 μg/ml, GIBCO-BRL, Gaithersburg, MD, USA), zymosan (50 μg/ml, Sigma-Aldrich, St. Luis, Mo), C097 (water-soluble derivative of imidazoquinoline, 20 μM, InvivoGen, San Diego, CA, USA), and can-

Diego, CA) in RPMI 1640 with additives as previously described [28]. Overnight incubation (16-20 h) was adequate to induce the optimal responses in this setting in previous studies [11]. Cytokine levels in the culture supernatant were then

Levels of CCL2, IL-1β, IL-6, IL-10, IL-12p40, transforming growth factor-ß (TGF-ß), tumor TNF-α, and sTNFRII cytokines were measured by enzyme-linked immuno-sorbent assay (ELISA); 10–100 μl/well supernatants were used for ELISA. The OptEIA™ Reagent Sets (BD Biosciences, San Jose, CA, USA) were used for ELISA of IL-1ß, IL-6, IL-10, IL-12p40, and TNF-α. For CCL2, sTNFRII, and TGF-ß ELISA, reagents were obtained from BD Biosciences and R & D (Minneapolis, MN, USA). IL-23 ELISA kit was purchased from eBiosciences, San Diego, CA. Intra- and

**Statistical analysis**: We used a two tailed Mann–Whitney test for comparison of two sets of numerical data. Kruskal-Wallis test was used for comparison of more than 2 sets of numerical data. When assessing differences in frequency between two groups, we used the Fisher exact test. For assessing differences in frequency among multiple groups, we used the Chi-square test and the Likelihood ratio. P value of less than 0.05 was considered nominally significant. Co-variance analysis was done with the use of analysis of variance (ANOVA) for a fixed factor or for a variable factor. NCSS2020 (NCSS, LLC. Kaysville, UT) was used for such statistical analysis.

**Clinical characteristics:** Frequencies of co-morbid conditions among the recruited ASD subjects are summarized in **Table 2**. These results are consistent with the results of our previous studies [12, 29]. Age and gender were not associated with ASD severity (data now shown). Frequencies of co-morbid conditions and the use of neurotropic medications did not differ due to ASD severity in 108 ASD subjects

**Changes in monocyte cytokine production depending on ASD severity**: We then examined whether monocyte cytokine profiles differed with ASD severity

cells/ml) with a panel of agonists of TLRs. This assay system was

cells/ml, InVivogen, San

**86**

**3. Results**

*Frequency of comorbid conditions in the study subjects.*

*1 Abbreviations used: ASD; autism spectrum disorder, GI; gastrointestinal, NFA; non-IgE mediated food allergy, ADHD: attention deficiency hyperactivity disorder, PANS; pediatric acute-onset neuropsychiatric syndrome, SSRI: selective serotonin reuptake inhibitor.*

*2 One ASD subject was excluded from this analysis due to lack of validation of ASD severity.*

#### **Table 3.**

*Frequencies of Co-morbid conditions and medication use did not differ due to ASD severity.*

and if such changes were affected by other clinical co-variables. ASD severity was shown to be associated with changes in production of TNF-α, IL-1ß, IL-10, and CCL2, and TNF-α/sTNFRII ratios (**Table 4**). However, production of inflammatory monocyte cytokines (TNF-α and IL-1ß) under several culture conditions is affected by presence of co-morbid conditions and the use of ADHD medications (**Table 4**).

**Changes in monocyte cytokine production depending on co-morbid conditions:** Since associations between ASD severity and monocyte cytokine profiles were often affected by other co-morbid conditions and medication use (**Table 4**), we also evaluated whether changes in monocyte cytokine profiles in ASD subjects were affected by the presence of co-morbid conditions.

*Co-morbid conditions with objective diagnostic measures:* We evaluated changes in monocyte cytokine profiles in association with co-morbid conditions which were

*1 ANOVA co-variance analysis revealed an association with the use of ADHD medications (p < 0.02 and p < 0.05 under culture conditions stimulated with CL097 and ß-glucan+LPS, respectively).*

*2 ANOVA co-variance analysis revealed an association with GI symptoms (p < 0.05).*

*3 ANOVA co-variance analysis revealed an association with PANS like symptoms (p < 0.05).*

*4 ANOVA co-variance analysis revealed an association with Disturbed sleep (p < 0.05).*

*5 Abbreviations used: CCL2; C-C chemokine ligand 2, IL; interleukin, LPS; lipopolysaccharide, TNF; tumor necrosis factor.*

#### **Table 4.**

*Differences in monocyte cytokine production depending on ASD severity.*

evaluated with objective measures as defined in the method section. We found that such co-morbid conditions were observed in more than 15% of our ASD study subjects; these include GI symptoms, seizure disorders, AR, and SAD. Our results revealed that there are significant associations between monocyte cytokine production and ASD co-morbid conditions as described above (**Table 5**). Presence of GI symptoms are mainly associated with changes in TNF-α production and the ratio of TNF-α/sTNFRII ratios under several culture conditions. Co-variance analysis showed that these parameters were mostly not affected by other clinical co-variables including medication use. The exceptions are TNF-α production and TNF-α/ sTNFRII ratios under zymosan stimulated cultures, which are affected by ASD severity (**Table 5**). Diagnosis of seizure disorders was the most notably associated with changes in IL-1ß production and IL-1ß/IL-10 ratios under the CL097 stimulated cultures, independent of clinical co-variables that we assessed (**Table 5**). The AR diagnosis is mainly associated with changes in sTNFRII production. The SAD diagnosis is mainly associated with changes in IL-6 and IL-10 production. Most of these cytokine parameters were again not affected by the other clinical variables that we assessed.

*Co-morbid conditions based on clinical diagnosis:* Although assessment of PANS like symptoms and sleep disorder were diagnosed without objective measures, given the high frequency of these conditions, we also assessed differences in monocyte cytokine parameters in association with these two co-morbid conditions. Significant differences in certain monocyte cytokine parameters were found in the presence of PANS like symptoms and sleep disorders (**Table 6**). PANS like conditions were associated with changes in inflammatory cytokines (IL-6 and IL-1ß), as well as sTNFRII and CCL2. Only IL-1ß production was affected by other clinical covariables. As for sleep disorders, changes in TGF-ß levels were mainly associated with the presence of sleep disorders and changes in TGF-ß productions was independent of clinical co-variables.

**89**

**Comorbid conditions**

**GI symptoms**

TNF-α (LPS) TNF-α (zymosan)2

TNF-α (ß-glucan)

TNF-α/sTNFRII (LPS)

TNF-α/sTNFRII (zymosan)2

TNF-α/sTNFRII (ß-glucan)

**Seizure disorders**

IL-1ß (CL097) IL-1ß/IL-10 (CL097)

TNF-α/sTNFRII

(CL097) CCL2 (CL097) IL-1ß (ß-glucan)

**Allergic rhinitis**

sTNFRII (LPS)

IL-1ß (CL097)2

sTNFRII (ß-glucan)2

sTNFRII (LPS + ß-glucan)

**Antibody deficiency**

IL-6 (medium)

IL-6 (LPS) IL-6 (zymosan)2

**ASD1 with comorbid condition**

**N = 81** 685.1 ± 744.83 1498.8 ± 1055.6 1961.6 ± 1108.2

0.69 ± 0.89 2.98 ± 2.61 6.84 ± 6.79

**N = 24** 3732.3 ± 1092.8

5.10 ± 7.98 3.89 ± 6.74 18862 ± 19577 1955.9 ± 996.9

**N = 20** 1538.1 ± 395.5 5038.3 ± 1368.3

497.8 ± 225.2 488.8 ± 249.4

**N = 31** 2303.5 ± 1935.8

19536 ± 9625 5510.5 ± 2085.6

**ASD without comorbid condition**

**N = 42** 373.0 ± 366.5 1047.6 ± 709.6 1609.7 ± 1135.7

0.50 ± 1.27 2.67 ± 5.68 3.88 ± 2.31

**N = 99** 4566.4 ± 1357.5

8.98 ± 12.44 8.04 ± 13.74 14588 ± 9672 2546.4 ± 972.1

**N = 103** 1284.0 ± 493.7 4285.8 ± 1315.6

414.1 ± 286.6 392.5 ± 297.3

**N = 92** 3869.2 ± 2581.629553 ± 18279

6309.2 ± 1974.7

388.8 ± 380.0

**Non-ASD Control**

**N = 26** 474.4 ± 505.8 1609.7 ± 748.3 1931.1 ± 948.5

1.28 ± 3.53 3.54 ± 3.61 13.5 ± 18.0

**N = 26** 3715.5 ± 1367.9

3.98 ± 2.53 4.31 ± 3.48 11499 ± 7621 21767.0 ± 999.9

**N = 26**

1172.5 ± 504.2 3715.5 ± 1357.9

340.8 ± 257.7 371.6 ± 354.3

**N = 26**

3373.3 ± 1562.9 20152 ± 12909 7538.5 ± 9310.0

p < 0.01

p < 0.01

p = 0.106

p < 0.05

p < 0.01

p < 0.05

p = 0.07179

p < 0.01 p < 0.001

p < 0.02

p < 0.02

p = 0.056

p < 0.01 p < 0.01 p = 0.113 p < 0.05 p < 0.05 p < 0.01

**Kruskal-Wallis test**

*Associations between Monocyte Cytokine Profiles and Co-Morbid Conditions in Autism…*

*DOI: http://dx.doi.org/10.5772/intechopen.95548*

#### *Associations between Monocyte Cytokine Profiles and Co-Morbid Conditions in Autism… DOI: http://dx.doi.org/10.5772/intechopen.95548*

*Autism Spectrum Disorder - Profile, Heterogeneity, Neurobiology and Intervention*

 **severity Level 1**

**ASD severity Level 2**

**N = 33 N = 37 N = 52**

3.78 ± 3.34 8.43 ± 10.43 8.73 ± 17.27 p < 0.05

TNF-α (CLO97)1 2390.2 ± 1786.9 2720.2 ± 983.3 2073.0 ± 1171.6 p < 0.01 IL-10 (CLO97) 1318.1 ± 595.3 918.2 ± 576.1 1009.9 ± 607.2 p < 0.05

TNF-α (ß-glucan)3 1816.2 ± 1139.4 2360.4 ± 1249.9 1512.2 ± 883.9 p < 0.005 IL-1ß (ß-glucan)4 2578.7 ± 922.0 2809.7 ± 984.4 2062.9 ± 949.7 p < 0.005 TNF-α (ß-glucan+LPS)1 2338.0 ± 259.0 2658.4 ± 984.5 2000.2 ± 916.5 p < 0.01 CCL2 (ß-glucan+LPS) 2708.0 ± 2477.4 1758.6 ± 16681 1574.8 ± 1553.1 p < 0.05 CCL2 (zymosan) 9105.2 ± 6631.1 7668.4 ± 4973.2 5870.3 ± 4414.4 p < 0.05

*ANOVA co-variance analysis revealed an association with the use of ADHD medications (p < 0.02 and p < 0.05*

*Abbreviations used: CCL2; C-C chemokine ligand 2, IL; interleukin, LPS; lipopolysaccharide, TNF; tumor necrosis*

**ASD severity Level 3**

**Kruskal-Wallis test**

**ASD5**

*under culture conditions stimulated with CL097 and ß-glucan+LPS, respectively).*

*Differences in monocyte cytokine production depending on ASD severity.*

*ANOVA co-variance analysis revealed an association with GI symptoms (p < 0.05).*

*ANOVA co-variance analysis revealed an association with Disturbed sleep (p < 0.05).*

*ANOVA co-variance analysis revealed an association with PANS like symptoms (p < 0.05).*

**Monocyte cytokine production**

TNF-α/sTNFRII (CLO97)2

*1*

*2*

*3*

*4*

*5*

*factor.*

**Table 4.**

evaluated with objective measures as defined in the method section. We found that such co-morbid conditions were observed in more than 15% of our ASD study subjects; these include GI symptoms, seizure disorders, AR, and SAD. Our results revealed that there are significant associations between monocyte cytokine production and ASD co-morbid conditions as described above (**Table 5**). Presence of GI symptoms are mainly associated with changes in TNF-α production and the ratio of TNF-α/sTNFRII ratios under several culture conditions. Co-variance analysis showed that these parameters were mostly not affected by other clinical co-variables including medication use. The exceptions are TNF-α production and TNF-α/ sTNFRII ratios under zymosan stimulated cultures, which are affected by ASD severity (**Table 5**). Diagnosis of seizure disorders was the most notably associated with changes in IL-1ß production and IL-1ß/IL-10 ratios under the CL097 stimulated cultures, independent of clinical co-variables that we assessed (**Table 5**). The AR diagnosis is mainly associated with changes in sTNFRII production. The SAD diagnosis is mainly associated with changes in IL-6 and IL-10 production. Most of these cytokine parameters were again not affected by the other clinical variables

*Co-morbid conditions based on clinical diagnosis:* Although assessment of PANS like symptoms and sleep disorder were diagnosed without objective measures, given the high frequency of these conditions, we also assessed differences in monocyte cytokine parameters in association with these two co-morbid conditions. Significant differences in certain monocyte cytokine parameters were found in the presence of PANS like symptoms and sleep disorders (**Table 6**). PANS like conditions were associated with changes in inflammatory cytokines (IL-6 and IL-1ß), as well as sTNFRII and CCL2. Only IL-1ß production was affected by other clinical covariables. As for sleep disorders, changes in TGF-ß levels were mainly associated with the presence of sleep disorders and changes in TGF-ß productions

**88**

that we assessed.

was independent of clinical co-variables.

*with allergic rhinitis was affected with the use of anti-seizure medications (p < 0.05). Changes in production of IL-6 (zymosan) and IL-10 production (LPS) with antibody deficiency was affected with the use of neuroleptics/SSRIs and PANS like symptoms.*

*3The results were expressed as a mean ± SD. Cytokine levels were shown as pg/ml.*


**6.2 The way forward**

*Innovation in the Food Sector Through the Valorization of Food and Agro-Food By-Products*

exceeding that of soybean and groundnut. Maggots can be harvested, processed into a meal that can be used to substitute or replace fish meal [61, 62]. Maggot grown on a mixture of cattle blood and wheat bran contained 92.7% dry matter, 47.6% crude protein, 25.3% fat, 7.5% crude fiber, 6.25% ash, and an amino acid profile comparable to fish meal [59] suggesting that animal wastes utilization can be used to produce insects which can be utilized as fish feed hence, reduce feed cost significantly, thus leading to a viable and sustainable aquaculture industry. The replacement of 25 percent fishmeal in catfish feed, culture with maggot gave high growth performance and profitability than fishmeal based diet [63, 64]. Several researches [65, 66] have been reporting on the use of red worms, black soldier fly, common housefly, and yellow mealworm as a source of protein to replace fishmeal. It is envisaged that the valorization of animal and animal by-products such as animal blood, offal of poultry, residues of traditional brewery waste, animal manure and fish wastes may contribute significantly on fish production hence, food

It is clear that fish consumption in Sub-Saharan Africa is increasing. In order to maintain the present amount of fish consumption, considerable additional quantities of fish are required through aquaculture. In turn, aquaculture requires feed as a major input for increasing production. Since commercial fish feed production in most of the sub-Saharan countries is limited, considerable investments are required in local and low costs feed manufacturing. Raw materials of plant and animal origin are sufficiently available in the region albeit the possible competition from livestock and human consumption. Therefore, valorisation of animal and agro-products in the Sub-Saharan countries is imperative/inevitable for increasing fish production at low cost.

In order to increase food nutrition from aquaculture production through valorization of agro-by-products in the sub-Saharan countries, the following are

• Strengthen the use of Public Private partnering by putting more emphasis in services related to the collection of feed ingredients and preservation

• Recognize small scale farmers as commercial ones and encourage small-scale

• Provide credit facilities for the private sector particularly for the small-scale

• Put emphasis on public private research partnerships and knowledge sharing

• Provide capacity building and general education for small holders in order to improve their technological, managerial and commercial skills in handling

• Public Private partnering must be embedded into an economic vision for

farmers to work together by forming associations (work groups)

**62**

nutrition and security.

**6.1 Recommendations**

aquaculture development

recommended:

holders

on valorisation

agro by-products

**6. Conclusion**

With the ever-increasing human populations in sub-Saharan countries, the demand for food would increase and natural resources will become even scarcer. This situation will be more worsen with severe climate changes. These trends necessitate for a critical assessment of the situation to enable devise informed solutions in addressing issues pertaining to agro by-product processing and valorization.