**5. Operating room fire: true magnitude of the problem**

According to the Emergency Care Research Institute (ECRI), approximately 200–240 surgical fires occur each year in the United States [28]. Other sources provide a much wider range of occurrences, ranging between 100 and 2260 annually [2, 24, 29, 30]. Generally speaking, the incidence of ORF appears to be similar to that of wrong-site surgery or retained surgical items [28], some of the most prominent categories of surgical "never events" [31–33]. As outlined in previous sections, the simultaneous presence of key components required for the ignition of a fire is the single biggest risk determinant (**Table 1** & **Figure 1**). Therefore, it is not surprising that surgical fires involve electrosurgical equipment in approximately 67–90% of all cases, and that supplemental oxygen administration was nearly universally present [2, 34]. Of importance, the operative environment is defined as being "oxygen-enriched" when the oxygen concentration is greater than 21% [17]. Most commonly and not surprisingly, given the previously outlined risk factors, ORFs result in burns to the head, face, neck, and upper chest [22]. Thankfully not all ORFs involve patients, operating room staff, or result in significant injury [2].


Advanced Functional Materials for Textile

**243**

**Chapter 14**

**Abstract**

*Dilan Canan Çelikel*

typical applications of active smart textiles.

also be called as the next-generation textiles.

phase-changing materials

**1. Introduction**

Smart E-Textile Materials

Smart textiles are one of the areas that provides added value to textile materials. It is a sector that has been developed with new technologies, new fibers, and textile materials. The production of smart or intelligent textiles cooperate with other branches of science like nanotechnology, materials science, design, electronics, and computer engineering, etc. Smart textiles are classified into three groups as passive smart textiles, active smart textiles and ultra smart textiles according to their performance characteristics. Passive smart textiles are the first generation of smart textiles and sense the external conditions; for instance, UV protecting clothing, conductive fibers, etc. As active smart textiles respond to external conditions, ultra smart textiles sense, react, and adopt themselves to conditions.  In other words, technology has also taken control of textile industry. Smart textiles have superior performance and functionalities for the applications ranging from simple to more complicated uses such as military, healthcare, sportswear, etc. Smart or intelligent textiles can

Many classifications related to smart textiles are available in the literature. In this chapter, the classifications based on the esthetic and performance functions are mentioned as two categories. Esthetic smart textiles use the technology for fashion design, because of their ability to light up and change color. Light-emitting clothes and luminous dresses are the typical and commercial examples for esthetic smart textiles. As for the performance, smart textiles are classified into three categories as

Passive smart textiles can only sense the environment, as they are just sensors.  Active smart textiles can sense the stimuli from the environment and also react to them; besides the sensor function, they also have an actuator function. Phase change materials, shape memory materials, and heat sensitive dyes are active smart textile applications.

passive smart textiles, active smart textiles, and ultra smart textiles.


**2. Vegan diet: risks**

*Veganism - a Fashion Trend or Food as a Medicine*

animals.

People become vegan for many reasons, including ethical care for animals and the natural environment, as well as circumstances related to health, spirituality, and religion [1]. As far as ethical principles are concerned, a vegan diet prevents the mass breeding and systematic slaughter of large numbers of animals on farms. Vegan food contains lower levels of cholesterol and fat than the usual diet. From the ecological point of view, the meat industry participates in the pollution of air, land and water, contributes to the exploitation and deforestation of forests and large land areas for the cultivation of crops intended for feeding a large number of farmed

The basis of a healthy vegan diet is all vegetables, vegetable proteins, good fats and whole grains. However, vegans should pay attention to the intake of calcium, magnesium and vitamin D. Adequate intake of vitamin B12 is especially important, which is available in certain types of herbal drinks such as herbal drinks that do not contain lactose or added sugars and are especially suitable for vegans. A wellplanned vegan diet can meet all the needs of the body. A poorly organized vegan diet can cause a lack of calcium, iodine, iron, vitamin B12 and vitamin D, which

According to surveys, vegans make up between 0.2% and 1.3% of the American

The benefits of a vegan diet are great - it reduces the risk of cancer, cardiovascular disease, myocardial infarction, stroke, rheumatoid arthritis, high blood pressure,

Many studies suggest a link between cancer and diet [4]. More than half of these

Vegetarians and vegans generally include greater amounts of plant foods, avoid

the intake of meat, and often adopt other healthy lifestyles compared to nonvegetarians [9]. Thus there is reason to suspect that vegetarian diets may protect against cancer. Factors associated with the high fiber content in vegetarian diets promote increased insulin sensitivity [10]. Plant-based diet is associated with lower circulating levels of total IGF-I and higher levels of IGFBP-I and IGFBP-2 compared with a meat-eating or even a lacto-ovo-vegetarian diet [11]. Insulin and IGF-I act as promoters for most normal and pre-neoplastic tissues. Therefore, their down- regulation may reduce cancer rates [12]. The strongest evidence linking specific foods to decrease risk of certain cancers includes the consumption of fruits and vegetables and whole grains [13]. Studies show a strong inverse relationship between dietary fiber intake and colon cancer in populations at low risk for the disease [14]. Several hypotheses have been postulated to explain this effect, 2 of which are outlined as follows. First, fiber increases the bulk of the stool, which decreases transit time and, in turn, could decrease the time colonic epithelial cells are exposed to potential fecal carcinogens. Second, bacteria in the gut can ferment fiber to short-chain fatty acids such as butyrate. These fatty acids may promote colonic cell differentiation and normal cell apoptosis [15]. It is estimated that an

cancer cases are potentially preventable. Diet affects approximately 30% of all

The diet enables the assessment of the connection between the disease and the intake of certain foods in relation to the usual diet [6]. Several studies have been published that deal with the relationship between dietary factors and overall cancer risk. The vegan diet is thought to be inversely related to the overall incidence of cancer. There are studies whose results for certain cancers are not in line with the diet. This lack of clarity may result from the heterogeneity of vegetarian diets among respondents in different countries, as they may vary widely in relation to the ratio of animal and plant foods eaten, food quality, cooking methods, limitations of measures used to quantify dietary nutrition, and other factors may affect the

must then be taken from vitamin and mineral supplements [2].

asthma, allergies and kidney stones.

development of cancer [7, 8].

population and between 0.25% and 7% of the general population [3].

cancers in developed countries and 20% in developing countries [5].

**34**

Most of the data on the vegan diet is based on the adult population. Less is known about the vegan diet of newborns and children. Children have a greater need for energy and protein, which is met by vegetables, tofu, beans, whole grains, nuts and seeds. It is recommended for 10% more protein in childhood than in adult vegans. For this reason, alternatives rich in low-fiber vegan proteins, such as tofu and seitan, may be desirable, as these foods usually result in high satiety and can support adequate protein intake. Because animal foods such as meat, dairy, eggs, fish and fowl are among the best sources of protein, vegans can sometimes lack sufficient protein in their diets. The American Dietetic Association, however, notes that protein requirements can be met if a variety of plant proteins are consumed. Complementary proteins, specifically, can be very helpful in providing all the essential amino acids required by the body. Complementary proteins are made up by two incomplete proteins, such as beans and rice, that, when combined, form a complete protein. These proteins do not need to be consumed at the same meal, just during the same day [18]. Though vegan diets are often rich in omega-6 fatty acids, they can lack sufficient amounts of omega-3 due to the restriction of omega-3-rich foods, including eggs, fish and other seafood. The omega-3 fatty acids found in animal sources, which include eicosapentaenoic acid and docosahexaenoic acid, are important for cardiovascular, brain and eye health. Plant foods rich in omega-3, such as flaxseed, only contain another type of fatty acid, known as alpha-linolenic acid. Vegans can purchase soy milk and breakfast bars fortified with DHA, as well as DHA supplements derived from microalgae. Because DHA can be retroconverted to EPA, sources containing DHA are sufficient for vegan diets. Vitamin D is important for bone health, and low levels have been linked to reduced bone mass. Sunlight exposure is best source of vitamin D. Therefore, vegans who do not get regular sun exposure need to consume dietary vitamin D through either fortified foods or supplements, since vitamin D occurs naturally in very few foods. Vegan sources of vitamin D that are often fortified include soy milk, rice milk and orange juice. Lack of vitamin B-12 is one of the biggest concerns with vegan diets, especially because vegan diets are typically rich in folacin, which can mask B-12 deficiency symptoms [6]. Severe B-12 deficiencies can result in both anemia and dementia, notes Harvard Medical School. So, if you are a vegan, be sure to consume B-12 fortified foods such as soy and rice beverages and breakfast cereals, or supplements. Because the bioavailability of iron in vegan diets is lower than in traditional diets, vegans should consume 1.8 times the amount of iron consumed by nonvegans. Good sources of iron for vegans include dried beans and dark, leafy green vegetables. Zinc is another mineral whose bioavailability is lower in vegans than in nonvegans. Vegan sources of zinc include legumes, soy products, grains and nuts. Some research suggests that because plant-based diets are low in iodine, vegans who do not consume key sources of iodine, including iodized salt and sea vegetables, are more likely to be deficient in the mineral. The American Dietetic Association notes that vegans tend to fall below the recommended intake of calcium. The calcium in certain calcium-rich vegetables, such as Swiss chard and spinach, is not absorbed well, making fortified foods such as fruit juices, breakfast cereal and soy and rice milk among the best choices for vegans [2].


**2. Research method**

#### **2.1 Grouping overview**

This study has collected clinical data from 298 female patients undergoing breast reconstruction surgery following breast cancer surgery, ranging in age from 25 to 67 years (mean age 42.2 ± 8.28 years). Before undergoing reconstructive surgery, all patients had conventional tumor therapy and were in a stable disease phase. In January 2016 and January 2018, 92 patients (US group) had their data gathered. Prior to surgery, these patients were subjected to a color Doppler ultrasound examination to examine abdominal vascular perforators using a GE Logiq 700 ultrasonography machine with a probe frequency of 10–13 MHz and a color Doppler frequency of 7.5 MHz, with a maximum velocity of 2 cm/s. The focus of the examination was to accurately detect where the target perforator vessels exited the deep fascia and entered the subcutaneous fat layer, and then to label the matching surface projection. As the observation group, data from 206 patients was gathered between January 2018 and January 2021 (CTA group). These patients underwent abdominal vascular perforator evaluation prior to surgery using a Philips Brilliance 256i CT scanner, which used a non-ionic iodine contrast agent (iopromide) with a volume of 90–100 mL and an injection rate of 4 mL/s, introduced via a vein on the back of the hand or at the elbow. The following CTA settings were employed with a single high-pressure injector: 120 kVp, 250 mA, detector 128 × 0.625 mm, pitch 0.977, 512 × 512 matrix, and a 350 mm field of view.

#### **2.2 CTA image data processing**

The HOROS software was used to evaluate the data and establish the location and trajectory of perforator vessels, which were then marked on the patient's body surface.

#### **2.3 Research indexs**

Baseline data were collected for both groups and compared, including age, body mass index (BMI), history of hypertension or diabetes, history of tumor chemotherapy, chest wall radiation history, abdominal surgery history, smoking history, and single/double pedicle flap blood supply. In addition, intraoperative and postoperative evaluation indices were statistically analyzed. Intraoperative evaluation indices included the number of perforators located preoperatively, the number of perforators utilized during the surgery, the decision-making time for perforators, perforator dissection time, flap harvesting time, flap total weight, and total operation time.


**2. Why the surveillance on cyanobacteria?**

60 Novel Approaches and Their Applications in Risk Assessment

Cyanobacteria are autotrophs and possess all the photosynthetic pigment (chlorophyll *a*, carotenoids, allophycocyanin, phycobilins, phycoerythrins) except chlorophyll *b* (Castenholz, 2001). Prochlorophytes are also cyanobacteria that contain chlorophyll *a* and *b*, but, opposing to other cyanobacteria, lack phycobiliproteins (Castenholz, 2001). Cyanobacteria have the ability to use low light intensities effectively, since they are able to produce the accessory pigments needed to adsorb light most efficiently in the habitat in which they are present, providing them a great advantage for the colonization of a wide range of ecological niches (van den Hoek et al., 1995; WHO, 1999). Phycobiliprotein synthesis is particularly susceptible to environmental influences, especially light quality. The chromatic adaptation is largely attributable to a change in the ratio between phycocyanin and phycoerythrin in the phycobilisomes. The photosynthetic pigments are located in thylakoids that are free in the cytoplasm near the cell periphery (Fig. 2). Cell colours vary from blue-green to violet-red due to the chlorophyll *a* masking by the carotenoids and accessory pigments. The pigments are involved in phycobilisomes, which are found in rows on the outer surface of the thylakoids (Fig. 2) (WHO, 1999). Cyanobacteria are also able of storing essential nutrients and metabolites within their cytoplasm. Prominent cytoplasmic inclusions such as glycogen and cyanophycin granules (polymers of the amino acids arginine and asparagine), polyphosphate bodies, carboxysomes (containing the primary enzyme for photosynthetic CO2 fixation, ribulose 1,5-bisphosphate carboxylase-oxygenase: RuBisCO) and gas vacuoles (Fig. 2) can be observed by electron microscopy. The occurrence of fimbriae (pili) is abundant in many cyanobacteria with varying patterns. Some filamentous forms are also able of gliding (sliding) (van den Hoek et al., 1995; WHO, 1999; Castenholz, 2001).

Fig. 2. Cyanobacteria cell structure. (A)Transmission electron micrographs showing the ultrastructure of an *Anabena circinalis* vegetative cell; (B) Schematic diagram of a cyanobacterial vegetative cell. S: external 4-layered cell wall; OM: outer membrane; PL: peptidoglycan layer; CM: cytoplasmic membrane; CW: cell wall; E: cell envelope; TH: thylakoid; PB: phycobilisome; CY: cytoplasm; GV: gas vesicle; GG: glycogen granules; N: nucleoplasmic region; C: carboxysome; PP: polyphosphate granule; CP: cyanophycin granule; LP: lipid droplets (adapted from van den Hoek et al., 1995; Castenholz, 2001).

Cyanobacteria are common constituents of the phytoplankton in aquatic environments. In optimal conditions these phytoplanktons can develop massively and form blooms, becoming the dominant organism in the water column and creating serious problems in water quality (Cood, 2000; Vasconcelos, 2006). The water quality deterioration produced by cyanobacterial blooms includes foul odours and tastes, deoxygenation of bottom waters (hypoxia and anoxia), fish kills, food web alterations and toxicity. Other threatening characteristic of these organisms is their ability to produce toxins that affects other living organisms and humans (Carmichael, 2001). The capacity of mass development together with the ability to produce potent toxins enlightens the importance of implementing regular monitoring programs for cyanobacteria and cyanotoxins in freshwater environments, in order to minimize potential health risks to animal and human populations that results from exposure through drinking and recreational activities. The implementation of surveillance programs on cyanobacteria involves understanding the ecophysiology of cyanobacteria, bloom dynamics, conditions that promote blooms, production of toxins and their impact in human and animal health (McPhail & Jarema, 2005).

Cyanobacteria possess some ecostrategies that allows them to overcome other organism and become dominant. In general there are four constraints on cyanobacteria growth as prerequisites for bloom enhancement: light, nutrients, temperature and stability of the water column. Cyanobacteria requires low light intensities for growth, compared with algae, which provides competitive advantages in lakes which are turbid due to growth of other phytoplankton. They also have a higher affinity for uptake phosphorous and nitrogen than many other photosynthetic organisms and they have a substantial storage capacity for phosphorous (Mur et al., 1999). Some genera like *Anabaena, Aphanizomenon*, *Cylindrospermopsis*, *Nodularia* and *Nostoc* have specialized cells (heterocysts) (Fig. 1) for nitrogen fixation and blooms of these genera can often be related with periodic nitrogen limitation. This means that they can compete other phytoplankton under conditions of phosphorous and nitrogen limitation (Briand et al., 2003; Sunda et al., 2006).

The success of some cyanobacteria is also due to the presence of gas vacuoles that provide buoyancy regulation. During water stratification conditions cyanobacteria can migrate in the water column, accessing light in the surface layers and nutrients near the sediment. During photosynthesis, carbohydrates are accumulated which makes them heavy and sinking away from light and when the carbohydrates are respired, buoyancy is restored. As large colonies sink faster than small ones or single cells, genera like *Microcystis*, *Anabaena*, *Aphanizomenon* and *Nodularia* have scum-forming strategies (Vance, 1965; Mur et al., 1999). Cyanobacteria also produce active substances that inhibits the growth of competing algae and grazers that feed upon them, which can also promote cyanobacteria proliferation (Briand et al., 2003; Granéli & Hansen, 2006; Sunda et al., 2006; van Apeldoorn et al., 2007; Figueredo et al. 2007). As a consequence of the characteristics mentioned above the cyanobacterial cells numbers in water bodies vary seasonally. In temperate regions, seasonal successions of organisms belonging to different phytoplankton taxa are often observed. Whereas at the beginning of

Risk Assessment of Cyanobacteria and Cyanotoxins,

the Particularities and Challenges of *Planktothrix* spp. Monitoring 63

Table 1. Cyanotoxins detected and correspondent taxa from which have been isolated, as well as their primary target in mammals.

Table 1. Cyanotoxins detected and correspondent taxa from which have been isolated, as well as their primary target in mammals.

Risk Assessment of Cyanobacteria and Cyanotoxins,

(reviewed in Pouria et al, 1998).

et al. 2001).

the Particularities and Challenges of *Planktothrix* spp. Monitoring 65

Example 2 – Symptoms after exposure during heamodialysis treatment: weakness, muscular pain, nauseas, vomiting, neurologic symptoms (head pain, vertigo, deafness, blindness and seizures), increase of hepatic damage biomarkers, hepatomegaly, hepatic failure and death

Fig. 3. Organizational chart of the steps involved in risk assessment (adapted from Dolah

supplied with untreated- or ineffective-treated water.

Besides the acute effects mentioned above, few papers reports the association between the ingestion of water contaminated with microcystins and the increase of hepatocarcinoma (Yu, 1995; Ueno et al., 1996) and colorectal cancer (Zhou et al, 2002) in human populations

Laboratorial studies have demonstrated that, in fact, microcystins, nodularins and cylindrospermopsin are genotoxic (reviewed in Zĕgura et al., 2011) and the carcinogenic

the summer a great variety of microalgae and cyanobacteria usually co-exist in the same water body, towards the end of summer this diversity may drop drastically as the result of the mass development of the cyanobacterial communities (blooms) (Sze, 1986). These blooms may be formed by a consortium of cyanobacteria producing different amounts of toxins at different rates, with the same bloom-forming species having both toxigenic and non-toxigenic strains, indistinguishable by morphological examination. Cyanobacterial blooms are complex and can develop in a rather sudden and unpredictable way.


*Edited by Alessandro Zorzi, Hechmi Toumi and Eric Lespessailles*

Osteoarthritis is the most common form of arthritis and cause of joint pain in adults. It affects more than 500 million people worldwide. Patients develop osteoarthritis when the structure of the protective cartilage that covers the bones begins to deteriorate. Recent research shows that multiple drugs in development are promising to treat osteoarthritis. Yet, most physicians agree that we still do not have enough effective treatments for this condition. What works for one person might not work for someone else. Consequently, osteoarthritis research is very important because understanding what causes osteoarthritis or why it progresses remains controversial. In fact, although articular cartilage is the primary target of the disease, both subchondral bone and synovium are often actively involved. This book describes the details of both surgical and non-surgical treatment options for osteoarthritis.

Published in London, UK © 2023 IntechOpen © Senkumar Alfred / iStock

Arthroplasty - Advanced Techniques and Future Perspectives

Arthroplasty

Advanced Techniques and Future Perspectives

*Edited by Alessandro Zorzi,*

*Hechmi Toumi and Eric Lespessailles*


**Neutrophils Plasticity: The Regulatory Interface in Various Pathological Conditions**

Suelen Martins Perobelli,

Triciana Gonçalves Silva and Adriana Bonomo

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/68130

#### **Abstract**

It is now known that neutrophils make up a population of complex cells with great plasticity, challenging the old view of neutrophil association with tissue damage and early phases of infection. Here, we discuss different contexts in which these cells can induce anti-inflammatory responses. Although distinct surface markers and cytokines profiles were shown, the most reliable characterization of suppressor neutrophil subtypes relies on their functional characteristics. One important example of inhibitory neutrophils generation comes from in vivo treatment with G-CSF, for 5 days, as for hematopoieticstem-cell-transplantation (HSCT). In this case,donor blood is enriched in degranulated granulocytes harboring a functional regulatory phenotype, characterized by IL-10 production. These cells, when transferred together with HSCT, are able to reduce graft-versus-host-disease, being influenced by Treg cells and influencing them back. Importantly, this protection is long lasting and specific, keeping immunocompetence to other antigens. This regulation is paramount in HSCT, and represents a simple approach to be applied in humans. In summary, we discuss the interaction of neutrophils with other cell types and its consequence in immunomodulation. We believe these features confer an important bridge between innate and adaptive immune system, building a new knowledge for an underestimated cell type.

**Keywords:** regulatory neutrophils, neutrophils subtypes, T cell inhibition, Cytokines, G-CSF, GVHD

© 2017 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


*Article* **Designing a User-Centered Interaction Interface for Human–Swarm Teaming**

**Mohammad Divband Soorati 1,\*, Jediah Clark 1, Javad Ghofrani 2, Danesh Tarapore 1 and Sarvapali D. Ramchurn 1**

**Abstract:** A key challenge in human–swarm interaction is to design a usable interface that allows the human operators to monitor and control a scalable swarm. In our study, we restrict the interactions to only one-to-one communications in local neighborhoods between UAV-UAV and operator-UAV. This type of proximal interactions will decrease the cognitive complexity of the human–swarm interaction to *O*(1). In this paper, a user study with 100 participants provides evidence that visualizing a swarm as a heat map is more effective in addressing usability and acceptance in human–swarm interaction. We designed an interactive interface based on the users' preference and proposed a controlling mechanism that allows a human operator to control a large swarm of UAVs. We evaluated the proposed interaction interface with a complementary user study. Our testbed and results establish a benchmark to study human–swarm interaction where a scalable swarm can be managed by a single operator.

**Keywords:** human–swarm interaction; situation awareness; user-centered interface design

#### **1. Introduction**

Unmanned aerial vehicles (UAVs) are now commonly used to rapidly gather situational awareness in emergency responses or security applications. However, most applications are limited to high operator (*n*) to vehicle ratios (*m*), i.e., *n* ≥ *m*. This severely limits the use of inexpensive, low-power, UAVs at scale. Hence, a number of works have focused on using a swarm of autonomous UAVs [1–3], where the UAVs have high degrees of autonomy and can coordinate with each other using simple rules and low-power communication. These limitations on computation and communication (e.g., to avoid using up their battery or to avoid detection by threats) introduce huge challenges when it comes to monitoring and managing the collective actions of the swarm [4]. Specifically, as the size of the swarm grows, so does the complexity that human operators face in terms of directing individual UAVs, orienting to the, possibly delayed, information relayed by the swarm, and monitoring the health of the swarm (e.g., UAVs may be taken down by threats such as fires or enemy attack).

Against this background, in this paper, we aim to the improve operator to vehicle ratios (to *n* < *m*) for large scale deployments of UAV swarms. We develop the notion of human–swarm teaming with proximal interactions, whereby, operators can direct the swarm via coarse instructions. The operator can have one-to-one interactions (in the worst case) with members of the swarm. Similarly, individual UAVs can only communicate with their neighbours (UAVs or human) within a limited range. By 'coarse instructions' we mean that they cannot individually instruct each UAV where to go but set a target for the swarm to achieve. This control approach is in a similar vein to the indirect parameter setting that exists in the literature [5,6]. Indeed, each UAV maintains a high degree of autonomy and can decide where to go and what to do based on its local inputs.

**Citation:** Divband Soorati, M.; Clark, J.; Ghofrani, J.; Tarapore, D.; Ramchurn, S. Designing a User-Centered Interaction Interface for Human–Swarm Teaming. *Drones* **2021**, *5*, 131. https://doi.org/ 10.3390/drones5040131

Academic Editor: Diego González-Aguilera

Received: 30 September 2021 Accepted: 2 November 2021 Published: 5 November 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Our approach builds upon interactive user interfaces to allow operators to visualize the state of the swarm (state estimation), direct the swarm (control), and monitor the information it generates (monitoring). Our interfaces combine direct manipulation with autonomous UAV decision-making. We do not force the swarm to maintain a communication channel with the human UAVs at all times [7,8]. The operator, instead, uses the messages that the UAVs broadcast (to their nearest neighbor) to estimate the swarm's state. Lindner et al. [9] evaluated the complexity of supervising a swarm of UAVs with a manned aerial vehicle by displaying the swarm as an avatar to the plane pilots. Despite the effectiveness in reducing the cognitive complexity, the resolution of the swarm's coverage display is greatly sacrificed. The pilots in the study reported that the monitoring of the swarm was complex and that more swarm autonomy and control options are required. Patel et al. [10] use an interface with augmented reality to specify the overall goal for the swarm with a mixture of robot and environment-oriented modalities. We use a more informative visualization compared to [9] and there is flexible autonomy that allows the operator to adjust the autonomy of the swarm by modifying the messages that are exchanged with the UAVs [11]. The operator can choose to not intervene and grant full autonomy to the swarm, which makes our study different from [10]. There are also other studies that focus on the autonomy level of the swarm [12,13], different human–swarm interaction methods [10,14,15] and visualization techniques for the human–swarm systems [8,16] but this paper advances the state of the art in the following ways:

When taken together, our methods and results provide a new benchmark for the development of human–swarm teaming models and interaction mechanisms that embed the notion of flexible autonomy.

#### **2. User Evaluation of Human–Swarm Visualization Methods**

To explore the basis in which display methods should be designed, this section presents findings on how users' acceptance and usability vary between individual UAV and heat map swarm coverage visualization methods. This user-interaction study provides the basis in which models, maps, and simulations are constructed in the following sections.

#### *2.1. Design and Procedure*

One hundred participants (62 female, 37 male, 1 non-binary) took part in an online visualization evaluation. Ethical approval was given by the University of Southampton ethics committee (ERGO number: 66360.A1). To be included in the user-study, participants were required to hold a BSc degree or an equivalent qualification. Participants watched a video of a swarm display showing individual UAVs, followed by a heat map to represent coverage.

Videos were created by recording a rudimentary UAV simulation displaying only swarm coverage with no human-interaction involved. Each video consisted of 35 s, where 50 drones were represented using black-point dots or a density heat-map indicating the density of UAVs in 2D space. After viewing each video, participants were asked questions from the System Usability Scale [17] and the System Acceptance Scale [18] to measure how easily each could be interpreted. After viewing both videos, participants were asked to provide their preferred display method given a variety of contextual factors related to swarm size, communication-constrained environments, coverage, time-criticality, error detection and transparency (see Table 1).

#### *2.2. Results*

Responses to the post-study questionnaire were analyzed using chi-square goodness of fit tests [19] to identify any preferences for heat map or individual drone displays for a variety of swarm/environmental contexts. The alpha level was corrected using the Bonferroni method [20], according to the number of analyses conducted. The chi-square goodness of fit analyses showed that heat maps were preferred when a larger swarm is being displayed, to display motion and coverage, and when time was a critical factor to task success (see Table 1). On the other hand, individual drone displays would be preferred for detecting errors in the swarm. There was no significant difference when communication would be constrained, when time was not a critical factor of task success, and for being more transparent with the operator. Table 1 shows the frequency of participants that reported either individual or heat map displays as their preferred display method, given each contextual factor.

**Table 1.** Table to show the frequency of display preferences among users for each contextual factor and respective Chi-square analysis results.

Note. Bonferroni Corrected *α* = 0.0071. \* = *p* < 0.0071.

Two repeated measured ANOVAs [21] were conducted to identify whether the coverage display method had an effect on the System Usability Scores and System Acceptance Scores reported by participants. The analyses found a significant effect of displaytype on usability scores (F(1,99) = 22.53, *p* < 0.001, *ηp* 2 = 0.185) and acceptance scores (F(1,99) = 29.89, *p* < 0.001, *ηp* 2 = 0.232) indicating that the heat map display method had a greater level of usability and acceptance among participants when compared with displaying individual UAVs. These results show that, overall, heat map displays are perceived as being more usable, acceptable, and more effective in displaying information when time is less available and when large numbers of UAVs are being represented.

#### *2.3. Conclusions*

This user-study provides evidence that heat map methods are more effective in addressing usability and acceptance in human–swarm interaction (see Figure 1). Further, in situations of larger swarm sizes, time-criticality, and displaying motion and coverage, a heat map has shown to have a higher preference among participants. Conversely, for detecting errors within the swarm, individual drone displays may be more appropriate. Due to 79 of the participants stating that a larger swarm sizes would be better displayed via a heat map, this display method shows greater promise for scalability. However, under conditions where detecting errors with individual drones is crucial to task performance, an individual drone display may be more appropriate, perhaps added as a diagnostic tool in addition to the heat map display, rather than being the core display method. Given these results, a heat map interaction method forms the basis for the development of human–swarm teaming interactions in the following sections.

**Figure 1.** Bar graphs to show mean usability (**a**) and acceptance (**b**) scores from subjective measures. Error bars = 95% CIs.

#### **3. Human–Swarm Teaming Model**

In this section, we describe the UAV model and the interactive user interface.  We will use the term UAV and agent interchangeably in the rest of this paper.

#### *3.1. UAV Model*

We consider that agents can be heterogeneous in terms of their capabilities (e.g., influence, dynamics, sensor). This means that they may gather information differently and at different speeds. The speed of the UAVs impacts on the delays in gathering information. In terms of completing a mission, the sensing capabilities may determine the number of dimensions that an operator has to consider and the size of messages that are exchanged. Here, we model UAVs that only return information about their own health; for instance, whether they are flying, the sensors are on or not, and whether they have detected a target or not. The approach can be extended to cover the detection of different kinds of targets, and we will leave this for future work. For now, we will only consider UAVs that can return an integer value denoting the state of their position, for example, the temperature that may be indicative of a fire, or the number of people in their area. We use a point model and we do not consider external environmental factors such as gravity, wind, etc.

#### *3.2. Operator and Swarm Interactions*

UAVs move around the mission zone, over the human operator, and over the disaster zone (see Figure 2). We can add an arbitrary number of operators and disaster zones, but in this paper, our scenario contains one operator, one disaster area, and 15 UAVs. The operator can communicate with the UAVs that are in their local neighborhood and send/receive information. Similar to other UAVs, the operator can also move around and explore the environment, but in this study we assume that the operator is fixed and the UAVs move at most 5 pixels at each simulation step. In our scenario of disaster management, we set the position of the human operator and the disaster as important points that need to be noted and stored by the UAVs. The motivation for marking both operators and disasters is to facilitate further use cases in disaster management, such as forming a safe path between the operator and the disaster area (not discussed in this paper).

Our interface design contains two heat maps, a belief and a confidence map, to comply to the findings of our initial user study (see Section 2). The operator uses these two maps to observe and control the swarms.

**Figure 2.** A map of a mission zone with UAVs, a disaster, and a human operator. All agents (UAVs and the operator) have belief and confidence maps with a lower resolution (40 × 40) compared to the actual images (600 × 600). The operator uses the interface to communicate with the UAVs in their local neighborhood.

#### 3.2.1. The Belief Map

UAVs continuously explore and observe the mission zone for potential disasters or the areas with the human operators. A matrix—a belief map—is used as a map representation of the mission zone. UAVs use the belief map to store their observation of the area. Figure 3a–c show the formation of the operator's belief map over time, where the location of the human operator and the disaster zone are highlighted. The location of the interest areas in the belief map emerge very early in the process. Over time, the UAVs obtain a stronger belief on the area that they explore. This is shown by intensity of the belief that increases over time and turns into red.

**Figure 3.** The operator's belief map at several simulation steps.

#### 3.2.2. The Confidence Map

Another heat map—a confidence map—not only represents the distribution of the swarm as seen by individual agents (see Figure 4) but also determines the agents' certainty level of the information in the corresponding cell of the belief map. We later show how the same map is used for controlling the swarm as well.

**Figure 4.** Confidence map at *t* = 10, 500, 1000 simulation steps.

Using the belief and confidence maps, the agents store their representation of the environment and keep their confidence level about its correctness. As agents move and explore the mission zone, the cells in the belief maps are updated. The corresponding confidence values of the recently visited cells in the confidence map also increase.

#### *3.3. Human–Swarm Collective Decision-Making Model*

Each drone has a limited perception and communication range that makes it hard for them to individually solve a complex task. The swarm can establish a distributed sensing on a large area of the environment and when working together, the swarm will be able to construct an opinion about a certain property of the environment. In our earlier work, we have shown how a swarm can collectively decide which half of an environment is brighter and stay adaptive to changes [22]. This class of problems in swarm robotics is called the *best-of-n problem*, that is, the capability of the swarm to find the best option among a finite set of alternatives [23]. An unsolved challenge is to provide a collective decision-making strategy that is capable of solving the best-of-n problem with spatial constraints [24]. In our context, the swarm is exploring the environment and the swarm's motion is limited (e.g., not permitted in no-fly zone). The second challenge is that in the literature, the collective perception of the features in an environment is limited to few options [25]. To the best of our knowledge, none of the existing collective decision-making strategies including Direct Modulation of Voter-based Decisions [26], Direct Modulation of Majority-based Decisions [27], Direct Comparison [25] that are widely common in the swarm robotics applications can be directly applied to our scenario, as UAVs need to continuously measure the environment and disseminate whenever another UAV or an operator is in range. We are building upon our previous work on adaptive decision-making in robot swarms and add a human in the loop to find a choice among a large set of options in a dynamic environment [22].

The operator, similar to the UAVs, has a confidence that is updated based on the neighboring UAVs. The operator can decide to manipulate their own map, regardless of the current state, and propagate it back to the swarm. The modified confidence map, once received by the UAVs, enforces an attraction/repulsion mechanism that controls the swarm. The operator can focus on the overall goal and implement strategies abstracted from the detailed interactions and behaviors of each UAV, making it an ideal multi-robot organizational scheme. The cognitive complexity of such a form of interaction between the human and the swarm, where the human has to operate a maximum *C* number of UAVs (the number of agents that can fit into a communication range of an operator), regardless of the swarm size, is *O*(1) [28].

The operator receives the maps from the UAVs in their neighborhood and may decide to temporarily explore a certain region in the mission zone as external information may indicate a disaster. The UAVs can be, temporarily or permanently, repelled from a prohibited region (e.g., high voltage cables). The operator can manipulate their own confidence map and introduce a major decrease/increase in the certainty of an interest zone, and constantly share it with the swarm. This forces the swarm to react to the new forces and collectively explore the areas with low confidence or avoid the areas with high confidence.

Agents continuously disseminate their belief and confidence maps as they meet. Equations (1) and (2) determine the update for each cell in the maps of agent *i* after communication with agent *j*.

$$c\_{i,t}(m,n) = \begin{cases} c\_{i,t-1}(m,n) & c\_{i,t-1}(m,n) \ge \theta\_{\text{high}} \\ c\_{j,t-1}(m,n) & c\_{j,t-1}(m,n) \ge \theta\_{\text{high}} \\ c\_{i,t-1}(m,n) & c\_{i,t-1}(m,n) \le \theta\_{\text{low}} \\ c\_{j,t-1}(m,n) & c\_{j,t-1}(m,n) \le \theta\_{\text{low}} \\ \frac{qp\_i c\_{i,t-1}(m,n) + (1-\varphi)\rho\_j c\_{j,t-1}(m,n)}{\rho\_i + \rho\_j} & o.w. \end{cases} \tag{1}$$

$$b\_{i,t}(m,n) = \frac{q\rho\_i b\_{i,t-1}(m,n) + (1-\varrho)\rho\_j b\_{j,t-1}(m,n)}{\rho\_i + \rho\_j} \tag{2}$$

where *bx*,*t*(*m*, *n*) and *cx*,*t*(*m*, *n*) are the cells in the belief and confidence maps of agent *x* at the time *t*. *θ*s represent the higher and lower bounds for the higher priority information stored in the confidence map based on human operator's influence. In each simulation step, the confidence map ages with the factor *α* < 1 to account for information quality decay. For a while, the operator's influence is locked and will circulate among the swarm until the quality decay brings the values back to the modifiable range of {*θlow*, *θhigh*}. The idea behind this locking mechanism is to overrule the swarm's decision and reserve a larger impact for the operator's commands. Each agent carries a reliability factor (*ρ*) that indicates to which extent its passing information is reliable. This can depend on the hardware capacity, battery level, authority to influence other agents, etc. *ρ* can be extracted from the incoming messages but it may pose a potential security threat if additional security measures are not put in place. Alternatively, the receiving agent can keep a list of *ρ* values and internally decide on whether an incoming message is from a reliable UAV or not (decreases scalability). *ϕ* determines how much each agent is open for influence from other agents. In our experiments, for simplicity, all agents have *ϕ* = 0.5 and *ρ* = 1.

#### *3.4. Swarm's Path Planning*

The operator can create an artificial potential field where high confidence is repulsive and low confidence is attractive. Using potential field-based algorithms to repel UAVs from each other and from obstacles while being attracted to the target zones was studied before [29,30]. As opposed to these studies, we use one interface for state estimation, visualization, and indirect parameter setting with proximal interactions.

The swarm continuously follows the downhill gradient of the potential calculated by applying Sobel filter on the confidence map [31]. UAVs move towards the least confidence area and, therefore, try to maximize their confidence about the environment.

The confidence of the boundaries is set to 1 to repel the UAVs. This also prevents them from getting locked in clusters around the corners or near the boundaries and keeps the UAVs inside the mission zone. The two forces applied to the UAVs are the local ( −→*fl*,*i*(*t*) = ∇*local*(*ci*,*t*)) and global gradients (−→*fg*,*i*(*t*) = ∇*global*(*ci*,*t*)) of their own confidence map. The total force is given by

$$
\overrightarrow{f\_i}(t) = \omega \overrightarrow{f\_{l,i}}(t) + (1 - \omega) \overrightarrow{f\_{\otimes,i}}(t) \; : \; \overrightarrow{f} \in \mathbb{R}^2,\tag{3}
$$

where −→*fi* determines the motion of the UAV *i* and *ω* specifies the weighting factor between local and global forces.

By defining the dynamics of each UAV as given in Equation (3), the mobility of the swarm can be controlled by adjusting the weight coefficient *ω*. Higher *ω* pushes the UAVs to explore their local neighborhood and stay in a small exploration range, while lower *ω* prioritizes the global certainty and the entire swarm moves towards the common targets. When the UAVs prefer to stay in their local neighborhood, their belief map will only be

precise about the area around them and the UAVs across the swarm will have diverse belief and confidence maps that might mislead the operator. On the contrary, lower *ω* may lead to a high congestion around the common targets. In this case, UAVs perform a flocking behavior that decreases the swarm utilization as all UAVs will have common targets and will not distributively explore the mission zone. Assigning different *ω* values to the swarm may prove useful for developing a diversity in the swarm behavior for exploration and exploitation. In this paper, *ω* = 0.99 and fixed for all experiments. The UAVs move in a 2D space and all forces are applied on a plane with constant altitude and parallel to the *xy*-plane.

#### **4. Simulation Platform**

There are several multi-UAV/swarm simulation environments based on commercial (e.g., MATLAB [32], X-Plane flight simulator [33]) and non-commercial platforms (e.g., ARGoS [34]). We designed our own simulation environment based on an open-source video game library, Arcade [35], to be able to freely modify the simulator and also use the high usability characteristic of a game environment. The parameters are optimized to run on low computational power machines. For our experiments, we used a local machine with a 1.4 GHz Intel Core i5 processor and 16 GB 2133 MHz of memory. The images taken from the mission zone have the resolution of 600 × 600 pixels. We use the term pixels in images or cells in matrices to refer to the same concept of information stored in the highor low-resolution images taken from the mission zone. The UAVs decrease the resolution of the images to 40 × 40 pixels to speed up the computation and decrease the load on the communication channels.

#### *Belief and Confidence Maps Setup*

The belief and confidence maps are initialized to zero. As the UAVs move, they observe a radius of 2 pixels from their current position (*r* = 2 px in belief and confidence maps and *r* = 30 px in high resolution) and communicate within a radius of 4 pixels (*r* = 60 px in high resolution). The confidence of radius 2 px (a disk with a diameter of 5 pixels) is set to one. The belief map gets updated based on the observed images. If there is a disaster or an operator within the vision range (*r*), the corresponding cells of the map will be set to one. Changes in the environment may make the stored information obsolete. Therefore, we associate time to the confidence by multiplying the confidence matrix with the decay factor (*α* = 0.0001) at each simulation step. The parameters can be tuned based on the experiments and the domain of application. For instance, a lower decay factor is suitable for dynamic environments as UAVs quickly lose their confidence and frequently need to explore the areas again. Here, we chose our parameters based on empirical observations for proving the concept. Cells with higher confidence clearly have greater impacts on the neighbors. As the UAVs have limited sensor range, the communication enables them to exchange information and broaden their perception to the boundaries of the complete zone.

Another advantage of using confidence maps as a basis for motion pattern is collision avoidance, as the attraction/repulsion mechanism indirectly results into a motion with low collision rates, assuming that there is no communication failure. Recently visited cells have high confidence values and as neighboring UAVs approach each other and share their confidence maps, they repel from the cells that their neighbor recently visited and collisions will be avoided. This also leads to spatial division of the mission zone between the neighboring UAVs (indirect coordination) as the UAVs prefer not to use the same paths that their neighbors recently took.

#### **5. Empirical Evaluation**

We designed four experiments to examine our interface and the swarm's behavior with different levels of autonomy. First, the swarm's performance in exploring the mission zone with full autonomy—no human intervention—is examined (Experiment I). We evaluate how the swarm reacts to dynamic environments with evolving disasters in Experiment II. In Experiment III, the level of autonomy decreases and we measure the swarm's response to human interactions. Finally, we perform a separate interactive user study to evaluate our interface in Experiment IV.

We ran 15 simulation runs with 1000 simulation steps and for each experiment the mission zone consisted of a disaster area, a human operator, and 15 UAVs that are positioned randomly at start of each trial. The swarm is examined with three different locations of the disaster area to make sure that the results are not biased to a certain setup.

#### *5.1. Experiment I: Autonomous Exploration*

In the first experiment, the operator monitors the swarm while the UAVs explore the area, gain confidence over the entire mission zone, build a belief map, and inform the operator about the situation. We measure the accuracy of the resulting belief map by comparing it to the actual situation.

Confidence maps evolve as the UAVs explore the environment and receive information from their neighbors. This evolution also depends on the decay factor (*α*). In this case, the confidence map does not age fast and the certainty accumulates until the entire area is marked with high confidence.

Figure 5 shows the area coverage accumulated during one simulation run (see Figure 5a) and an averaged coverage for all 15 simulation runs (see Figure 5b). The distribution of the swarm is almost uniform, except near the boundaries (white), where there are denser footprints due to boundary avoidance. Figure 5b also shows a circular area at the right side of the zone (dark blue) with sparser coverage. This area is the communication range of the operator with high confidence values. The reason for having a high confidence in this region is that the confidence maps that the operator receives always have high values around the operator, as this is where the UAVs visited most recently. When an UAV enters this region, it receives an update from the operator and by averaging its confidence with the one from the operator, the UAV notices an area with high confidence values and refuses to explore it. By averaging over all simulation runs we see that this region is less explored.

**Figure 5.** The area coverage with the swarm at the end of a randomly selected run (**a**) and the average over all 15 simulation runs (**b**) in Experiment I. The frequency of distances between the UAVs in autonomous exploration experiment is also shown (**c**) where *d* is width/length of the mission zone.

Figure 5c shows the frequency of distances between the UAVs. The low frequency of distances around zero shows that the collisions are rare. The two minor peaks before and after the global maximum are caused by the clusters of UAVs around the boundaries (white box in Figure 5b).

Figure 6 shows the accuracy of the mapping over time. The error in the belief map is the absolute difference between the UAVs' belief maps and the actual disaster scenario in the mission zone. Using a belief map of size 40 × 40, the theoretical maximum level of error is 1600. Nonetheless, the highest error we see during our experiments is 21 and for visibility we use that as our maximum error in the figures. Similarly, with the human intervention, the confidence can theoretically reach a very high or a very low value, but in our experiments we bound the range to [0, 2000]. Errors in the belief map start at around 14 pixels, which is the number of cells in the belief map that cover the disaster zone and the operator (normalized to 0.6; maximum is 21). The swarm explores the area and gradually ¯ constructs an accurate map and the error in the belief map decreases. The confidence level increases over time and with all the cells in a confidence map of size 40 × 40 set to one, the confidence can rise up to 1600 (normalized to 0.8; maximum is 2000). Figure 6 shows the precision of the belief map that the operator receives (see Figure 6c) and the average belief error among the UAVs (see Figure 6b). We can also observe the growth in the confidence level (see Figure 6d–f. We separately plot the progress in the maps of the swarm and the operator to compare the quality of the swarm's perception and the information that is passed on to the operator (monitoring quality).

**Figure 6.** The precision of mapping is shown in the error level of the belief map. This figure shows the belief error and the confidence level for the autonomous exploration experiment. The top row shows the errors in the belief map and the bottom row depicts the confidence levels over time. (**a**,**d**) are from one simulation run. (**b**,**e**) show the median values over all simulation runs. (**c**,**f**) are the belief error and the confidence level of the operator.

#### *5.2. Experiment II: Response to Evolving Disaster*

In this experiment, we examine the adaptivity of the mapping in a dynamic environment with an evolving disaster. As opposed to Experiment I, here we assume that the disaster zone is not fixed and it can move to another point in the area. The disaster zone moved between 150 < *t* < 350 simulation steps. We observe how this change is reflected in the performance of the swarm and the operator's understanding of the environment. Note that the swarm still has full autonomy and the operator only monitors the swarm. Figure 7a shows the direction of the disaster displacement and Figure 7b shows the belief map when the motion is stopped. The operator's belief map gradually adapts to the new position of the disaster (see Figure 7b,c). Figure 8a,b show the effect of a dynamic environment on the error in the belief of the swarm and the operator over time. The error increases at first as the environment starts to change, but after a while the swarm successfully adapts itself.

(**a**) moving disaster (**b**) before change (**c**) after change

**Figure 7.** In Experiment II, the disaster area moves from point A to B (**a**) and the belief of the swarm gradually adapts to the change (**b**,**c**).

**Figure 8.** Median errors in the belief maps of the swarm (**a**) and the operator (**b**) in Experiment II. The disaster zone moves between 150 and 350 simulation steps (green area) as shown in Figure 7.

This behavior can also be seen in Figure 7b,c by the former positions of the disaster slowly fading away and the new positions appearing in the belief maps. Shortly after, the UAVs successfully map the environment and notify the operator. The results from the evolving disaster experiment show that the swarm is resilient to changes in the environment and the operator's belief map stays updated.

#### *5.3. Experiment III: Human Interaction*

After investigating the swarm's autonomous behavior, we now focus on the human– swarm communication link.

The decision of whether to influence the map or not can entirely affect the autonomy of the swarm (i.e., flexible autonomy). Leaving the confidence map untouched allows a fully autonomous swarm approach while manipulating the entire map would create an almost fully manual control over the swarm's behavior.

In this experiment, we observe the behavior of the swarm with the operator's control and evaluate the task completion. The operator can lead the swarm to explore an area of interest (attraction) or prevent them from flying over a no-fly zone (repulsion) by manipulating its own confidence map and disseminating it to the neighboring UAVs. For example, the operator can send a new confidence map with attraction in the middle of the mission zone to its neighboring UAVs. As the UAVs continuously update their confidence map based on their neighbors, some of them may receive the operator's confidence map and move according to the potential field.

Figure 9 shows the influence of the operator's command on the confidence of an UAV and how the UAV is controlled. Figure 9a illustrates the confidence map that the operator introduces to the swarm to attract them to the middle of the arena. After receiving the command, the UAV in this example computes the mean of its previous confidence map (see Figure 9b) and the command to update its map (see Figure 9c). As a result of the attraction force, the UAVs move towards the area in the middle and this influence diminishes. The UAVs' presence in the middle of the arena can be inferred as the values in the middle of the confidence map increase (see Figure 9d). The operator's communication range determines the controlling impact of the operator on the swarm.

**Figure 9.** An example of the operator's command at *t*0 (**a**), an UAV's confidence map before (**b**) and after receiving the command (**c**), and after swarm's reaction to the influence by moving to the center (**d**) in Experiment III.

Using wider communication ranges, the operator influences more UAVs and therefore the effect is more significant. Figure 10 shows the effect of a human interaction on the confidence level of the swarm. At *t* = 300 the operator disseminates a command once and it takes 100 to 200 simulation steps to significantly influence the entire swarm. In case of attraction, UAVs quickly move to the cells with low confidence and immediately compensate the drop in the certainty level (see the valley in Figure 10a), while in repulsion the UAVs cannot visit the cell to recover from the added value to the confidence as they are repelled from that region. This explains the recovery of the confidence levels in Figure 10a and the bias that stays in the swarm for the rest of the experiment in Figure 10b.

**Figure 10.** Influence of the operator's command at *t* = 300 on the swarm's confidence.

So far, we have covered the temporary attraction and repulsion. We also assume a disaster situation where UAVs must avoid a certain area at all times. An example can be the frequent flying zone of an airport or a no-fly area with high voltage cables. For that, the operator can continuously apply a repulsion force from the prohibited zone. Figure 11a shows the confidence bias that the operator introduces to the swarm. The cells highlighted with red mark the barrier or the prohibited zone. The blue area in Figure 11a can be either empty, which allows the swarm to keep the knowledge about the confidence and only manipulate the values in the interest zone or it can be set to zero, which may gradually decrease the swarm's confidence level on all the other areas. Figure 11b shows the average distribution of the UAVs in all simulation runs as a result of the permanent repulsion at the center. From the low coverage at the center of Figure 11b, it is clear that the operator successfully controlled the swarm to avoid the area in the middle of the mission zone. As the communication range is limited, there is a delay until all UAVs become aware of the new force introduced by the operator. In the meantime, even though the operator command is already issued, some UAVs are unaware and cross the prohibited zone. The ring in the

distribution around the repellent zone is caused by the area with the equilibrium between different repulsion forces.

**Figure 11.** Operator's confidence map for introducing a permanent repulsion (**a**) and the swarm's distribution (**b**).

We increased the task complexity and simulated the influence of five permanent no-fly zones of length L to 1.8 L (imposed restrictions by the operator) in the middle of the arena (see Figure 12a,b). We assume that the messages cannot travel through the obstacles. Figure 12c shows the error in the operator's belief map with and without barriers. As the size of the barrier grows, the precision of the mapping decreases. For 1.8 L, the barrier is too spacious for the swarm to adequately travel around it and keep the operator updated about the situation, and the precision of the belief map decreases (i.e., the belief error does not significantly decline over time). Our results show that the swarm is able to follow a wide range of control commands and continue to offer a precise mapping of the environment.

**Figure 12.** Error in the operator's belief map for different sizes of the repulsion zones.

#### *5.4. Experiment IV: Human–Swarm Interaction User Study*

In this final experiment, we performed an additional technology-probe user study. We compared the situation awareness of a fully autonomous swarm, a human–swarm system, where only the target is specified by the operator (Task I). Despite the preference of the users, discussed in Section 2, we conducted another experiment where the operator controls individual drones (Task II). We posted a call for participation in a small online community. Fifteen computer scientists with BSc and MSc degrees in Computer Science or Engineering were recruited. The users were given a remote access to the machine with the simulation environment. We briefed the users about the goal of the experiment in a 10 to 15-minute video call before the experiment. We made another call for 10–15 minutes to collect their feedback after they performed the experiment. Users could observe the belief and confidence maps. They were asked to predict the significance of the areas that needed to be further explored. The number of clicks on the confidence map (number of commands) was left as a choice to the users. In Task II, the users had access to the list of UAVs. They

could select the UAVs and assign a target to it by clicking on a cell of the confidence map. The Black line-point in Figure 13 shows the median belief error using this approach.

For Task I, we asked the users to operate the swarm using our interface. They needed to only click on a cell in the confidence map that they wanted to be explored. They did not need to select single UAVs themselves. The blue points in Figure 13 show the median belief error using our interface. In both experiments, the disaster moves in a random direction and the human operator waits for a while to let the swarm explore the area (neglect benevolence [36]). This explains the overlapping performance between all experiments (autonomous exploration, operator controlling a single UAV, and operator controlling the swarm) at the early stage of the experiment. Once the new location of the disaster emerges, we observe that the operator frequently commands the swarm to narrow the exploration area to the points with higher belief values. As shown in Figure 13, the accuracy with human intervention is better than the results from the fully autonomous exploration. We also observe that the performance with our interface (Task I) and the individual UAV control (Task II) is almost identical. However, the cognitive workload of having to select individual UAVs first and then selecting the target area is much higher than only picking the target areas. For larger swarm sizes, controlling individual UAVs is not feasible. On the contrary, our interface is scalable. The operator can select areas of interest, regardless of the swarm size and the swarm will self-organize to assign individual target points.

**Figure 13.** Median errors in the operator's belief map with autonomous exploration (Experiment II, red line), human control of individual UAVs (Experiment IV: Task II, black line-point), and human– swarm teaming with our interface (Experiment IV: Task I, blue points).

#### **6. Discussion and Future Work**

Establishing a reliable interaction between human operators and a swarm comes with plenty of issues. In this paper, we focused on the issue of situation awareness with a swarm and an operator when all interactions are local without imposing a high cognitive complexity. Our results show that the same map can be used for both visualization and control of the swarm. The operator has to interact with the swarm via this map and there is no need for one-to-one interaction with each member of the swarm, which can be overwhelming for a human operator. We made a few assumptions to prove the concept. We have not performed a parameter sensitivity analysis and our parameters were chosen experimentally. For instance, our swarm size was fixed when studies show that the swarm density has an effect on performance [37]. Another point of discussion is the scalability of our approach. Our method relies on local interactions and allows for scalable human– swarm interaction in mapping an unexplored environment. The only limit for scalability is that as the exploration area grows, the mapping resolution decreases. A potential solution for this can be registering local images and constructing a global image at the human operator's side.

There are several directions that this research can be extended: (1) We plan to investigate the effect of multiple dynamic disaster areas and human operators with collaborative/competitive management strategies. Besides the competing operator commands, there may also be intruder attacks by other UAVs or operators that need to be detected

and eliminated from the decision-making process; (2) The effect of noise (communication, positioning, and sensing) on the performance of the swarm and detecting malfunctioning/lost UAVs need to be investigated as well; (3) Speed versus accuracy can be studied with several messaging qualities. (4) One of our next steps will be to co-create an interaction interface for human–swarm teams with experts from the industry and perform a more comprehensive user study; (5) Variations that may be induced by the operator bias in the human-in-the-loop experimentation; (6) In some use-cases it might prove useful to direct a subset of the swarm by a more complex communication protocol that allows access to certain members of the swarm without affecting the entire swarm; (7) We also plan to address some of these challenges and implement our approach on physical UAVs and evaluate the performance of the swarm in real world applications.

#### **7. Conclusions**

A user study with 100 participants showed that users prefer to have a global overview of the swarm's coverage instead of monitoring individual agents, especially for large swarm sizes and when the time is critical. We showed that a human–swarm interaction based on a global view of the swarm with a human-in-the-loop decision-making can assist the swarm to map a dynamic environment. Using our approach, human operators do not require a fully-connected communication network with all UAVs to be able to monitor the swarm and control them. In our experiments, an operator controlled a group of simulated UAVs and successfully explored the environment even when it was dynamic. Our approach is easily reproducible and can serve as a basis for monitoring and controlling the swarm in various application domains such as agricultural missions, space exploration, etc.

**Author Contributions:** Conceptualization, M.D.S., D.T. and S.D.R.; methodology, M.D.S., D.T. and S.D.R.; software, M.D.S. and J.G.; validation, M.D.S. and J.C.; formal analysis, M.D.S. and J.C.; investigation, M.D.S. and J.C.; resources, M.D.S.; data curation, M.D.S. and J.C.; writing—original draft preparation, M.D.S., J.C., J.G., D.T., and S.D.R.; writing—review and editing, M.D.S. and J.C.; visualization, M.D.S. and J.C.; supervision, D.T. and S.D.R.; project administration, S.D.R.; funding acquisition, M.D.S. and S.D.R.; All authors have read and agreed to the published version of the manuscript.

**Funding:**We acknowledge funding from the UKRI Trustworthy Autonomous Systems Hub (EP/V00784X/1).

**Institutional Review Board Statement:** The study was conducted according to the Data Protection Act 1998, and approved by the Ethics Committee of University of Southampton (Ethics and Research Governance Online number 66360.A1 on 23 August 2021).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** Not applicable.

**Acknowledgments:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


**3. Legal framework and policies**

Tanzania power sector has undergone through different turbulent periods, changes and reforms since the country attained its independence in 1961. Most of the changes and reforms have sent positive signals to those who are interested in developing or starting electricity project in the country. The changes and reforms include laying down a National Energy policy (NEP), Electricity Industry Policy and The electricity act of 2008, and guidelines for sustainable liquid biofuel development. Some of the legal framework and policies are elaborated in the following section.

#### **3.1 National energy policy (NEP) -2003**

The first national energy policy (NEP) for the Country was formulated in 1992. Since then the energy sector has undergone a number of changes, necessitating adjustments to the initial policy. These changes include change in the way the role of the government from service provider to service facilitator, liberalization of the market and encouragement of private sector investment. With these changes, the energy policy of 1992 was replaced in 2003.

The objective of the 2003 NEP is to ensure availability of reliable and affordable energy supply and use in a rational and sustainable manner in order to support national development goals. The National Energy Policy of 2003 aims to establish energy production, procurement, transmission, distribution and end-user systems in an efficient, environmentally sound, sustainable and gender-sensitized manner.

Key objectives of the 2003 NEP regarding to Renewable Technologies (RT) and services include:

The policy encourages public and private partnerships to invest in the provision of energy services. It also seeks to promote private initiatives at all levels and stresses the need to make local and foreign investors aware of the potential of the Tanzanian energy sector. To implement the policy several laws have been enacted, among them are:


**Abstract**

The chapter presents an analysis of selected magnesium alloys as structural materials to be used in production of parts as well as their technological parameters in some manufacturing processes: metal forming and joining. Taking into account the analysis of microstructure and mechanical properties of conventional and new magnesium alloys and requirements of their possible applications (aviation, automotive, sport, etc.), the study of forming parts/products based on description of plastic formability of magnesium alloys in the processes of bulk metal forming (forging, extrusion, KOBO extrusion, rolling) and joining (friction stir welding) has been presented. Upsetting test, backward extrusion, and KOBO extrusion of complex cross-sectional profiles and forging process were conducted using magnesium alloys such as AZ31, AZ61, AZ80, WE 43, and Mg alloy with Li for the production of thin-walled profiles and forged parts. The range of temperatures and extrusion rate for manufacturing of these profiles were determined. Tests also covered the analysis of microstructure of Mg alloys in the initial state as well as after the extrusion process. The recommendations for realization of metal forming and joining processes of selected magnesium alloys have been presented.

**Keywords:** magnesium alloys, plastic formability, forging, extrusion, KOBO extrusion, rolling, joining, friction stir welding


**Author details**

Ewa Dudzińska Medical University of Lublin, Poland

\*Address all correspondence to<EMAIL_ADDRESS>
© 2020 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

**49**

*Single-Nucleotide Polymorphisms in Inflammatory Bowel Disease*

Medicine. 2017;**33**(1):82-88. DOI:

[9] Lee A, Kanuri N, Zhang Y, et al. IDO1 and IDO2 non-synonymous gene variants: Correlation with crohn's disease risk and clinical phenotype.

[10] Kennedy NA, Lamb CA, Berry SH, Walker AW, Mansfield J, Parkes M, et al. The impact of NOD2 variants on fecal microbiota in Crohn's disease and controls without gastrointestinal disease. Inflammatory Bowel Diseases. 2018;**24**(3):583-592. DOI: 10.1093/ibd/

[11] Meddens CA, van der List ACJ, Nieuwenhuis EES, Mokry M. Noncoding DNA in IBD: From sequence variation in DNA regulatory elements to novel therapeutic potential. Gut.

Kenison JE, Yeste A, Murugaiyan G, Sambanthamoorthy S, et al. AHR activation is protective against colitis driven by T cells in humanized mice. Cell Reports. 2016;**17**(5):1318-1329. DOI:

10.1159/000458006

PLoS One. 2014;**9**:115848

izx061

2019;**68**:928-941

[12] Goettel JA, Gandhi R,

10.1016/j.celrep.2016.09.082

[13] Bosca-Watts MM, Tosca J,

or no bug. World Journal of Gastrointestinal Pathophysiology.

2015;**6**(1):1-12

10.1155/2018/6914346

Anton R, Mora M, Minguez M, Mora F. Pathogenesis of Crohn's disease: Bug

[14] Dudzińska E, Gryzinska M, Kocki J. Single nucleotide polymorphisms in selected genes in inflammatory bowel disease. BioMed Research

International. 2018;**2018**:6914346. DOI:

[15] Iida T, Onodera K, Nakase H. Role of autophagy in the pathogenesis of inflammatory bowel disease. World Journal of Gastroenterology.

*DOI: http://dx.doi.org/10.5772/intechopen.92051*

[1] Vezza T, Rodríguez-Nogales A, Algieri F, et al. Flavonoids in

inflammatory bowel disease: A review.

[2] Etienne-Mesmin L, Chassaing B, Gewirtz AT. Tryptophan: A gut microbiota-derived metabolites

[3] Lan D, Niu J, Miao J, Dong X, Wang H, Yang G, et al. Expression of guanylate cyclase-C, guanylin, and uroguanylin is downregulated proportionally to the ulcerative colitis disease activity index. Scientific Reports. 2016;**6**:25034. DOI: 10.1038/

[4] Rubin DT, Ananthakrishnan AN, Siegel CA, Sauer BG, Long MD. ACG Clinical Guideline: Ulcerative colitis in adults. The American Journal of Gastroenterology. 2019;**114**(3):384-413. DOI: 10.14309/ajg.0000000000000152

[5] Agouridis AP, Elisaf M, Milionis HJ. An overview of lipid abnormalities in patients with inflammatory bowel disease. Annals of Gastroenterology.

[6] Mahdi BM. Role of HLA typing on Crohn's disease pathogenesis. Annals of Medicine and Surgery. 2015;**4**:248-253

Gil-Kulik P, Chomik P, Świstowska M, Gryzińska M, et al. Imbalance of controlled death in peripheral blood lymphocytes in Crohn's disease and ulcerative colitis. Medicina (Kaunas, Lithuania). 2019;**55**(6):231. DOI: 10.3390/medicina55060231

[8] Atreya R, Neurath MF. Current and future targets for mucosal healing in inflammatory bowel disease. Visceral

[7] Dudzińska E, Szymona K,

regulating inflammation. World Journal of Gastrointestinal Pharmacology and Therapeutics. 2017;**8**(1):7-9. DOI:

**References**

Nutrients. 2016;**8**:211

10.4292/wjgpt.v8.i1.7

srep25034

2011;**24**(3):181-187

*Single-Nucleotide Polymorphisms in Inflammatory Bowel Disease DOI: http://dx.doi.org/10.5772/intechopen.92051*


**MELANOMA – CURRENT CLINICAL MANAGEMENT AND FUTURE THERAPEUTICS**

Edited by **Mandi Murph**

#### **Melanoma - Current Clinical Management and Future Therapeutics**

http://dx.doi.org/10.5772/58516 Edited by Mandi Murph

#### **Contributors**

Monica Neagu, Carolina Constantin, Estelle Leclerc, Carlo Riccardo Rossi, John D'Orazio, Erin Wolf Horrell, Kalin Wilson, Paola Savoia, Paolo Fava, Isabel Pires, Luis Resende, Felisbina Luisa Queiroga, Joana Moreira, Justina Prada, Rolland Gyulai, Zsuzsanna Lengyel, Zsolt Kádár, Mandi Murph, Duane Miller, Yasuhiro Nakamura, Miriam Jasiulionis, Ana Carolina Monteiro, Mariana Mariana Toricelli, Hinrich Abken, Jennifer Makalowski, Norma E. Herrera, Corina-Daniela Ene, Ilinca Nicolae

#### **© The Editor(s) and the Author(s) 2015**

The moral rights of the and the author(s) have been asserted.

All rights to the book as a whole are reserved by INTECH. The book as a whole (compilation) cannot be reproduced, distributed or used for commercial or non-commercial purposes without INTECH's written permission. Enquiries concerning the use of the book should be directed to INTECH rights and permissions department (permissions@intechopen.com).

Violations are liable to prosecution under the governing Copyright Law.

Individual chapters of this publication are distributed under the terms of the Creative Commons Attribution 3.0 Unported License which permits commercial use, distribution and reproduction of the individual chapters, provided the original author(s) and source publication are appropriately acknowledged. If so indicated, certain images may not be included under the Creative Commons license. In such cases users will need to obtain permission from the license holder to reproduce the material. More details and guidelines concerning content reuse and adaptation can be foundat http://www.intechopen.com/copyright-policy.html.

#### **Notice**

Statements and opinions expressed in the chapters are these of the individual contributors and not necessarily those of the editors or publisher. No responsibility is accepted for the accuracy of information contained in the published chapters. The publisher assumes no responsibility for any damage or injury to persons or property arising out of the use of any materials, instructions, methods or ideas contained in the book.

First published in Croatia, 2015 by INTECH d.o.o. eBook (PDF) Published by IN TECH d.o.o. Place and year of publication of eBook (PDF): Rijeka, 2019. IntechOpen is the global imprint of IN TECH d.o.o. Printed in Croatia

Legal deposit, Croatia: National and University Library in Zagreb

Additional hard and PDF copies can be obtained from<EMAIL_ADDRESS>
Melanoma - Current Clinical Management and Future Therapeutics Edited by Mandi Murph p. cm. ISBN 978-953-51-2036-0 eBook (PDF) ISBN 978-953-51-7237-6


**Author details**

Anthony Luder1,2

1 Ziv Medical Centre, Safed, Israel

2 Azrieli Faculty of Medicine, Bar Ilan University, Safed, Israel

\*Address all correspondence to<EMAIL_ADDRESS>
© 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

**277**

*Treating the Enemy: Victims of the Syrian Civil War in Israel*

deliberately targeting ambulances in civil war. Available from: https:// www.independent.co.uk/news/ world/middle-east/syria-war-russiahealthcare-ambulances-civil-weaponsarmy-government-a8654841.html

[8] Zarka S, Lerner A. Complicated War Trauma and Care of the Wounded: The Israeli Experience in Medical Care and Humanitarian Support of Syrian Refugees. New York: Springer; 2018

[9] Ziv Medical Center treats Syrian casualties. Available from: https:// youtu.be/RBXZx3A79Tg [Accessed: 09

[10] Young SS, Denise C, Lewis DC, Gilbey P, Eisenman A, et al. Conflict and care: Israeli healthcare; providers and Syrian patients and caregivers in Israel. Global Qualitative Nursing

[11] Osmo L. For Syrian refugees in Greece, Israel is no longer the enemy. Available from: https://www.ynetnews. com/articles/0,7340,L-5473381,00.html

[Accessed: 04 May 2019]

June 2019]

Research. 2016;**3**:1-15

[Accessed: 04 May 2019]

*DOI: http://dx.doi.org/10.5772/intechopen.87936*

[1] Fleck D. The Handbook of International Humanitarian Law. Second ed. USA: Oxford University Press; 2008. ISBN 0-19-923250-4

[Accessed: 04 May 2019]

[2] Elise B, Widney B. Doctors in the crosshairs: Four years of attacks on health care in Syria (PDF) (Report). Physicians for Human Rights. Available from: https://s3.amazonaws.com/PHR\_ Reports/doctors-in-the-crosshairs.pdf

[3] Syrian medical voices from the ground: The ordeal of Syria's healthcare professionals (PDF) (Report). The Center for Public Health and Human Rights of the Johns Hopkins Bloomberg

School of Public Health and the Syrian American Medical Society; February 2015. Available from: https:// www.sams-usa.net/wp-content/ uploads/2016/09/Syrian-Medical-Voices-from-the-Ground\_F.pdf [Accessed: 04 May 2019]

[4] Spagat M. Attacks on medical workers in Syria: Implications for conflict research. PLoS Medicine.

[5] Tammy L, Lisa C. Chicago doctors risk arrest, torture to aid dire medical care void in Syria. NBC Chicago. Available from: https:// www.nbcchicago.com/investigations/

Chicago-Doctors-Syrian-War-

[6] Physicians for Human Rights. The Syrian conflict: Eight years of devastation and destruction of the health system. Available from: https:// phr.org/resources/the-syrian-conflicteight-years-of-devastation-anddestruction-of-the-health-system/

[Accessed: 04 May 2019]

[7] Hall R. Syrian and Russian forces 'weaponised healthcare' by

Hospital-295954681.html [Accessed: 04

2018;**15**(4):e1002560

May 2019]

**References**

*Treating the Enemy: Victims of the Syrian Civil War in Israel DOI: http://dx.doi.org/10.5772/intechopen.87936*


*3.1. Assumptions*

To apply the proposed approach, the content of every image in the UBEAR dataset needed to be represented as a graph. For that purpose, first, images and binary masks with precise ear localization were scaled down to have only 0.25 of their original size. Next, superpixel detection was performed with the use of the SLIC algorithm [26]. The expected number of superpixels, which is a parameter of SLIC algorithm, was determined by their expected average area *A*. Two possible configurations were considered with *A* = 256 and *A* = 128. They led to around 300 and 600 superpixels per image respectively.

Having superpixels generated, two graphs were created: input graph and expected, output graph. Nodes of both graphs corresponded to superpixels. In the input graph, the feature vector assigned to node contained the average intensity of image pixels covered by a given superpixel (Figure 3a,b). In the expected, output graph, it was the average intensity of pixels taken from binary masks multiplied by scaling constant *W* > 0. It should be noted that in the latter case, values assigned to nodes need not be equal either to zero or *W* as the borders of superpixels need not coincide with the borders of the ear region (Figure 3c,d). Two possible values of scaling constant were considered. These were *W* = 1 and *W* = 100. This constant was introduced based on our earlier experience with classic CNN applications. Such a procedure allowed avoiding, during network training, local minima with all responses equal to zero.

Nodes in the considered graphs must be connected with directed edges. To determine which airs of nodes should be connected, first, the adjacency of all superpixels was examined. Two nodes were connected with an edge if there existed a path (path in this context means a sequence of superpixels) of length shorter than or equal to a given number *D*, connecting corresponding superpixels. Here, also two configurations were analyzed, where *D* = 1 and *D* = 2 (Figure 8). Selecting a higher value of *D*, we should be able to increase the size of the visual field, i.e., increase the number of input nodes, which influences the single output node. Loops connecting nodes were allowed because thanks to that the node could express its influence on the output assigned to this node.

After a series of initial trials, a network architecture with *L* = 4 layers *φ* was found to be the optimal one. The number of filter groups in those layers, and hence the number of output graphs, was equal to 20, 10, 5, and 1, respectively. The number of filters in the given group corresponded to the number of layer inputs. In the first layer, it was one, and in the subsequent layers, it depended on the output of the previous layer. In all layers except the last one, the ReLU activation function was used. The last layer had an identity activation function assigned. The number of GMM components in all filters *ϕ* was equal to *J* = 4. As before, while training, MSE loss, as well as the Adam optimizer were used. This time, however, a smaller learning rate, equal to 10−4, was considered. For weight initialization, the Glorot scheme was used [40].

To detect ears based on the output of the network, simply the node (superpixel) with the highest response was sought (Figure 9a,b). In order to evaluate if this detection was correct, it was checked whether the superpixel (its centroid) lied inside the bounding box surrounding the ear region. That rectangle was found using the original binary masks provided in the UBEAR dataset and was slightly enlarged to take into account the size of the superpixels (Figure 9c,d).

In all the experiments, images from the UBEAR dataset were split into three sets: training D*TR*, validation D*VA*, and test set D*TE*. The split was made made based on person identifiers, i.e., images of the same person were assigned always to the same set. This should allow checking if the trained models were able to generalize the acquired knowledge and respond correctly for new people. The validation set was used to prevent overfitting by the selection of the optimal model from among models created in the training phase. The number of people in the discussed sets was equal to 75, 25, and 26, respectively. Only left ears were considered since the proposed approach should work for right ears in the same way. What is more, mirror transformation of GMM filters should also allow applying the network trained with left ears to work for right ears, as well.

**Figure 8.** Graphs generated for images shown in Figure 3 for different superpixels' number and different node neighborhoods (parameters *A* and *D*, respectively): (**a**,**b**) full graph with all edges; (**c**,**e**) superpixels with the local neighborhood of the selected node; (**d**,**f**) selected graph node with its neighborhood. As the image scale was preserved, it can be observed that when *A* was smaller, the smaller image region was processed by CNN. Consequently, not only *D* but also *A* influenced the size of the effective visual field.

**Figure 9.** Detection results for images shown in Figure 3: (**a**,**b**) network output (it was scaled back using *W* and cut to [0, 255] interval); (**c**,**d**) detection visualization (the green rectangle represents the expected bounding box, and the red dot indicates selected superpixel; the red line shows the orientation of filters in the network, and the vertical line corresponds to basic orientation).

## *3.2. Experiment I*

In the first experiment, the CNN network was trained on a UBEAR subset D*TR M* containing only heads in their standard orientation (pose M in the UBEAR dataset). Every combination of parameters *A*, *D*, and *W* was tested to select the optimal one. The obtained results and cardinalities of the considered training, validation, and test sets are gathered in Table 2. In all cases, results were satisfactory with the correct detection rate bigger than 90%. It is worth noting that in three cases, the detection accuracy for the training set was equal to 100%. This seemed, however, to be slightly overfitted, and as the best configuration, the one with *A* = 256, *D* = 2, and *W* = 100 was indicated.

The closer analysis of the results revealed that, surprisingly, representation with *A* = 256 was not worse than representation with a bigger number of superpixels obtained when *A* = 128. It could be expected that in the latter case, when more details are given, the accuracy would increase. The explanation can be the fact that in both cases, the same network architecture was used, and consequently, for smaller *A*, the effective visual field was also smaller. What is more, in both cases, the same number 2000 of training iterations was used, and perhaps, more details required longer training. Nevertheless, since for *A* = 256, the results were satisfactory and, thanks to the simpler representation, graph processing was faster, this seemed to be a reasonable choice for the discussed problem. In the case of other parameters, the configurations with *D* = 2 and *W* = 100 seemed to lead to models with better generalization abilities. For them, the detection accuracy was higher when validation and test sets were considered. Those observations were also confirmed by the training characteristic depicted in Figure 10. For optimal values of the parameters (Figure 10a), the best model, the one with the smallest validation error, could be easily selected in the early stage of training. In other cases (Figure 10b), both errors seemed to decrease slowly, and maybe, further training could provide a better solution. Not without significance is also the random initialization of network weights.

**Table 2.** The ear detection accuracy of the networks trained using the D*TR M* set for different combinations of parameters *A*, *D*, and *W*. Additionally, the last column contains the cardinality of every considered set. It is worth noticing that networks trained using only samples for basic head orientation (pose M) can successfully detect ear not only for other people, but also for different orientations (poses U and D). Naturally, in the latter case, detection accuracy was significantly smaller.

In Figure 11, additional samples of good and wrong detections of the optimal network are presented. It can be observed that correct detections were possible in different illumination conditions and with different background, as well as in situations when there were additional objects located in the ear neighborhood. The typical reasons for the detection mistakes were: questionable annotations (head orientation slightly different than expected), position of the ear close at the image border, and highly occluded ears.

To have a better insight into the process of single graph processing, also the selected outputs of convolutional layers *φ* are presented in Figure 12. In classic CNNs the first layers are usually responsible for the detection of some local image characteristics. Although here, the interpretation

was not that obvious, that kind of behavior to a certain extent can also be observed. In Figure 12a, for example, the detection of vertical edges seemed to take place.

**Figure 10.** Training characteristics of two models. The plots present errors for training D*TR M* and validation D*VA M* sets that were calculated every 50 epochs. On the left, the training run for the best parameter combination is depicted. It can be observed that to select the optimal network, the model generated after 600 epochs should be chosen. On the right, another combination of parameters was used. This time, however, no epoch can be indicated where model overfitting seemed to take place. Probably, the training should be continued further.

**Figure 11.** Examples of detections for the best, trained network for images in D*VA M* and D*TE M* : (**a**–**c**) correct detections; (**d**–**f**) wrong detections. The network is able to respond correctly in different illumination conditions and for different backgrounds. Problems can be observed when ears are not fully visible (image border or occlusion) and when the head orientation is different than expected (wrong annotation). The convention of the result presentation was described in Figure 9's caption.

**Figure 12.** Selected, raw outputs of convolutional layers for the image shown in Figure 3a and the optimal network. In the first two layers (*φ*1 and *φ*2), the person outlines can be still observed, so probably some local image characteristics are extracted here. This behavior is typical also for classic CNNs. In the final layer *φ*4, the output allows finding ear position. For visualization purposes, every output was scaled separately (they cannot be compared). Red color denotes negative and green positive values.