*Editorial* **DNA Vaccine Development at Pre- and Post-Operation Warp Speed**

**Karl Ljungberg 1 and Maria Isaguliants 2,3,4,5,\***

Received: 17 November 2020; Accepted: 17 November 2020; Published: 4 December 2020

DNA is a rapidly developing vaccine platform for combatting cancer, infectious and noninfectious diseases. Plasmid DNA used as immunogens encode proteins to be synthesized in the cells of the vaccine recipients. Introduction of DNA vaccines into the host induces antibody and cellular responses against the encoded protein. In this way, the induction of immune response mimics the events occurring during natural infection with an intracellular pathogen. There are a few distinct ways in which the vaccine antigen can be processed and presented, which shape the resulting immune response and which can be manipulated. Characteristically, the antigen synthesized within the host cell is processed by the proteasome, loaded onto, and presented on MHC Class I molecules. Processing can be re-routed to the lysosome, or the immunogen can be secreted for further presentation on MHC Class II. Vaccine efficacy is also highly dependent on DNA delivery. DNA immunogens are often administered by intramuscular or intradermal injections, but the immune response can be significantly enhanced by subsequent electroporation of the injection site, which enhances the delivery up to 1000-fold, thereby facilitating dose sparing. Other techniques may also be employed. For instance, noninvasive introduction by biolistic devices such as gene guns and biojectors, skin applications with plasters and microneedles/chips, sonication, magnetofection, and even tattooing has been shown to improve the efficacy of delivery. The debate regarding the pros and cons of different routes of delivery is intense but the answer to which route of administration is better for a DNA vaccine is too complex to give a straightforward answer. It depends on multiple factors such as the choice of antigen and vector, expressing tissues and cells, and the disease it targets. A number of studies have compared the effect of delivery methods on the level of immunogen expression, and the magnitude and specificity of the resulting immune response. According to some, the delivery route determines the immunogenic performance; according to others, it can modulate the level of response but not its specificity or polarity. All in all, research on the optimization of DNA vaccine design, delivery, and immunogenic performance has led to a marked increase in their efficacy in larger species and humans.

This Special Issue describes the continuing efforts to increase the potency of DNA vaccines by manipulating plasmid DNA, adding adjuvants or immunomodulators (also delivered in the form of plasmid DNA), and by using a wide panel of novel delivery systems. These efforts are described in seven experimental papers and four reviews.

When we launched the "Advances in DNA Vaccines" Special Issue, little did we know that the world would face a new infectious disease threat just months after the submission deadline, a threat which would change the entire game plan for vaccine development. The current SARS-CoV-2 pandemic has forced vaccinology into a new intense phase of development, transformation, and innovation. Hit by the COVID-19 epidemics, the world has come to realize the importance of efficient vaccines against

viral infections. It is comforting that the contributing authors were working hard in this field before the epidemic broke out: all publications of this issue focus on the development of vaccines against viral infections, such as HIV by Kilpelainen et al. [1], Louis et al. [2], and Akulova et al. [3]; human hepatitis C virus by Masalova et al. [4]; Ebola virus by Bazhan et al. [5]; Zika virus [2]; influenza by Hinkula et al. [6] and Louis et al. [2]; and Epstein–Barr virus by Wojtak et al. [7].

The majority of studies presented in this issue used different forms of plasmid DNA: genes optimized for expression and/or consensus immunogens [2,3,7], polyepitope constructs [5], or plasmids with viral enhancers increasing protein expression and thus allowing one to reduce the DNA dose (Chapman & Rybicki [8]). A stillmore efficientway to ensure high-level expression of theimmunogenis to use RNA/DNA layered alphavirus vectors, which provide a superior expression of immunogens in comparison with conventional plasmid DNA technology (Lundstrom [9]). Besides, immunization with alphavirus DNA vectors elicits an immune response compared with the conventional plasmid at a 1000-fold lower DNA dose, allowing for considerable vaccine sparing [9].

In most of the studies presented in this Special Issue, plasmid DNA was introduced by intramuscular or intradermal injections, with or without subsequent electroporation. Two more sophisticated ways of indirect delivery of DNA encoding immunogens presented include immunization with recombinant BCG as a vehicle to express HIV-1 and SIV antigens [1] and by mesenchymal stem cells made to express nonstructural proteins of hepatitis C virus [4]. In both cases, delivery using bacterial or eukaryotic cells as a vehicle allowed the researchers to significantly increase the cellular response against viral antigens as compared with immunization with naked plasmid DNA.

A recent development is the use of plasmid DNA to encode an adjuvant; for instance, a cytokine or another noncytokine immunomodulator with the power to enhance and shape the immune response. In this Special Issue, we see demonstrations of the efficient use of an immunomodulatory plasmid DNA encoding IL-36 gamma, giving a boost to an immune response against HIV, Zika virus, and influenza in mice [2]. The review by Shrestha & Grubor-Bauk gives an excellent example of the use of a novel cytolytic platform of DNA immunization based on truncated mouse perforin [10], whereas Hinkula et al. [6] use a plasmid DNA encoding the TLR5 ligand flagellin as an immunomodulator which, if administered together with formalin-inactivated whole influenza A vaccine, increases the antibody response 200-fold, hemagglutination inhibition (HAI) titer by 100-fold, and the cellular response against flu by 40-fold.

An overview of the experimental papers in this Special Issue reveals that DNA vaccines of the pre-COVID-19 era were not yet mature enough to be an instrument of human vaccination, as six out of seven experimental papers describe the immunization of mice; only one, by Akulova et al., describes the results of the Phase II clinical trial of a HIV DNA vaccine candidate [3]. However, today, the situation is rapidly changing.

The spread of the SARS-CoV-2 virus and the COVID-19 disease has triggered an unprecedented surge in vaccine research and development, as well as attracting an unsurpassed amount of funding in a very limited time—and for good reasons. Worldwide infections with SARS-CoV-2 are increasing rapidly and, at the time of writing this editorial, approach 40 million diagnosed cases; the actual number of infections is much higher. Over one million deaths have resulted from a disease that did not exist for us a year ago. This new virus disseminates in a way that has been impossible to predict; traditional countermeasures such as hygiene, physical distancing, various levels of public lockdowns, and quarantines have proven to be insufficient, highlighting the urgent need for prophylactic vaccine(s) to be applied worldwide as the only way to stop viral spread.

The ongoing COVID-19 crisis has led to a very rapid development of vaccine candidates from academia and small and medium-sized biotech companies, as well as pharmaceutical giants. Now, nine months down the road, several candidates have already gone through discovery, preclinical testing demonstrating immunogenicity, and, in some cases, efficacy, process development, toxicity testing, production and recruitment for and completion of Phase 1 safety and immunogenicity studies. The most advanced candidates have already proceeded into Phase 2 and 3 testing and scaling up; preparation for large-scale production is well under way. It is hard to comprehend what an achievement this

really is and it is important to recognize that a lot of the progress has been made through performing multiple steps in the vaccine (Research & Development) R&D process in parallel and at great financial risk: in the case where a vaccine candidate does not show the expected immunogenicity or safety, for instance, the resources invested in that candidate will be lost. In a traditional stepwise approach to vaccine development, unsuccessful candidates would have been deselected at an earlier stage and the financial risks would have been mitigated.

It is important to emphasize that this approach is only made possible through numerous initiatives both from states, not-for-profit organizations such as the Coalition for Epidemic Preparedness Innovations (CEPI) and the Bill and Melinda Gates Foundation, and private equity alike. One such initiative is Operation Warp Speed, to which the US Congress has directed almost 10 billion USD. Operation Warp Speed aims to deliver 300 million doses of a safe, effective vaccine for COVID-19 by January 2021 as part of a broader strategy to accelerate the development, manufacturing, and distribution of COVID-19 vaccines, therapeutics, and diagnostics (https://www.hhs.gov/about/news/2020/06/16/fact-sheet-explainingoperation-warp-speed.html).

At no time in history has biomedical research moved forward at such a high pace. The research community, the pharmaceutical sector, and funding agencies have responded to the threat with dedication. Moreover, the dissemination of data has never been faster. Rapid publication of data has been a key factor in the battle against the virus and the role of preprint servers such as biorxiv and biomedrxiv cannot be underestimated. For instance, the genome sequence of SARS-CoV-2 was published online less than two weeks after the Wuhan Municipal Health Commission in Wuhan City, China, reported a cluster of 27 pneumonia cases of unknown etiology, and only one day after the Chinese CDC reported that a novel coronavirus had been detected as the causative agent of COVID-19 (https://virological.org/t/novel-2019-coronavirus-genome/319/26). Since then, COVID-19 research has exploded; never before have studies on a previously undescribed disease and disease-causing agent provided so much knowledge in such a short time. The remarkable progress in synthetic biology and manufacturing of DNA sequences, based on the availability of sequences, not biological materials, enabled the rapid development of vaccine candidates against COVID-19. We see this as an argument in favor of the synthetic nucleic acid-based vaccines.

Thus, it is not surprising that some of the most advanced vaccine candidates against COVID-19 represent novel previously unlicensed technology platforms exploiting synthetic genes, such as adenovirus vector platforms (e.g., Astra Zeneca, CanSino Biologics, Sputnik-V; https://clinicaltrials. gov/ct2/show/NCT04437875), mRNA (e.g., Moderna, Sanofi, Curevac, Pfizer/BioNTech), and, of course, DNA, such as INO-4800 by Inovio, thought to be the furthest ahead among four DNA-based vaccines that have started human testing for COVID-19 (https://clinicaltrials.gov/ct2/show/NCT04336410). A concise review by Liu in this Special Issue underpins pre-COVID-19 era efforts to make DNA vaccines more efficient, comparing them with similar efforts made for mRNA vaccines [11]. At the moment, the reader may get an impression that the mRNA vaccines have the upper hand, specifically referring to the many advanced mRNA COVID-19 vaccine candidates in clinical trials. However, the INO-4800 DNA vaccine also showed promise in a Phase 1 trial: INO-4800 was safe and well-tolerated and induced immune responses in a majority of participants (https://www.fdanews.com/articles/199278 inovios-covid-19-vaccine-trial-placed-on-partial-hold). Taking this stand, we are far from expressing DNA vaccine pessimism. It is likely that the emergence of RNA vaccines as a viable vaccine modality and the technological developments that are driven by this field may also impact the development of new DNA vaccine and related technologies. Given the tremendous need for: (i) high speed in vaccine development relying on the availability of sequence information, (ii) the ease and speed of vaccine production using already established platforms, and (iii) no cold chain to allow for worldwide distribution, which are provided by DNA vaccines, one can be sure that the first vaccines of this type will soon see licensure for human use and find clinical application abroad.

When the smoke of the battle against COVID-19 and the race for an effective vaccine settles, we will face a new normal where the dissemination of SARS-CoV-2 is hopefully controlled, although the way we interact with each other and the way we work would be changed once and forever. People would gradually forget the times of hardship that we are currently facing. It may happen that our collective memory will be shorter than the T cell memory. Nevertheless, we hope that the insights gained and lessons learned in vaccinology will prevail and that investments will continue to be made in this domain to further advance vaccinology and prevention of infectious diseases in all corners of the world, since every dollar invested in vaccinology has an enormous potential return in saving lives and improving peoples' health. As scientists and vaccine advocates, we have to act to preserve this momentum. One day, we will be out of the COVID-19 pandemic, but we can be sure that new challenges will appear on the horizon. We have to keep up the pace to be prepared.

**Author Contributions:** Conceptualization, M.I.; original draft preparation, K.L., M.I.; revisions and editing K.L., M.I.; funding acquisition, M.I. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by LZP-2018/2-0308 and RFBR 17-54-30002 to M.I., and by COST action ENOVA CA1623.

**Conflicts of Interest:** The authors declare no conflict of interest.


**3.5 Microvascular anastomosis and breast reconstruction**

The recipient artery and vein were dissected and reserved, with the distal ends being ligated and the proximal ends being clamped. The blood vessels within the chest wall were trimmed. The flap was moved to the breast defect site and temporarily fixed, and the veins and arteries below the abdominal wall were trimmed. Microscopic vascular anastomosis was performed using 9-0 or 10-0 prolene suture thread. The patency of the anastomosis was checked. After the flap blood supply was established, the shape of the breast was formed.


**4.2. Current industrial trends**

based etching solutions. Furthermore, lower expenses are involved and the method produces

Laser-based front-contact openings and metal deposition are considered to constitute the most appropriate method for fabricating cells with Ni/Cu contacts on the industrial scale. In 1990, laser-based contact patterning was done for EFG polycrystalline silicon solar cells [63]. The laser process was used to fabricate laser-grooved buried contact (LGBC) solar cells at the University of New South Wales (UNSW), Australia [64]. Later on, a BP Solar was involved to

Laser-based ARC ablation can not only help in patterning the grid more easily with Ni deposition, but can also assist in producing cells with higher efficiencies. Thin-width laser grooves for the LGBC cells and selective emitter doping can be performed with the laser-based process for solar cell applications. Thinner finger lines and selective emitter doping ensure that a high open-circuit voltage (Voc) and higher efficiencies can be realized. Moreover, the heavy diffusion step for selective emitter solar cells can be conducted relatively easily [66, 67]. At Fraunhofer ISE, laser chemical processing (LCP) was demonstrated involving the performance of local etching by the use of phosphorus doping and laser grooving with damage free silicon microstructuring [68]. LCMD and LTC methods were also adopted to pattern the front-contact

grid and Ni seed layer deposition simultaneously for solar cell applications [51, 52].

silicon with a record 17.5% efficiency was also reported [71].

**4. Potential and commercial aspects**

The front and rear contacts of the solar cells can be patterned by a process called 'mechanical scribing'. Here, an artificial diamond tip is used to scribe a uniform and shallow depth over a given surface area. The method has the advantage of process simplicity and it can be helpful in patterning the solar cell surface with high throughputs. A mechanical scribing method was used to make grooves for buried contact solar cells [69]. Passivated emitter rear-cell (PERC) solar cells with a back-contact formed by a mechanical scribing process were also reported [11]. Solar cell efficiencies of more than 20% were achieved for such PERC solar cells. An artificial diamond tip of about 10 μm was used to pattern SiOx and SiNx passivation layers, while Sienriched SiNx layers were found to be etched away easily. Solarex Corporation, USA, used a diamond blade to form deep grooves with widths within the range of 25-45 μm for solar cell applications [70]. Mechanical V-texturing for buried contact solar cells on multi-crystalline

Copper can be the best alternative to silver in the front-electrode formation of crystalline silicon solar cells. The main motivation derives from the fact that it exhibits conductivity almost equal to silver, while its cost is about 100 times lower. The higher efficiency potential of the copper in terms of lower shading losses and higher FFs also provides additional benefits compared

less waste.

*3.4.3. Laser ablation*

288 Solar Cells - New Approaches and Reviews

*3.4.4. Mechanical scribing*

produce LGBC solar modules commercially [65].

Additional steps for front-contact patterning and Ni seed layer formation with a sintering step followed by Cu electroplating makes the process more complex for Ni/Cu metallized silicon solar cells. At present, Ni/Cu contacts have seen limited implementation on the industrial scale. Modules based on laser-grooved buried grid (LDBG) cells from BP Solar were only produced commercially [79]. However, there has been a lot of progress in terms of increases in solar cell efficiencies. More than 20% efficiencies have already been achieved at Fraunhofer ISE, Interuniversity Microelectronics Centre (IMEC), Kaneka, Roth & Rau Research and Schott Solar [78, 80-83]. Many research institutes have been involved in investigating metallization schemes composed of Ni/Cu metal stacks [8, 10, 13, 42, 84-90].

Thus far, efficiencies of up to 23.5% for a Cu-plated heterojunction-type solar cell have been achieved by Kaneka [80]. Roth & Rau Research also came up with a 22.3% efficient hetero‐ junction cell with Ni/Cu contacts [81]. For a passivated emitter rear cell (PERC), solar cell structure efficiencies above 20% have already been demonstrated by Fraunhofer ISE, Schott Solar and IMEC. Fraunhofer ISE and Schott Solar presented efficiencies of 21.4% [82] and 21.3% [91], respectively. However, at IMEC, an industrially applicable Ni/Cu-plated i-PERC-type solar cell with an average efficiency of 20.5% (more than 100 cells) and a best cell efficiency of 20.79% has been fabricated [83]. IMEC also investigated the application of Ni/Cu contacts for a rear-junction solar cell on a n-type substrate and achieved a best cell efficiency of 20.5% [78]. Ni/Cu metal stacks were also applied to contact laser-doped selective emitter (LDSE) solar cells. LDSE-type solar cells with efficiencies of 19.8% and 19.64% were developed at Hyundai Heavy Industries and Shinsung Solar, respectively [86, 92]. A 19.33% efficient LDSE-type solar cell with Ni/Cu contacts was presented at the UNSW [93]. More recently, a laser-doped Cuplated bifacial silicon solar cell exceeding 19% efficiency was developed at UNSW [94]. Solar cells consisting of Cu-plated contacts and exceeding 20% efficiency are presented in Table 2.

**Table 2.** Solar cells consisting of Cu-plated contacts exceeding 20% efficiency.


**6. Histopathological studies**

in TBARS level. The standard hepatoprotective drug Silymarin maintained the decreased lipid

Concomitant cellular oxidative stress was manifested by reduced GSH levels and in‐ creased lipid peroxidation. The inverse linear relationship between the ROS level and the GSH level indicated that free radical species were generated by exposure to anti TB drugs which reduced intracellular antioxidant levels. The results indicate that, the herbal drug *Phyllanthus amarus* has very good hepatoprotective effect in liver damage. The results were

Superoxide dismutase and Catalase mutually function as important enzymes in elimination of Reactive oxygen species (ROS). SOD is the major attractive metalloprotein in the antioxidant family. The defensive antioxidant enzyme next to SOD is CAT. CAT is an enzymatic antioxi‐ dant widely distributed in all animal tissues, and the highest activity is found in the red cells and liver. Both are the key component of the antioxidant defense system. In the present study, the observed decrease in SOD and CAT activities were presumably associated with the increased oxidative stress caused by these toxicants that might be due to low level of zinc (a metal constituent of the enzyme SOD) in liver tissue (Arun and Balasubramanian, 2011) Therapy at 200-400 mg/kg b.w. reversed the SOD and CAT activity in the liver tissues and protected from free radical induced oxidative stress.  Since, phosphatase is a constituent of all the body tissues; it plays an important role in inorganic pyrophosphates activity. ATPase activity may be considered as a marker for assessing hepatocellular damage induced by hepatotoxic agents.

In our experiment, a concurrent fall was found in ATPase in liver after toxicants exposure. It might be due to dysfunctional and dystrophic changes in the mitochondria and cell membrane permeability. This damage was also very clearly visible in histopathological studies after toxicant administration. These observations are substantiated by other authors (Gao and Zhou, 2005; Krithika and Verma, 2009).The effect of the extracts on ATPase was as pronounced with

Liver damage induced by toxicant was associated with a variety of biochemical abnormalities following loss of integrity of the cell membrane or interference with normal hepatocytes

The reason for hepatoprotective effect of the extracts may be due to presence of lignans and

The results of this study showed that common medicinal plant, *P. amarus* even at minimal dose 300mg/kg was found to have beneficial hepatoprotective properties in albino rats as all the

flavonoids which might have scavenged the free radical offering hepato protection.

peroxidation level to the normal limits in the liver.

290 Pharmacology and Nutritional Intervention in the Treatment of Disease

author (Gnanadesigan *et al.,* 2011).(Table 5)

100mg/kg as with 200-400mg/kg b.w.

metabolism and function.

**5. Conclusion**

presented in the Table 4.

(Table 6)

**Figure 1.** Hepatocytes of the normal control group showed a normal lobular architecture of the liver (400 x).

**Figure 2.** Hepatocytes of the INH + RIF+ PZA treated group showed liver cell necrosis and inflammation also observed in the centrilobular region with portal triaditis (400 x).

um tuberculosis strains by DNA sequencing: a randomized trial. Annals of Clinical

Hepatoprotective effect of *Phyllanthus amarus*

http://dx.doi.org/10.5772/57373

293

[4] Eminzade S, Uras F and Izzettin FV. (2008). Silymarin protects liver against toxic ef‐ fects of anti-tuberculosis drugs in experimental animals. Nutrition & Metabolism,

[5] Devaraj VC, Gopala KB, Viswanatha GL, Kamath JV, Kumar S (2011). Hepatoprotec‐ tive activity of Hepax-A polyherbal formulation. Asian Pacific Journal of Tropical Bi‐

[6] Joshi H, Parle M (2007). Pharmacological evidences for antiamnesic potentials of *Phyllanthus amarus* in mice. African Journal of Biomedical Research, Vol. 10; 165 –

[7] Lee S B, Bablanian R and Esteban M. (1996). Regulated expression of the interferoninduced protein kinase p68 (PKR) by vaccinia virus recombinants inhibits the repli‐ cation of vesicular stomatitis virus but not that of poliovirus. Journal of Interferon

[8] Reitman and Frankel. (1957). A colorimetric method for the determination of serum glutamic oxalacetic and glutamic pyruvic transaminases. Amer. J. Clin. Pathol. 28:

[9] Sharma SK and Krishnamurthy CR (1968). Production of lipidperoxides of brain. J

[10] Misra HP and Fridovich I (1972). The role of superoxide anion in the autoxidation of epinephrine and a simple assay for superoxide dismutase. J Biol Chem. 247: 3170–

[12] Ranawat L, Bhatt J and Patel J. (2010). Hepatoprotective activity of ethanolic extracts of bark of Zanthoxylum armatum DC in CCl4 induced hepatic damage in rats. J. Eth‐

[13] Muriel P, EscobarY (2003). Kupffer cells are responsible for liver cirrhosis induced by

[14] Thirunavukkarasu C, Sakthisekaran D (2003).Influence of sodium selenite on glyco‐ protein contents in normal and N-nitrosodiethylamine initiated and Phenobarbital

[16] Adebisi SA, Oluboyo PO, Okesina AB (2000).Effect of drug-induced hyperuricaemia on renal function in Nigerians with pulmonary tuberculosis. Afr J Med Med Sci.

[15] Zimmerman L (1973).Podiatry and the law.J Am Podiatry Assoc, 63; 691-3.

[11] Aebi H (1984). Catalase in vitro; Methods Enzymol. 105. 121–126

carbon tetrachloride. J. App. Toxicol., 23(2):103-108.

promoted rat liver tumors. Pharmacol Res 48(2):167-73.

Microbiology and Antimicrobials, 8:4.

and Cytokine Research 16, 1073-1078.

5:18.

173.

56-63.

3175.

3-4:297-300.

Neurochem; 15: 147-9.

nopharmacol. 127(3):777-780.

omedicine. 142-146.

**Figure 3.** Hepatocytes of the *P.amarus* treated group showed cuboidal hepatocytes with moderate portal triaditis and their lobular architecture was normal (400 x).


*2.3. Statistical Analysis*

Statistical analysis was carried out with Stata15 for Windows. The comparison of quantitative variables between the groups was based on analysis of variance adjusted for the participants' age, education level, and civil status (ANCOVA). The estimation of the effect size of the pairwise comparisons was based on Cohen's-d coefficients (|d| > 0.20 was considered low, |d| > 0.5 was considered moderate, and |d| > 0.8 was considered high) [52]. In addition, Finner's procedure (a familywise error rate stepwise method which has demonstrated more powerful than Bonferroni correction) controlled the increase in Type-I error due to multiple comparisons [53].

 Each regression was adjusted in five blocks/steps: (a) First block-step entered and set the covariates participants' age, education, and civil status; (b) Second block added ED-related variables (EDI-2 total, onset of the ED, and duration of the ED); (c) The third block included global psychopathological state (SCL-90R GSI); (d) The fourth block entered NSSI (0 = absent; 1 = present); and (e) The fifth block included personality traits (TCI-R scale scores). The specific predictive capacity of each step/block was measured as the increase in the *R*2 coefficient (Δ*R*2).

 This method constitutes an extension of multiple regression modeling, which aims to estimate the magnitude and significance of hypothesized associations in a set of variables with the advantage of allowing for the testing of mediational links (direct and indirect effects) [54]. Structural equation modeling (SEM) was used by defining the maximum-likelihood estimation of parameter estimation and testing goodness-of-fit through standard statistical measures: The root mean square error of approximation (RMSEA), Bentler's Comparative Fit Index (CFI), the Tucker-Lewis Index (TLI), and the standardized root mean square residual (SRMR). Adequate model fit was considered non-significant by χ2 tests and if the following criteria were met [55]: RMSEA < 0.08, TLI > 0.9, CFI > 0.9 and SRMR < 0.1. In this study, ER was defined as a latent variable defined by DERS scale scores, and the personality profile as a latent class defined by TCI-R scale scores.

#### **3. Results**

#### *3.1. Sample Characteristics*

Table 1 includes the description and the comparison between the four groups of the study defined by ED diagnosis and sex. Differences emerged with regards to civil status, education and age.

SD: Standard deviation. \* Bold: Significant comparison (0.05 level). ED: Eating disorder; HC: Healthy control.

#### *3.2. ER and Negative Affect Measures and Comparison between Groups*

The first block of Table 2 includes the results of the ANCOVA (adjusted forage, civil status, and education) comparing the four study groups (ED-women, ED-men, HC-women, and HC-men) with regard to DERS scales, EDI-2 scales, and the binge-eating/purging levels (these two last measures were only compared between ED groups). Pairwise comparisons between ED-women and ED-men reached significance in all measures (more ER difficulties for ED-women), except for DERS awareness and the EDI-2 interpersonal distrust (no differences between the two groups were obtained). ED-women registered higher mean scores in all the measures compared to HC-women. The same occurred with ED-men compared to HC-men (except for on EDI-2 perfectionism). No differences between the two HC groups (women and men) were found.

The second block of Table 2 contains the prevalence of NSSI and the comparison between the groups (comparison between the groups was based on logistic regression adjusted by the participants' age, education, and civil status). The proportion of ED-women who reported the presence of this behavior was higher than the proportion reported by ED-men (44.2% vs. 16.1%, *p* < 0.001), as well as the proportion reported by the HC-women (44.2% vs. 21.8%, *p* < 0.001). No significant differences were found comparing the HC groups (women and men) or between ED-men and HC-men.

Figure 1 includes a radar-chart for the study variables in the four groups. To allow for easy interpretation, z-standardized means were plotted.

SD: Standard deviation; NSSI: Non-suicidal self-injury; ED: Eating disorder; HC: Healthy control; EDI-2: Eating Disorder Inventory-20; DERS: Difficulties in Emotion Regulation Scale. Bold: Significant comparison (0.05 level); † Effect size in the moderate (|d| > 0.50) to large range (|d| > 0.80); 1 Results obtained in logistic regression. — Binge and purging episodes werenotregisteredfortheHCgroup.

\* **Figure 1.** Radar-chart with the z-standardized means by group (*n* = 1082). DERS: Difficulties in Emotion Regulation Scale; ED: Eating disorders; EDI-2: Eating Disorders Inventory-2; HC: Healthy controls; TCI-R, SCL-90R: Symptom Checklist-90 Items-Revised; Temperament and Character Inventory revised.

## *3.3. Comparison of ER between ED Subtypes*

Table 3 includes the ANCOVA (also adjusted for age, education, and civil status) comparing DERS scores between the ED types (AN, BN, BED, and OSFED), stratified by sex. In the female subsample as a whole, greater ER difficulties were associated with BN, followed by BED and OSFED. The lowest DERS scores were found in AN. In the male subsample, greater ER difficulties were registered in OSFED group, followed by the BN and BED groups. AN males had the lowest DERS scores. Results obtained in the men subsample must be interpreted with caution due to the low sample size of the groups.

## *3.4. Predictive Capacity of the Study Variables on ER*

Table 4 includes the final models of the two multiple regressions measuring the predictive capacity of study variables on the DERS total score. In the ED-females model, emotion regulation difficulties were predicted by higher EDI-2 total scores, more pronounced psychopathology, higher levels in the novelty seeking and cooperativeness traits, and lower levels in the reward dependence and self-directedness traits. No significant predictive contribution of the NSSI on the DERS-total was found in the ED-females group.

For the ED-males model, DERS-total scores increased for men who reported higher scores on the EDI-2, those with higher psychopathology and lower levels in persistence.

BED: Binge eating disorder; OSFED: Other specified feeding or eating disorder.

self-injury. SCL-90R GSI: Global Severity Index of the questionnaire; Symptom Checklist-90 Items-Revised; TCI-R: Temperament and Character Inventory-Revised; B: Non-standardized B-coefficient; SE: standard error; Beta: Standardized B-coefficient; 95% CI: 95% confidence interval.

#### *3.5. Pathways Analysis*

Figure 2 includes the path-diagram with the standardized coefficients of the SEM obtained in the ED group (Table S1, supplementary material, includes the complete results valuing direct, indirect. and total effects). Goodness-of-fit was obtained (all the fit statistics were in the adequate range). The latent variable measuring ER difficulties (labeled as DERS in the figure) was directly increased for patients who presented higher ED severity (higher EDI-2 total), higher psychopathology (higher SCL-90R GSI), and who were younger. Higher scores in the latent variable measuring the personality construct (labeled as TCI-R in the figure) were also direct predictors of greater ER difficulties. ED severity and the psychopathology levels mediated the relationships between personality measures and ER, as well as between sex and ER: Higher levels in the TCI-R construct and being female increased EDI-2 interoceptive awareness and SCL-90R scores, which contributed to increases on the DERS.

**Figure 2.** SEM: Standardized coefficients (ED subsample; *n* = 718). Continuous line: Significant parameter (0.05 level). Dash line: Nonsignificant parameter. GSI: Global Severity Index.

#### **4. Discussion**

The present study attempted to address a relevant issue in the psychopathology of male patients with ED. It aimed to provide a better knowledge regarding ER in this clinical population, analyzing and comparing ER difficulties between male and female patients with ED and HC, which was rarely studied before. Findings from this study provide new information for the treatment approach of male patients with ED, a minority in the field of ED that runs the risk of being overlooked.

Our first main finding confirmed that patients with ED, both males and females, showed greater global ER difficulties than HC. Although these results are not in accordance with prior research indicating that males with ED did not differ from HC males in emotion regulation strategies, such as emotion recognition [34], they are in line with previous studies which found that negative affect and difficulties in ER predicted disordered eating in both males and females in community samples [26,27]. These discrepancies may be due to the fact that the study by Goddard et al. [34] focused on emotional recognition and not on ER. Moreover, our results support previous findings in clinical samples that

have reported decreased effective ER strategies among female patients with ED when compared with HC [10,56], suggesting that a lack of effective ER skills may prompt individuals to use disordered or abnormal eating behaviors to regulate negative affect [57], as well as contribute to body dissatisfaction and disordered eating in males [30]. Therefore, ER difficulties may act as an important etiological feature [57] or risk factor for the occurrence of EDs [23]. Although previous studies have focused primarily on females with ED, our findings also offer the possibility of generalizing these findings to males with ED.

When comparing male and female patients with ED, female patients with ED engaged in more dysfunctional ER strategies than males with ED, displaying greater scores on all DERS scales, except for DERS emotional awareness. There were no differences between male and female controls with regard to ER difficulties. First, these findings confirmed our hypothesis that both males and females with ED displayed a lack of emotional awareness. Second, the fact that female patients with ED scored higher in the most of the DERS scales, such as nonacceptance of emotional responses, limited access to emotion regulation strategies, lack of emotional clarity, and difficulties in engaging in goal-directed behavior or in controlling impulsive behaviors when experiencing negative emotions suggests that there are indeed gender-related patterns of ER in ED. However, we cannot fully exclude gender-related response bias, since males may have had a tendency to minimize or underestimate (intentionally or unintentionally) the difficulties related to their ER in order to prevent their culturally imposed, self-perceived masculinity ideals from being threatened [58]. Furthermore, males with ED appear to more often use externalizing behaviors (e.g., hetero-aggression) or to engage in drug or alcohol use/abuse to deal with emotions whereas females with ED tend to use more internalizing behaviors, such as NSSI [41]. Our results support these observations, with females with ED in the present study showing significantly more NSSI behaviors than males with ED.

Regarding ED types, our findings showed higher ER difficulties in patients with binge eating-related behaviors (BN, BED, and OSFED) compare to patients with restrictive behaviors (AN), in both males and females with ED. These results are consistent with previous studies reporting more ER difficulties among patients with binge-eating behaviors [44,59], but they are discrepant to other studies reporting less severe ER difficulties in BED patients and no significant differences between AN-R and other ED subtypes [5]. However, while females with BN showed the greatest ER deficits compared to females with other ED diagnoses, males diagnosed with OSFED were those who displayed the most ER difficulties. These differences suggest that females and males with ED engage in different disordered eating behaviors for alleviating negative affect and emotional instability. Females with ED seem to present more binge eating and purging behaviors for ER, whereas males with ED are prompt to use more heterogeneous ED-related symptomatology for alleviating emotional distress (e.g., high levels of exercise).

In terms of primary predictors, higher general psychopathology was the shared factor associated with ER difficulties in both males and females with ED. However, ED severity and different personality traits were identified as differential predictors in females and males with ED. Increased ED severity, higher novelty seeking, higher cooperativeness, lower reward dependence, and lower self-directedness were related to higher ER difficulties in females with ED, while lower persistence was associated with ER difficulties in males with ED. Thus, in females with ED, difficulties in ER were associated with a tendency to be more impulsive and intolerant of routine, and which are linked with seeking little emotional support, the unwillingness to be sociable, and having difficulty in expressing feelings and thoughts [60]. On the other hand, in males with ED, difficulties with ER were associated with low persistence, that is, a tendency to be less perseverant in situations of frustration and fatigue [60]. In light of our results, our findings suggest that personality differences may impact ER difficulties, therefore, it would be important to assess for personality traits and consider potential gender-related differences [61,62]. In this regard, it may also be useful to apply ER-based adjuvant treatments focused on reducing impulsivity and increasing self-directedness and reward dependence for females with ED,

and specific treatment approaches for males with ED where increased persistence management are specifically addressed.

Finally, another emergent finding was that both ED severity and general psychopathology mediated the relationships between personality and ER difficulties. This may open a new line of research that allows for knowing if the improvement in the ED symptomatology could establish changes in emotion dysregulation. In this sense, a previous study in females with ED found ER improvements after CBT (treatment as usual, without any specific module addressing ER), especially in patients with BN. This study found that improvements in ER were the largest in those with a better treatment outcome [23]. In this line, our results reinforce this concept, suggesting that ED severity and psychopathology may be associated with ER difficulties. In addition, although our study is transversal and does not allow us to analyze the causality, we suggest the existence of a bidirectional pathological process that has ER difficulties acting as a maintenance factors for the ED. However, these findings do not exclude the possibility that ER is also a vulnerability factor for ED. The lack of longitudinal studies analyzing individuals before developing the ED does not allow us to identify if the ER is an etiopathogenic factor of the disorder or if, on the contrary, difficulties in ER are aggravated with the ED. It is most likely that ER is probably acting in both directions, both as a vulnerability factor and as a maintenance factor for the disorder (which is aggravated by psychopathology). With this in mind, we hypothesize that treatment enhanced with a module aimed at improving ER skills could benefit the treatment outcome of ED patients. Further studies should address this point.

Also, the results suggest that, a more dysfunctional personality profile and being female increased the risk of higher ED severity and general psychopathology, which contributed to an increase in ER difficulties in patients with ED. In this vein, a recent study found that depression moderated the association between ER difficulties and binge eating in patients with BED, suggesting that individuals who experience more intense emotions are more affected by difficulties in ER [8]. Again, the above is consistent with the need for treatment based on addressing the difficulties of ER in ED patients, since, although being aware of one's own emotions is not sufficient for an adaptive emotional regulation [44], it is the first step to improving it.


**5. Conclusions**

*Colloids - Types, Preparation and Applications*

with cost-effective nature.

Pd-Co catalyst, Ru-Co and Ru-Ni [101–104].

with either noble metal or transition metals, some examples are Pd-Au catalyst,

*(a) Polarization curves (b) Tafel plots of samples (c) polarization curves of PtCoFe@CN 1st and 10000th cycles (d) Amperometric i-t curves of PtCoFe@CN. Reprinted with the permission from [107]. Copyright ©*

Transition metal can be alloyed with transition metal itself and thus it can be a very good alternative for noble metal electrocatalysts because of the cost-effective nature of them. These alloys could either be binary alloys or ternary alloys, the authors will discuss examples from both binary and ternary alloys. Ni can form alloy with various other transition metal, but Ni-Mo binary alloy is considered best for the EH2ER. Zhang et al. worked on the synthesis of MoNi4 supported over the MoO2 cuboids over the Ni foam. This catalyst has the activity similar to Pt/C with zero onset potential and ƞ10 = 15 mV and very low tafel slope of 30 mV dec−1 [105]. Ni binary alloys face the problem of corrosion which can be overcome by using the carbon support along with the Ni based alloy. Co also form binary alloys with Fe using N-doped carbon-based support. Also, Co can be alloyed with Mo forming good electrochemical catalyst for EH2ER such as Co3Mo having an overpotential of ƞ10 = 68 mV and tafel slope of 61 mV dec−1 [106]. Ternary alloys in the recent times have gained popularity for EH2ER electrochemical catalysts as electronic and morphological features of catalyst can be tuned by variation in compositions of various metals and thus it can act as the good promising substitute for the noble-metal-based EH2ER catalyst. One such example is the use of small amount of Pt (4.6%) to the Fe-Co binary alloy to form the PtCoFe@CN electrocatalyst which demonstrated activity similar to that of commercial 20% Pt/C having an overpotential of ƞ10 = 45 mV as depicted in **Figure 10(b) [**107]. Research is still going on to prepare the ternary alloys without using the noble metals in it and that will really be the landmark in this field as it will be a catalyst

**210**

**Figure 10.**

*2017, American Chemical Society.*

This chapter discuses a thorough study of recent acheivements by C-NCs-based elelctroctalysts for ECO2RR and EH2ER. In this chapter, the authors have tried to summarize role of C-NCs in ECO2RR and EH2ER and the scope of these two important electrocatalytic reaction in combating the energy crises for human kind in introduction part. The examples that are discussed in this chapter were taken from recent reports in literature. The first part of this chapter sheds light on colloidal synthesis of nanocystals. The second part of this chapter emphasizes on effect of shape, size and composition in determining the catalytic activity, selectivity and stability of C-NCs for ECO2RR. Here the effect of ligand functionalization and MOF/NCs hybrid system on ECO2RR activity is also illustrated. The third part of this chapter addreses the role of C-NCs-based electrocatalysts on EH2ER, its activity and stability. In this part, indepth study about mechanism of EH2ER is also discussed. Although a lot has been done in ECO2RR but ECO2RR still face a big challenge in selectivity, similarly in EH2ER a lot of research has been dedicated to find a substitute of state-of-art Pt/C catalyst which is very much expensive. However, researchers have been sucessful in discovering the costeffective electrocatalysts which are as good as Pt/C but they are still facing the stability issues.


Approach 3: Non-structured approach.

*"I enjoyed seeing children take the initiative for their learning. I have seen them as protagonists who enjoyed the freedom they had when it came to making decisions about the challenges. But I have to look for strategies to accompany them in their learning, sometimes I felt that I did not know what to do or that the students did not need me. I am sure that is simply due to my lack of experience. I must learn how to motivate, guide and give feedback so that students feel that they are accompanied by me, that I am interested in and value what they are doing." "A positive aspect of the unstructured session was that since the groups worked alone, I had more freedom to intervene when I saw that I could help, and especially to be with those students who were more motivated. This type of task, I think, is more suited to my profile as a future teacher".*

*"I think that this format demands more from the students, stimulates them more and therefore their learning will go beyond the driving force. I think that I see myself as the future teacher choosing this type of activity when it comes to presenting the challenges. "*

#### *3.4. Correlation Analysis between Reflection and the Dimensions of Professional Identity*

For the three cooperative instructional approaches, the whole assessment of the narrative reflection (added scores of the ten reflective dimensions) was correlated with the students' assessment of the four identity dimensions: self-esteem (Id1), task perception (Id2), job motivation (Id3), and future perspectives (Id4) (Table 7). When considering all the correlation slopes and the corresponding confidence interval of reflection assessment with identity reflection, similar correlations were obtained between the reflection on the instructional approaches (A1, A2, and A3) and the categories of students' reflection on professional identity (Table 7).

## **4. Discussion and Conclusions**

#### *4.1. Activating Cooperative Learning Based on Non- to Structured Instructional Approaches Produced Identity in Practice*

The pre-service students were generally motivated towards implementing cooperative physical challenges in physical education classes in the schools, especially in the creative aspect of designing and implementing the physical challenges, since this implied considering others and coordinating the skills of group members to overcome individual limitations. This finding has also been documented in other studies promoting cooperative learning where the composition of the groups, the task the group was to undertake, the social skills training needs, and the assessment of both the learning and the instructional practice [32–34] (especially when studies focus on primary school children [3]) have been reported.

Regardless of the reflective dimensions, there were significant di fferences between approaches, with the assessment scores of the R1 (The focus of reflection) and R3 (Inquiring) to be higher than R2 (Prior conceptions and beliefs) and R4 (Transformation), as shown in Figure 1. Results also showed that the non-structured instructional approach was scored higher than the structured and semi-structured instructional approaches (Figure 1). Therefore, the pre-service students' scores were higher when they reflected on identifying cooperative teaching challenges and on focusing on questions and hypotheses about professional action. This was highly scored when they carried out the cooperative challenges with no established materials or student roles in the group.

Qualitative analysis of reflective narratives indicated that, while the pre-service teachers had positive experiences with cooperative challenges (there were only six dropouts), a number encountered di fficulties with implementing them in the classrooms. Issues identified included students socializing during group activities, individuals' poor awareness of being a member of a group, and poor regulation of roles. Similarly, Gillies and Boyle [23] reported that teachers found some di fficulties in the classroom when implementing cooperative learning. These included time management, the organization required to implement cooperative learning, and a reported concern with socializing in the groups.

The qualitative analysis of the reflective narratives also helped the pre-service teachers' perceptions of cooperative learning. All concurred that they, as well as the children, had had positive experiences. They talked about the children confronting new experiences, distributing roles themselves in the non-structured approach, learning how to communicate, and managing to interact with each other more e ffectively. As pointed out by Slavin [21], when cooperative learning is well structured, students understand how they are to work together to achieve their group's goal, which benefits students socially and academically. When implementing the non-structured approach, in particular, positive interdependence was enhanced when dual responsibility was achieved, that is, when students understood that they were not only required to complete their part of the challenge but to ensure others did likewise [23]. Other key elements that were critical to the e ffective implementation of the cooperative physical challenges included the pre-service teachers facilitating promotive interaction for the whole group, promoting responsibility for contributing to the groups' challenge, and demonstrating that dialogical intrapersonal bonds helped the groups to progress.

#### *4.2. Pre-Service Teachers Can Construct and Transform Their Professional Identities through Reflective Practice*

Regardless of the instructional approach, there was a significant association between reflection and professional identity. Reflective practices o ffer opportunities for pre-service teachers to engage in active and meaningful problem identification, evaluation, and innovation, through which they can hone and develop their identities [20]. Activating cooperative physical challenges in the three instructional approaches and the associated reflection was positively correlated with the four dimensions of professional identity defined by Kelchtermans [29], that is, self-esteem, task perception, job motivation, and future perspectives. Students' reflections primarily broadened and deepened perspectives on professional identity and impacted positively in the latter stages of implementing the cooperative

challenges in the schools. Key elements that were critical in transforming the pre-service teachers' professional identity were: promoting feedback, motivation, and engagement with the members of the teams, which, in turn, increased pre-service teachers' self-esteem, demonstrating that being implicated in democratic decisions helped students in the task perception, and proving that continuous practice at schools is needed to gain job motivation, i.e., their subjective feeling of competence increased because they gained experience with collaborative learning in educational practice [32].

Pre-service teachers were positioned in relation to both reflecting on cooperative teaching and personal consideration about professional identity. Research has shown that the transformation of professional identity becomes a powerful learning experience when it is articulated through reflective narratives [26,35], and particularly when it is shared and discussed in the process of dialogical and collaborative reflection. Indeed, our results showed the need to mix quantitative and qualitative methods to reduce misunderstandings of purpose [35] and that professional practice can indeed improve the knowledge of the discipline and the contexts in which we work, but are also about the self-awareness of how we conduct ourselves to deliberate and take action as professionals [26]. Our findings suggest that pre-service teachers shaped their identities not only over practice at schools but also through individual and community dialogical structures. Körkkö et al. [34] and Yuan and Mak [20] posit the fact that in order to transform professional identity, the highest level of reflection is gained when pre-service teachers have viewed the matter of reflection in several ways and decided on a course of action. Engaging pre-service teachers in reflection in-action (during the action) and on-action (after the action) is a way for them to establish a renewed perspective of their professional identity.

#### *4.3. Pre-Service Teachers, When Engaged in Individual and Community Reflection, May Activate Interpretation and Critical Competences Evaluation of Their Practice*

By reflecting, the pre-service teachers reported the aspects that either helped or hindered them when carrying out the associated instructional activities in the schools. First, the relationship between the theory given in the seminars on collaborative and reflective learning and the professional teaching practice itself. Secondly, the role of using collaborative and reflective methodologies to improve their level of self-awareness as both individuals and future professionals. The usefulness of using collaborative and reflective methods in identifying strengths, weaknesses, or gaps in their training was evaluated highly as well as the need to formulate activities based on new methods not only in their training as individuals but also as future professionals. Aspects they highlighted as being positive were: the development of both inter- and intrapersonal skills and the need to organize and implement the collaborative activities in the semi- and non-structured approaches in which the non-assignment of student roles or materials were highly evaluated.

When the pre-service teachers carried out the cooperative challenges in the initial stage, they worked to learn and at the same time, were responsible for their peers. When the pre-service teachers oversaw teaching physical activity cooperative challenges at the school, they were responsible for the PE students' learning outcomes. Among other skills, when the pre-service teachers developed cooperative challenges in a group, each student co-constructed knowledge and skills that were based on mutual dialogue, respect, help, cooperation, thinking before acting, learning to reflect and share ideas, proposals, and doubts. When the pre-service teachers designed and developed a cooperative challenge, these were to develop cognitive, psychological, and motor skills. The cooperative physical challenges were designed taking into account that the participation of all the students had to be possible regardless of the motor capabilities of each student. There had to be positive interdependence to achieve the common goal, i.e., all the members of a team had to work together to overcome the cooperative challenge.

In the teaching execution of a cooperative challenge at the schools, the pre-service teachers always gave instructions to generate a positive interdependence among the PE students of a team, especially when they were aware of the di ffering motor skills some group members had. Furthermore, it was essential that the challenges could be solved in a number of ways, so that dialogue, creativity, and trial-and-error could be promoted. The roles for the primary-school students were defined to fulfill the most e fficient way to execute the cooperative challenge. The roles facilitated the group deploying both social skills and academic abilities. The roles also enabled the group to solve the cooperative challenge in a satisfactory way with regard to academic knowledge while practicing social skills such as conflict resolution, negotiating, e ffectively communicating ideas, respecting the views of others, reaching a consensus from collective responses, and taking advantage of time. The roles reduced the prospect of some students adopting passive or dominant attitudes within the group and created an interdependence among the members of the group.

The pre-service teachers were motivated by the cooperative educational proposal in the physical education classes, fundamentally valuing the fact that they could provide the primary-school students with enough resources to develop social competences. It is recognized that education should also include aspects such as encouraging others to participate, listening to others, encouraging and supporting all peers who participate in a group, criticizing ideas and not persons, and expressing satisfaction for the success of others. Self-regulated learning through the promotion of structure and motivation encourages students to increase the ability to control and influence one's learning processes positively [36]. Both cooperative and reflective learning, when embedded in the development of cooperative physical challenges, represent educational strategies in which the learner's competences are promoted. In this respect, both the pre-service teachers and the primary-school students were involved in structured and non-structured instructional approaches in which values, attitudes, and beliefs contributed to sustainable, e ffective, and grounded sustainable learning [2,9,37–40].

All in all, we have identified the ways in which pre-service teachers represented their professional identity through reflection on instructional teaching approaches by maximizing the intake of competences, such as solving cooperative challenges, critical thinking through continuous self-assessment of professional practice, and communication through providing didactic instructive approaches. The cooperative physical challenge approaches helped pre-service teachers to identify aspects of their professional identity that ensured not only an e ffective intake of the sustainable competences but how to develop, understand, and self-assess professional practice experiences that at a later state fostered personal critical professional competence [20,34,38–40].

#### *4.4. Limitations of the Study*

The data presented in this research are consistent with the analysis performed, although there are some limitations that should be taken into account in future studies to obtain more generalized results and conclusions. One aspect that we consider fundamental for later studies is to expand the number of physical challenges in relation to each of the planned instructional approaches (structured, semi-structured, and non-structured), in order to carry out a longitudinal study that would allow us to understand how the development of individual professional identities evolves. Another aspect to consider would be to provide feedback with each of the student's cooperative groups once each of the sessions at the school had finished, in order to help the pre-service teachers in their process of self-knowledge and reflection in relation to their professional identity. In relation to the research instruments used, it would be convenient that, in addition to using the narratives centered on the cooperative learning dimensions, associations between the dimensions of cooperative learning are considered quantitatively with each of the dimensions of professional identity for each of the pre-service teachers to establish individual professional identity profiles. In future studies, we also intend to develop other research instruments, such as rubrics and questionnaires, focused on each of the aspects that make up professional identity and to understand in greater depth how cooperative physical challenges influence pre-service teachers' personal and professional development.

**Author Contributions:** Conceptualization, D.C.; methodology, J.C., D.C., R.B. and T.S.; formal analysis, R.B., D.C. and T.S.; investigation, D.C. and J.C.; resources, D.C.; data curation, R.B., J.C. and T.S.; writing—original draft preparation, J.C. and T.S.; writing—review and editing, J.C.; visualization, T.S.; supervision, D.C.; funding acquisition, D.C.

**Funding:** This research was funded by the Institute of Education Sciences, University of Girona (ICE-UdG).

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study, in the collection, analyses, or interpretation of data, in the writing of the manuscript or in the decision to publish the results.


*2.1. Semantic Topology Graph*

The construction of the semantic topology graphs is the basis of loop closure detection, so this section first introduces the construction process of the semantic topology graphs (see Figure 2). *2.1. Semantic Topology Graph* The construction of the semantic topology graphs is the basis of loop closure detection, so this section first introduces the construction process of the semantic topology graphs (see Figure 2).

**Figure 2.** Flow chart for constructing semantic topology graph. **Figure 2.** Flow chart for constructing semantic topology graph.

For each obtained image, semantic segmentation is performed to extract landmarks. After preprocessing, the image is divided into landmark regions and contours. The obtained landmarks are selected and sent to AlexNet to extract third convolutional layer (Conv3) features. Then, Gaussian random projection is used to reduce the dimensionality of the feature vectors, and low-dimensional feature vectors are output. In addition, the Hu moment is calculated according to the obtained contours. At the same time, the preprocessed semantic segmentation images are used to extract the For each obtained image, semantic segmentation is performed to extract landmarks. After preprocessing, the image is divided into landmark regions and contours. The obtained landmarks are selected and sent to AlexNet to extract third convolutional layer (Conv3) features. Then, Gaussian random projection is used to reduce the dimensionality of the feature vectors, and low-dimensional feature vectors are output. In addition, the Hu moment is calculated according to the obtained contours. At the same time, the preprocessed semantic segmentation images are used to extract the center of the landmark region as a node, and the landmarks seen from the same viewpoint are connected by undirected edges to establish a semantic topology graph according to the co-visibility information.

Finally, a random walk descriptor is exploited to describe the topological graph structure. Here, the images obtained before the loop closure detection are called dataset images.

#### 2.1.1. Landmark Extraction

Previous methods [30,31] have employed object proposal [42] to extract landmark regions even though it contains a lot of irrelevant feature information. In particular, the proposed method adopts semantic segmentation to extract landmarks, and this can accurately obtain the range of landmark regions.

DeepLabV3 + [43] is one of the most influential semantic segmentation models. It is better than FCN [44], U-Net [45], and SegNet [46] for some datasets, and it is widely used in the field of engineering technology. The ADE20K dataset [47,48] covers a wide range of scenes and object categories. Furthermore, it provides dense annotations, so it is used to train the DeepLabV3 +. The pre-trained DeepLabV3 + model can be applied for extracting landmarks.

Here, DeepLabV3 + was used to fuse the shallow features outputted by the encoder with the deep features generated from the ASPP module so that it could produce high-precision semantic segmentation results.

#### 2.1.2. Landmark Selection

2.1.3. CNN Features

global feature in the landmark region.

Figure 4e) to the CNN feature to describe the landmark.

Due to the effects of illumination and dynamic disturbance in the images obtained by the robot, as well as the inherent defects of the semantic segmentation model, there was a lot of noise, as well as dynamic and secondary landmarks, in the semantic segmentation image that was obtained via the model discussed in Section 2.1.1. To overcome these problems, the landmarks were preprocessed to obtain significant landmark regions. Then, the dynamic regions from the landmarks obtained by preprocessing were removed, and the distinctive patches were selected.

As shown in Figure 3, the semantic segmentation image (see Figure 3b) was filtered to remove the regions; its area was less than the specified threshold (the threshold was 100 in this paper). Figure 3c was obtained by merging region filtered out with the surrounding area. Through the above-discussed procedures, the secondary landmarks and holes were filtered out to obtain the obvious landmark region with clear boundaries. *Remote Sens.* **2020**, *12*, x 7 of 27

**Figure 3.** Selection of the landmarks: (**a**) raw image, (**b**) semantic segmentation image, (**c**) the result of filtering, and (**d**) the result of eliminating pedestrian dynamic landmarks. **Figure 3.** Selection of the landmarks: (**a**) raw image, (**b**) semantic segmentation image, (**c**) the result of filtering, and (**d**) the result of eliminating pedestrian dynamic landmarks.

The AlexNet network architecture had 8 layers, of which the first 5 layers were convolutional layers and the last 3 layers were fully connected layers. There was a pooling layer after the 1st, 2nd, and 5th convolutional layers, but there was none after the 3rd and 4th convolutional layers. Each convolutional layer had activation function ReLU and normalization. The input of the network was a 227 × 227 3-channel image, and the output feature of the third convolutional layer was 13 × 13 × 384 = 64.896. According to the research of [27], the output features of the third convolutional layer of AlexNet perform best under appearance changes. We found that the output features of the fully connected layer had strong semantic information that was robust to viewpoint changes but poor for appearance changes. At the same time, it was proved that the AlexNex obtained by pre-training in the object recognition task was better than the CNN model based on place recognition training when considering the characteristics of the entire image under the viewpoint changes. Other advanced networks such as VGG, ResNet, and DenseNet have complex architectures, as well as a lack of research and utilization in the field of loop closure detection. Therefore, this article used the relatively lightweight and mature AlexNet in the loop closure detection field to extract CNN features. Based on the above research, the proposed method employed the output of the Conv3 of the AlexNet as the

The landmark proposal extracted by the object proposal method contained a large amount of irrelevant feature information. This led to a certain amount of noise influence in the CNN feature description. However, the landmark area extracted by semantic segmentation in this paper only contained the landmark feature and no other unrelated features. Figure 4a–c is introduced in Section 2.1.2, this section explains the landmark area and contour extraction. The landmarks from the filtering result (see Figure 4c) were selected to get Figure 4d, and the contour binary (Figure 4e) of the corresponding landmark was obtained by a Canny operator. Then, the landmarks (see Figure 4d) were resized to 227 × 227 pixels and input to the pre-trained AlexNet to extract features. As a result, the features of each landmark could be represented by a 64.896-dimensional vector. In order to keep the original size information of the landmark, this paper added the Hu moment of the contour (see

the network was pre-trained for object recognition tasks on the ILSVRC dataset [50].

In order to overcome the impact of dynamic scenes, the semantic information of landmarks was then used to eliminate the pedestrian dynamic landmarks. At the same time, the pedestrians and long-term parking car region were merged, and the merged area could be used as car landmarks in subsequent work. Figure 3d was obtained by the above-mentioned operations. After excluding dynamic landmarks, in the follow-up loop closure detection, the dynamic landmarks were no longer matched. Furthermore, the number of pixels could be calculated for the landmark region. The distinctive landmarks were selected according to the number of landmark pixels and semantic information combined with experimental scenes for loop closure detection. In addition, dynamic landmarks were determined by scene content and landmark semantic information. In other words, according to the movement status of the landmarks in each experimental scene, we removed the moving landmarks in the dataset images and prevented them from participating in subsequent experiments. Formally, we denoted t as the number of distinctive landmarks selected in the image. (t was 5 or 10 in this work).

#### 2.1.3. CNN Features

CNN features have appearance invariance, so they far surpass manual features in the field of image retrieval and classification. AlexNet [49] won the 2012 ImageNet competition champion, and the network was pre-trained for object recognition tasks on the ILSVRC dataset [50].

The AlexNet network architecture had 8 layers, of which the first 5 layers were convolutional layers and the last 3 layers were fully connected layers. There was a pooling layer after the 1st, 2nd, and 5th convolutional layers, but there was none after the 3rd and 4th convolutional layers. Each convolutional layer had activation function ReLU and normalization. The input of the network was a 227 × 227 3-channel image, and the output feature of the third convolutional layer was 13 × 13 × 384 = 64.896. According to the research of [27], the output features of the third convolutional layer of AlexNet perform best under appearance changes. We found that the output features of the fully connected layer had strong semantic information that was robust to viewpoint changes but poor for appearance changes. At the same time, it was proved that the AlexNex obtained by pre-training in the object recognition task was better than the CNN model based on place recognition training when considering the characteristics of the entire image under the viewpoint changes. Other advanced networks such as VGG, ResNet, and DenseNet have complex architectures, as well as a lack of research and utilization in the field of loop closure detection. Therefore, this article used the relatively lightweight and mature AlexNet in the loop closure detection field to extract CNN features. Based on the above research, the proposed method employed the output of the Conv3 of the AlexNet as the global feature in the landmark region.

The landmark proposal extracted by the object proposal method contained a large amount of irrelevant feature information. This led to a certain amount of noise influence in the CNN feature description. However, the landmark area extracted by semantic segmentation in this paper only contained the landmark feature and no other unrelated features. Figure 4a–c is introduced in Section 2.1.2, this section explains the landmark area and contour extraction. The landmarks from the filtering result (see Figure 4c) were selected to get Figure 4d, and the contour binary (Figure 4e) of the corresponding landmark was obtained by a Canny operator. Then, the landmarks (see Figure 4d) were resized to 227 × 227 pixels and input to the pre-trained AlexNet to extract features. As a result, the features of each landmark could be represented by a 64.896-dimensional vector. In order to keep the original size information of the landmark, this paper added the Hu moment of the contour (see Figure 4e) to the CNN feature to describe the landmark.

The obtained high-dimensional vector contained redundant landmark feature information, and a large amount of computational cost was required to calculate landmark similarity. Due to the real-time requirements of visual SLAM, the Gaussian random projection method [51] utilized in [30] was employed to reduce the dimensionality of the feature vector to 2048 dimensions.

*Remote Sens.* **2020**, *12*, x 8 of 27

**Figure 4.** Extraction of landmark regions and contours: (**a**) raw image, (**b**) semantic segmentation image, (**c**) the result of filtering, (**d**) the selected landmark regions, and (**e**) the contours of the selected landmarks. **Figure 4.** Extraction of landmark regions and contours: (**a**) raw image, (**b**) semantic segmentation image, (**c**) the result of filtering, (**d**) the selected landmark regions, and (**e**) the contours of the selected landmarks.

#### The obtained high-dimensional vector contained redundant landmark feature information, and 2.1.4. Graph Representation

a large amount of computational cost was required to calculate landmark similarity. Due to the realtime requirements of visual SLAM, the Gaussian random projection method [51] utilized in [30] was employed to reduce the dimensionality of the feature vector to 2048 dimensions. 2.1.4. Graph Representation In order to preserve the spatial relationship of the scene, a semantic topology graph was constructed from a single image obtained by the robot. When the robot was initialized, the camera captured the first image and started to create the semantic topology graph. In this paper, each landmark was abstracted as a node containing category and pixel number information. Additionally, the node was located at the center of the landmark region.

In order to preserve the spatial relationship of the scene, a semantic topology graph was constructed from a single image obtained by the robot. When the robot was initialized, the camera captured the first image and started to create the semantic topology graph. In this paper, each landmark was abstracted as a node containing category and pixel number information. Additionally, the node was located at the center of the landmark region. The truncated random walk proposed by Perozzi et al. [52] was used to describe the semantic topological graph and represented each node as a fixed-length embedding vector. In order to enrich the feature expression, the node index and the number of pixels were adopted to describe the nodes. The node index was obtained according to the 150 semantic categories of the ADE20K dataset, and the number of pixels was acquired from the landmark region.

The truncated random walk proposed by Perozzi et al. [52] was used to describe the semantic topological graph and represented each node as a fixed-length embedding vector. In order to enrich the feature expression, the node index and the number of pixels were adopted to describe the nodes. The node index was obtained according to the 150 semantic categories of the ADE20K dataset, and the number of pixels was acquired from the landmark region. Then, the random walk descriptor of each node was calculated, and each node was used as the target node. In the semantic topological graph, the next adjacent node was randomly selected until the walking depth was reached and a random walking path was obtained. In this paper, random walks were performed on each node to obtain random walk paths. Finally, each target node could be expressed as a matrix = {} ∈ ℝ× . Since each node contained information about the index and the number of pixels, a matrix = {} ∈ ℝ×2 was finally obtained. In Then, the random walk descriptor of each node was calculated, and each node was used as the target node. In the semantic topological graph, the next adjacent node was randomly selected until the walking depth *n* was reached and a random walking path was obtained. In this paper, *m* random walks were performed on each node to obtain *m* random walk paths. Finally, each target node could be expressed as a matrix *M* = n *mij*o ∈ R*m*×*n* . Since each node contained information about the index and the number of pixels, a matrix *M* = n *mij*o ∈ R*m*×2*n* was finally obtained. In addition, the random walks followed certain rules, i.e., they would not repeat the same path and would not return to the previous nodes during each walk. The selection of *m* and *n* was related to the number of nodes in the graph structure. Based on the research of Gawel et al. [36] and the number of landmarks selected in Section 2.1.2, *m* was selected as 10, 20, or 50, and *n* was 3 or 5 for experiments.

Figure 5 shows an example of a constructing descriptor. After the image in Figure 5a was obtained by the robot, the landmark nodes were extracted to obtain Figure 5b. Then, the nodes in Figure 5b were connected by undirected edges according to the co-visibility information to get Figure 5c. The semantic topology graph (see Figure 5d) was used to describe the geometric connection of the nodes in Figure 5a. Furthermore, a random walk graph descriptor (see Figure 5e) was constructed according to the semantic topology graph (see Figure 5d). The blue node (car) was used here to construct a descriptor for the target node. For the purpose of illustration, a graph description matrix *M* ∈ R5×6 was made to represent the geometric characteristics of the image by five random walks with a depth of 3 each time. The last row of Figure 5e corresponds to the random walk path shown by the black arrow in Figure 5d. Figure 5f used a matrix to quantify the description of Figure 5e, where the red box represents the index of nodes and the number of pixels. The index of the target node car was 21, and 68,192 was the number of pixels contained in the car landmark. In the same way, 2 and 112,918 were the index and pixel number of the building node, respectively; 3 and 13,488 were the index and pixel number of the sky node, respectively; 5 and 10,697 were the index and pixel number of the tree node, respectively; and 7 and 101,572 were the index and pixel number of the road node, respectively. For visualization, some node information was omitted. experiments. Figure 5 shows an example of a constructing descriptor. After the image in Figure 5a was obtained by the robot, the landmark nodes were extracted to obtain Figure 5b. Then, the nodes in Figure 5b were connected by undirected edges according to the co-visibility information to get Figure 5c. The semantic topology graph (see Figure 5d) was used to describe the geometric connection of the nodes in Figure 5a. Furthermore, a random walk graph descriptor (see Figure 5e) was constructed according to the semantic topology graph (see Figure 5d). The blue node (car) was used here to construct a descriptor for the target node. For the purpose of illustration, a graph description matrix ∈ ℝ5×6 was made to represent the geometric characteristics of the image by five random walks with a depth of 3 each time. The last row of Figure 5e corresponds to the random walk path shown by the black arrow in Figure 5d. Figure 5f used a matrix to quantify the description of Figure 5e, where the red box represents the index of nodes and the number of pixels. The index of the target node car was 21, and 68,192 was the number of pixels contained in the car landmark. In the same way, 2 and 112,918 were the index and pixel number of the building node, respectively; 3 and 13,488 were the index and pixel number of the sky node, respectively; 5 and 10,697 were the index and pixel number of the tree node, respectively; and 7 and 101,572 were the index and pixel number of the road node, respectively. For visualization, some node information was omitted.

*Remote Sens.* **2020**, *12*, x 9 of 27

would not return to the previous nodes during each walk. The selection of and was related to

of landmarks selected in Section 2.1.2, *m* was selected as 10, 20, or 50, and was 3 or 5 for

**Figure 5.** Construction of the topology graph and extraction of the descriptor: (**a**) raw image, (**b**) creation of nodes, (**c**) connection of undirected edges, (**d**) construction of topology graph, (**e**) random walk descriptor, and (**f**) descriptor matrix. **Figure 5.** Construction of the topology graph and extraction of the descriptor: (**a**) raw image, (**b**) creation of nodes, (**c**) connection of undirected edges, (**d**) construction of topology graph, (**e**) random walk descriptor, and (**f**) descriptor matrix.

#### *2.2. Loop Closure Detection 2.2. Loop Closure Detection*

This section introduces the algorithm of loop closure detection, and its flowchart is shown in Figure 6. Firstly, dataset images (candidate images) that matched the current image (query image) were retrieved. Then, the appearance similarity was calculated according to the CNN and contour features between images. In addition, geometric similarity was obtained by using the random walk descriptor. Finally, loop closure detection was performed according to the overall similarity. This section introduces the algorithm of loop closure detection, and its flowchart is shown in Figure 6. Firstly, dataset images (candidate images) that matched the current image (query image) were retrieved. Then, the appearance similarity was calculated according to the CNN and contour features between images. In addition, geometric similarity was obtained by using the random walk descriptor. Finally, loop closure detection was performed according to the overall similarity.

#### 2.2.1. Obtain Candidate Images

When the robot enters a previous environment again, it needs to retrieve the candidate images from the dataset images. In this article, by controlling the number of landmarks that shared the same label between the query image and each image in the dataset, candidate images of the current query image were obtained. The smaller the number of shared nodes, the more candidate images and the longer the retrieval time, and vice versa. A reasonable setting of the number of shared nodes can improve the speed and accuracy of loop closure detection, so the number of shared nodes was set as 1 to obtain candidate images. In other words, when both the query image and an image in the dataset images have a landmark with the same label, the dataset image is considered to be one candidate image. According to the same principle, each query image can obtain a candidate image set, and each image *Remote Sens.*  in the candidate image set has at least one landmark (node) with the same label as the query image. **2020**, *12*, x 10 of 27

**Figure 6.** Flow chart of loop closure detection. **Figure 6.** Flow chart of loop closure detection.

#### 2.2.1. Obtain Candidate Images 2.2.2. Appearance Similarity

image.

When the robot enters a previous environment again, it needs to retrieve the candidate images from the dataset images. In this article, by controlling the number of landmarks that shared the same label between the query image and each image in the dataset, candidate images of the current query image were obtained. The smaller the number of shared nodes, the more candidate images and the longer the retrieval time, and vice versa. A reasonable setting of the number of shared nodes can improve the speed and accuracy of loop closure detection, so the number of shared nodes was set as 1 to obtain candidate images. In other words, when both the query image and an image in the dataset To calculate the appearance similarity between the candidate image and the current query image, it is necessary to match the landmark of the query image with all landmarks of the candidate image. By using the semantic information of landmarks, we employed the nearest neighbor search based on the cosine distance (see Equation (1)) of CNN features to match the landmark pairs of the same label in the two images so that only the landmarks of the same category were matched to speed up the matching process. In the matching process, we used a bidirectional matching method, i.e., landmark pairs were accepted only if they were mutual matches.

$$d\_{ij}^{\text{coosine}} = \frac{1}{2} \left( 1 - \frac{v\_i^q \cdot v\_j^c}{\|v\_i^q\| \|2\| \cdot v\_j^c\|\_2} \right) \tag{1}$$

) (1)

2.2.2. Appearance Similarity where *v q i* denotes the feature vector of the *i*-th landmark of the query image and *v c j* describes the feature vector of the *j*-th landmark of the candidate image.

To calculate the appearance similarity between the candidate image and the current query image, it is necessary to match the landmark of the query image with all landmarks of the candidate image. By using the semantic information of landmarks, we employed the nearest neighbor search based on the cosine distance (see Equation (1)) of CNN features to match the landmark pairs of the same label in the two images so that only the landmarks of the same category were matched to speed While calculating the similarity of the CNN features, the geometric shape of the landmark was introduced as a penalty factor to eliminate the false positive phenomenon, i.e., the CNN features were similar but the contours were different. The references [30,31,53] used the difference between the long side and the wide side of the region proposal of the landmark pair to measure the shape difference. However, because they only used the long side and wide side difference of the bounding

 ⋅

∥ ∥2∥ ∥2

1 2 (1 −

 cosine =

up the matching process. In the matching process, we used a bidirectional matching method, i.e.,

box, the influences of the scale and rotation was omitted. When the viewpoint changed drastically, the rotation of the landmark caused a large change in the aspect ratio of the bounding box. In the end, the shape penalty factor was too large, resulting in a low appearance similarity.

In order to solve the above problems, Hu moments [54] were used to describe the irregular contour features of landmarks, which possessed invariance about rotation, translation, and scale. Due to the wide range of Hu moments, the logarithm method was used for data compression in order to facilitate comparison. At the same time, considering that the Hu moment may have a negative value, absolute value was taken before the logarithm, as shown in Equation (2):

$$\mathbf{c}\_{i} = \text{sign}(\text{hu}\_{i}) \text{log} |\text{u}\_{i}| \text{ } i = 1, 2, \dots, 7 \tag{2}$$

where *sign*(*x*) is the sign function.

Hu [54] constructed seven invariant moments to describe geometric shape. Therefore, each landmark contour could be expressed as a feature vector by seven Hu moment values through Equation (3):

$$\mathbf{C} = (\mathbf{c}\_1, \mathbf{c}\_2, \mathbf{c}\_3, \mathbf{c}\_4, \mathbf{c}\_5, \mathbf{c}\_6, \mathbf{c}\_7) \tag{3}$$

Through the contour feature vector, the shape difference of landmark contour between query image and candidate image could be calculated by Equation (4):

$$\gamma\_{ij} = \exp\left(\max\_{m=1\dots\mathcal{T}} \frac{\left|\mathbf{c}\_m^{qi} - \mathbf{c}\_m^{cj}\right|}{\left|\mathbf{c}\_m^{qi}\right|}\right) \tag{4}$$

where *c qi m* and *c cj m* denote the *m*-th Hu moment of the *i*-th landmark contour in query image and the *m*-th Hu moment of the *j*-th landmark contour in candidate image, respectively.

According to cosine distance of the CNN feature and shape similarity obtained by the above calculation, the appearance similarity of the landmark pair between the query and the candidate images could be obtained by Equation (5):

$$d\_{\rm ij} = 1 - d\_{\rm ij}^{\rm cosine} \cdot \gamma\_{\rm ij} \tag{5}$$

In Equation (5), when the contour shape of the landmark is close, γ*ij* is close to 1. If γ*ij* is larger, it indicates that the contour difference of the landmark is large. In addition, when *dij* is close to 1, it means that the landmarks both have similar CNN features and geometric shapes. Furthermore, when *dij* is a negative number, it indicates that the geometric shapes of the landmarks differ greatly. If *dij* is small, it reveals that there may be differences in the CNN features or geometric shapes.

#### 2.2.3. Geometric Similarity

In visual SLAM, the accuracy of loop closure detection is particularly important. Therefore, it is necessary to consider both appearance similarity and geometric similarity during loop closure detection. Thus, the random walk graph descriptor proposed in Section 2.1.4 was used to calculate geometric similarity for graph matching. Denote that the vectorized form of the descriptor matrix *M* = n *mij*o ∈ R*m*×2*n* using *G* ∈ R2*mn* is a concatenation of the columns of *M* into a vector.

In Section 2.1.4, we obtained the random walk descriptor of the dataset images and only needed to construct the semantic topology graph to extract the descriptor for the query image. Since the number of pixels was much larger than the node index in value, the absolute size of the feature vector of the description changed greatly. Therefore, it was more appropriate to use cosine similarity to express the relative difference of graph descriptors by Equation (6):

$$\mathcal{S}\_{\mathcal{S}}\Big(\mathcal{G}\_{q\prime}\mathcal{G}\_{\mathcal{C}}\Big) = \frac{\mathcal{G}\_{q\prime}\mathcal{G}\_{\mathcal{C}}}{\|\!\|\!\!\!\!G\_{\prime}q\prime\|\_{2}\cdot\|\!\!\!\!G\_{\mathcal{C}}\|\_{2}}\tag{6}$$

where *Gq* and *Gc* denote the feature vector of the random walk descriptor in query image and candidate image, respectively. The denominator is the product of the corresponding vector modulus length. After getting a similarity score, it needed to be normalized with Equation (7):

$$\mathcal{S}\_{\mathcal{S}} = \frac{1}{2} + \frac{1}{2} \mathcal{S}\_{\mathcal{S}} \Big( \mathcal{G}\_{\eta}, \mathcal{G}\_{\mathcal{c}} \Big) \tag{7}$$

Through Equation (7), the similarity score in the range of [0, 1] could be obtained.

#### 2.2.4. Overall Similarity

This section discusses the calculation of the overall similarity between the query image and the candidate image. We not only considered the appearance characteristics of the image but also added geometric constraints. Sections 2.2.2 and 2.2.3 obtained the appearance and geometric similarities of a single landmark pair. Through Equations (8) and (9), we scored each best matched landmark pair *I i q* , *I j c* between the query image *Iq* and the candidate image *Ic*. The similarity score between each landmark *i* in the query image and the most similar landmark *j* selected by the nearest neighbor search method in the candidate image was first computed, and then the scores were assigned to the candidate image as the mean value of individual scores of its landmarks. Finally, through Equation (10), the overall similarity score of each candidate image was obtained.

$$
\hat{S}\_{q,\mathcal{E}} = \frac{1}{t} \sum\_{i,j} d\_{ij} \tag{8}
$$

$$
\hat{\mathbf{S}}\_{\mathcal{S}} = \frac{1}{t} \sum\_{i,j} \mathbf{S}\_{\mathcal{S}} \tag{9}
$$

where *t* denotes the number of the landmarks in the candidate image *Ic* (including unmatched landmarks), *i* represents the *i*-th landmark of the current query image, and *j* is the most similar landmark selected by the nearest neighbor search method in the candidate image. Moreover, the sum is done only on the best matched landmark pairs selected by the nearest neighbor search method.

$$\mathcal{S}\_{\text{all}} = \mathcal{S}\_{\text{g}} \mathcal{S}\_{\text{g}\mathcal{L}} \tag{10}$$

In Equation (10), geometric similarity *S*ˆ *g* is used as a penalty factor for appearance similarity score to filter candidate images with similar local features but large differences in geometric information. In the experiments, it was normalized to [0, 1]. We normalized a set of overall similarity scores between the current query image and all candidate images (in Section 2.2.1). When the overall similarity score set of the current query image be *X* = {*x*1, *x*2, . . . , *xm*}, *m* denotes the number of candidate images retrieved from the current query image. In addition, *xi* (1 < = *i* < = *m*) denotes the overall similarity score between the current query image and the *i*-th candidate image. Through Equation (11), each value in *X* can be normalized to [0, 1]:

$$y\_i = \frac{x\_i - \min(X)}{\max(X) - \min(X)} \tag{11}$$

where *Y* = *y*1, *y*2, . . . , *ym* is the normalized score set and *yi* is one of the score values.

After obtaining the normalized similarity score for loop closure detection, it is often necessary to perform time and space consistency verification. In this article, the geometry check was not added. Nevertheless, the proposed method that integrates visual, spatial, and semantic information was still found to improve performance.


**4. Conclusion**

The efficiency of the acetic acid fermentation process can be assessed using a simplified system with mixed cultures. Some of the aspects evaluated in this system were the availability of sugar in the medium, the inoculum ratio of the cultures used, and the speed of agitation. By optimizing this system, the resulting acetic acid levels can be increased.