**5. Hox genes and adult stem cells**

Hox genes act as transcriptional regulators, which have been involved in the differentiation of stem cells into several lineages and different cell types. One of the main steps to initiate vasculogenesis and angiogenesis is the differentiation to endothelial lineage from pluripotent stem cells. Studies have suggested that Hox genes contribute to the differentiation of EPCs into mature endothelial cells (**Table 1**). In the next section, we will present the evidence for the role of Hox genes in the differentiation of adult stem cell.


**1. Introduction**

Symmastia after breast implants is a rare and highly recurrent complication [1]. The incidence is unknown [2]. In 1983, Symmastia (from Greek syn - together, masto - breast) was defined for the first time as a confluence of the mammary borders through the midline and loss of the intermammary fold [3]. Symmastia can be congenital or acquired, when it is due to the poor positioning of the implants that cross the midline. The first reference for correction of acquired symmastia was in 1988 [4]. The acquired symmastia can be monocapsular (**Figure 1(a)**) (when the implant pocket communicate becoming only one), or bicapsular (**Figure 1(b)**) (when there are two capsules, one or both implants can be medialized and there is a loss of insertion of the midline adhesions) [5]. It is considered severe when both sides of the capsule need to be corrected. When the implants are medialized, but with the medial

#### **Figure 1.**

*(a) Representation of retromuscular monocapsular symmastia. (b) Representation of retromuscular bicapsular symmastia. (c) Representation of simasty correction with central membership points and making new retro pectoral pocket.*

sternal fascia intact, the term "medial malposition" is preferred [5]. Khan [6] suggests an etiological classification for the types of symmastia (**Table 1**). In this work, we describe three cases of patients with symmastia after breast reconstruction who were treated surgically with an integration of techniques.

#### **Table 1.**

*Khan's etiological classification of symmastia.*

#### **2. Objective**

To report an integration of techniques for the surgical treatment of symmastia after breast implant reconstruction, also indicated for the treatment of symmastia after breast augmentation.

#### **3. Methods**

For the discussion of this report, a bibliographic survey was carried out in the Medline and Cochrane Database, using the descriptors "symmastia", "breast reconstruction" and "surgical treatment". All articles were reviewed, with inclusion only of those that referred to acquired symmastia.

#### **4. Procedure**

The patients underwent a surgical procedure under general anesthesia, and all were discharged the next day. Initially, the desired breast furrow was marked with methylene blue, as well as the midline and the inner edge of the breasts to be reconfected.

The prostheses and/or expanders were removed through a scar in the breast crease (previous or not) or by the scar from the mastectomy.

The midline region was scarified with a small curette or electrocautery, followed by adhesion points with non-absorbable threads: at least three points on the midline with 2–0 mononylon, following previous demarcation and a sequence of at least three points for delimitation of the new medial margin of the breasts (on each side of the store).

Then, the anterior capsule flap was made. Decreasing the power of the electrocautery, the anterior capsule of the pectoralis major muscle was detached, which was folded, in order to make a neo pectoral pocket. In two cases, partial capsulectomy was performed and material was sent for anatomopathological examination. The anterior capsule was fixed to the posterior by some points of mononylon 2–0 to close the previous pocket and inferiorly it was fixed with several points on the lower edge of the previous space. This prevented the accumulation of liquids (seroma/hematoma) in the old pocket, as well as sliding movements of the new prosthesis over the previous space. Thus, the previous area was completely closed and the manufactured pocket can be expanded laterally, inferiorly or superiorly, according to the need of each case.

Finally, new breast implants were placed, suction drains were inserted and the usual synthesis of the tissue planes was carried out (**Figure 1(c)**). The drains were removed on an outpatient basis, after 2 to 5 days.

There were no major complications. Patients 1 and 3 required drainage for a longer time due to high blood flow rate, and patient 2 used antibiotics for 2 weeks in the postoperative period, due to a slightly hyperemic irradiated breast.

#### **5. Results**

Case 1: Patient operated on 03/15/2017 presenting mono capsular symmastia (previously submitted to bilateral mastectomy in another service, with resection of the areola on the right and resection of inverted T skin on the left; placement of an anatomical expander on the right 450 cc and prosthesis anatomical left of 490 cc textured and evolving at the time with partial necrosis of flaps on the left). Right breast

previously irradiated. Symmastia correction was performed (there was complete communication of 6 cm vertically in the pre-sternal region) as described and placement of 455 cc microtextured super high profile prostheses and correction of left breast scars. **Figures 2**–**5**. There was no recurrence of symmastia, but she evolved with distant and locoregional tumor recurrence on 03/03/2019, and death on 07/2020 **Table 2**.

Case 2: Surgery performed on 23/05/2018. Patient underwent bilateral mastectomy in another service with resection of the nipple areola complex on the right and implant placement - expander prosthesis on the right 460 (Becker 35) and 390 anatomic prosthesis on the left. She presented bicapsular symmastia, separated only by a capsule beam, with complete loss of sternal adhesion. The implants were changed to ultra-high profile round prostheses 590 cc on the right and high profile 450 cc on the left. On that occasion, fat grafting was also performed on the right breast (70 cc). **Figures 6**–**11**. Subsequently, 120 cc lipografting was performed on the right on 7/7/2019 and the reconstruction of the areomamilar complex (CAM) on 7/2020 was reconstructed. Right breast previously irradiated **Table 2**.

**Figure 2.** *Case 1: Preoperative.*

**Figure 3.** *Case 1: Immediate postoperative period.*

**Figure 4.** *Case 1: Postoperative 5 months.*

**Figure 5.** *Case 1: Postoperative 3 years.*

#### **Table 2.**

*Summary of cases regarding the type of simastia, implants before and after correction of the problem, association of fat graft during implant replacement and anatomopathological examination of the implant capsule.*

**Figure 6.** *Case 2: Preoperative.*

**Figure 7.** *Case 2: Immediate postoperative period.*

**Figure 8.** *Case 2: Postoperative 3 weeks.*

Case 3: Surgery performed on 06/2020. Patient previously submitted to bilateral mastectomy in another service with resection of the nipple-areola complex on the right and placement of 440 textured anatomical implants. Bicapsular symmastia, separated only by the capsules, and disinsertion of the skin of the pre-sternal region. Correction of symmastia and placement of 455 cc super high profile microtextured round prostheses and fat grafting 110 cc on the right and 60 cc on the left. Subsequently, a new fat graft was performed (40 cc on the right and 30 cc on the left) with reconstruction of the CAM on the right and correction of scars on the left **Figures 12**–**18** and **Table 2**.

**Figure 9.** *Case 2: Postoperative 1 year.*

**Figure 10.** *Case 2: Postoperative period 1 year and 8 months.*

**Figure 11.** *Case 2: Postoperative 2 years and 6 months.*

**Figure 12.** *Case 3: Preoperative.*

**Figure 13.** *Case 3: Intraoperative detail: Marking with methylene blue for adhesion points.*

#### **Figure 14.** *Case 3: Intraoperative detail: Anterior capsule dissection.*

**Figure 16.** *Case 3: Immediate postoperative period.*

**Figure 17.** *Case 3: Postoperative 1 month.*

**Figure 18.** *Case 3: Postoperative 9 months.*

#### **6. Discussion**

There are not many case series in the literature on correction techniques for symmastia [2] discriminating symmastia after breast reconstruction.

The most likely cause of retropectoral symmastia is exaggerated dissection in the medial region of the breast [2, 7–9]. Other facts can corroborate. Some patients have the insertion of the pectoralis major muscles as thin as 3 to 4 mm at their origin, along the sternum from the 2nd to the 5th ribs, predisposing to this complication after placement of retromuscular implants. A wrong elevation of the pectoralis minor muscle (whose insertion may be less than 1 cm from the insertion of the pectoralis major by 24%, according to anatomical study [10]), with a consequent poor positioning of the implants, would result in a medial force vector for the implant [10, 11]. This fact, added to the excessive medial dissection, with involvement of the internal mammary artery and perivascular fibers, in patients with fine insertion of the pectoralis major muscle would stimulate the sliding of the implants to the midline, with dehiscence of the sternal pectoralis major muscle [11]. However, Hammond [7] disagrees with this because the pectoral muscle would always push the prostheses up and out toward the armpits and

basically credits the symmastia to a technical error. In the case of breast reconstruction, the use of expanders could favor the rupture of the medial fibers of the pectoral [12].

Excessive size and/or wide-base implants would also favor symmastia [1, 8, 13].

Recurrence after treatment of symmastia is a common problem [14]. Therefore, the integration of techniques is suggested in this work.

The simple removal of the implants and reconstruction in a second step would be the safest and most simple technique and preferred by some authors [9, 15]. However, this resource is usually left for recurring cases.

Basically, capsulotomy/capsulorrhaphy, sternal dermal adhesions, creation of a new pocket and integration of techniques (including muscle repair and use of dermal matrices) are described for the correction of symmastia [2].

The incision for approaching the implants in this study was the previous mastectomy scar (case 1) or in the breast crease. Scars in the pre-sternal area or intermammary fold must be avoided due to the poor esthetic result and the possibility of keloids [13].

Regarding adhesion techniques, after removing the implants and checking whether the symmastia was mono or bicapsular, the central region (intermammary groove) was scarified, and non-absorbable points were given between the central anterior and posterior capsule and medially to delimit the breasts, similar to Pavelecini et al. [13]. However, differently from these authors, it was decided to create a new pre-capsular space for the implant through the folding of the anterior capsule, maintaining the retropectoral space, to guarantee the correct position of the prostheses. Becker [14], in a series of 5 cases for correction of symmastia, shows only one case after breast reconstruction. It suggests central adhesion in the sternum with non-absorbable threads after anterior and posterior capsulectomy, and use of the same implant store, but with replacement for adjustable implants which were gradually filled in 1 week after surgery. It presents good results, but it is a more expensive technique (due to the use of expanding prostheses).

For the construction of a neopectoral pocket, in the reported cases, the periprosthetic capsules were "mature". Very thin capsules would make it difficult or technically make the procedure impossible, but this is rare [9]. In cases of breast reconstruction, the most usual is a relatively thick capsule, especially in irradiated cases, and this fact allowed the creation of this new retromuscular space, in the described cases, without difficulty. The new implant was repositioned, but there would also have been a contracture correction, as suggested by some authors, if it had previously been so. Furthermore, it also allows the capsules to be studied, as in cases 1 and 3. Creating a new space is technically less difficult and more accurate than trying to reduce the old space with sutures only [9, 16]. It also minimizes trauma and optimizes the size and position of the new space [17]. Changing implants while maintaining the same previous space as symmastia is probably a mistake because it would put greater pressure on the lines of correction sutures [16, 18]. This allowed that, in the reported cases, the implants were increased in size, as the patients requested it. Care was taken to use prostheses with greater projection and smaller diameter. So there was no problem in creating a larger pocket.

Regarding the technique of the neopectoral pocket, the fact that the lateralized muscle is not repaired, which could favor the force vector to push the implant medially [11], has been questioned. However, with the integration of techniques, medial capsulorrhaphy would totally or partially solve this problem.

The inability to make a new retromuscular pocket or the presence of visibly calcified capsules [9] or sick capsules (intense inflammatory process) would contraindicate the technique of making the new pectoral pocket.

In relation to other techniques, simple capsulorrhaphy, which consists of suturing between the anterior and posterior edges of the medial.

medial capsule, is not recommended due to high recurrence [19, 20]. Lateral or upper capsulectomy was not indicated in the reported cases, since with the change of plan, there was no healing restriction for the placement of implants. Bostwick suggests the correction of symmastia with excision of the central capsule and capsulorrhaphy of the flaps of the central capsules, associated with partial capsulectomies/lateral capsulotomies, in addition to textured implants for better adherence. Zingaretti et al. [21] describe a complex technique involving capsulectomy in the medial quadrant, adhesion points and medial capsulorrhaphy through transcutaneous points on the entire edge of the breast, but without changing the plan for placing the implants, which are necessarily smaller in order not to tension capsulorrhaphy points. Out of 10 retromuscular cases, it discriminates only 1 case after breast reconstruction.

Among the changes to a new pocket, the change from the submuscular plane to the subcutaneous plane, although suggested by some authors [8, 11] can be a less interesting option in many cases of primary cosmetic augmentation surgery, due to the possibility of palpation of the implant, capsular contracture or rippling development [16]. This would happen much more likely in cases of breast reconstruction. In addition, as in cases 2 and 3, it is possible to associate fat grafting in the same surgical procedure (for refinement of breast reconstruction). The option using acellular dermal matrices (ADM) to cover implants in the subcutaneous tissue would be an excellent option, but the cost is actually much higher, sometimes contraindicating the procedure. The change from the submammary to submuscular plane in cases of pre-pectoral symmastia is also widely performed [6].

Among the associated techniques, the use of dermal matrices facilitates and increases the possibility of correcting implant misplacement, in addition to integrating the matrix with the tissue [12]. However, the costs are high. Spear et al. [22], reported good results in their series of cases with the use of ADM used in breast reconstruction reviews: 5 cases of symmastia that were corrected with capsulorrhaphy with dermal graft medially (graft "onlay") or associated with confection change in the neo pectoral pocket, to avoid changing the implant to the old plan. Similarly, other authors also use a medial sling of the ROM to correct symmastia in cases of breast reconstruction [12, 23, 24] or cosmetic mammoplasty [25].

Parsa et al. [9] suggest anterior and posterior capsulectomy in the central region for its obliteration, and confection of a limited posterior capsule flap, followed by anterior capsulorrhaphy to make a new mammary board. This technique also depends on a posterior mature capsule (this dissection is already technically more difficult [9] in cases of retromuscular implants) and could also be used in retromammary stores. Foustanos and Zavrides [19] in only one case described capsulotomy and medial caspulorrhaphy associated with transcutaneous points to define the medial edges of the breasts.

The option for microtextured silicone prostheses was indicated, in principle, due to greater adherence [1] in relation to smooth implants (nanotexture). However, as a new pocket is made, these prostheses can also be used. Polyurethane prostheses could be used, especially in cases of subglandular or subfascial revision, since they do not undergo rotation or displacement, can improve rippling and capsular contracture [18]. Castello et al. [18] indicate polyurethane prostheses in the neosubpectoral pocket only in refractory cases of contracture.

As for the postoperative period, Kalaria et al. suggest that the patient rest between 4 and 6 weeks for abduction and lateral rotation of the humerus [11] due to the repositioning of the pectoralis major muscle. However, if the correction is adequate

as suggested in the reports, such prolonged rest can be debilitating [7]. Therefore, conventional postoperative rest and the use of a bra with medial support in the adhesion area for 3 months were indicated to stabilize the surgical correction [2].

The only meta-analysis found [2] in relation to symmastia, although it does not reach conclusions about which surgical techniques are more efficient to correct it, describes an algorithm for both congenital and acquired symmastia. It advises the evaluation of the capsule as a more effective surgical procedure and, eventually, the combination with a muscle repair (which was done in this series of cases). The use of ADM would be reserved in cases of recurrence or for patients who desire large volume implants. The metaanalysis concludes by advising an integration of techniques to bring better results, as described in this article.

Finally, to avoid the occurrence of symmastia, Sanchez et al. [10] advise that when the access is inframammary, in addition to careful medial dissection, to start the elevation of the pectoralis major muscle medially and proceed laterally, with direct muscular visualization so that there is no elevation of the pectoralis minor muscle, since its fibers may be close to or intertwined with the costal insertion of the pectoralis major. The choice of the size/base of the implants in relation to the patient's chest is also essential, following directions from Selvaggi et al. [15] and Tebbetts [26].

#### **7. Conclusion**

Here we describe an integration of techniques for correction of retromuscular symmastia resulting from bilateral breast reconstruction with implants, namely, points of adhesion in the pre-sternal region and alteration of the breast implant pocket with anterior capsule flap, maintaining subpectoral space. The technique is easily reproducible and versatile, and can be used to correct other misplaced breast prostheses, as well as to correct capsular contractures. Furthermore, it can be associated with lipotransfer, usually beneficial for esthetic refinement in cases of breast reconstruction.

There were no displacements of the implants in the postoperative period or capsular contractures so far, and the patients reported being satisfied.

#### **Author details**

Márcia Balbina Lorenzo Hoyos Rede de Assistência à Saúde Metropolitana, Sarandi-Paraná, Brazil

\*Address all correspondence to<EMAIL_ADDRESS>
© 2023 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


*Edited by Matthew Reyes*

This Edited Volume "Consumer Behavior and Marketing" is a collection of reviewed and relevant research chapters, offering a comprehensive overview of recent developments in the field of psychology. The book comprises single chapters authored by various researchers and edited by an expert active in the research area. All chapters are complete in itself but united under a common research study topic. This publication aims at providing a thorough overview of the latest research efforts by international authors and open new possible research paths for further novel developments.

ISBN 978-1-78923-855-6

Consumer Behavior and Marketing

Published in London, UK © 2020 IntechOpen © Shubhashish5 / iStock

## Consumer Behavior and Marketing

*Edited by Matthew Reyes*

## Consumer Behavior and Marketing

*Edited by Matthew Reyes*

Published in London, United Kingdom

### *Supporting open minds since 2005*

Consumer Behavior and Marketing http://dx.doi.org/10.5772/intechopen.77647 Edited by Matthew Reyes

#### Contributors

Abigail Chivandi, Mammo Muchie, Michael Olorunjuwon Samuel, Farruh Kushnazarov, Urška Starc Peceny, Simon Mokorel, Vesna Kuralt, Jurij Urbančič, Tomi Ilijas, Alireza Ziaei-Bideh, Mahsa Namakshenas-Jahromi, Umut Ayman, Anıl Kemal Kaya, İpek Halim, Zul Ariff Abdul Latiff, Marlene Amorim, Mario Rodrigues, Nazri Nordin, Mohamed Azmi Ahmad Hassali, Matthew Dingee

#### © The Editor(s) and the Author(s) 2020

The rights of the editor(s) and the author(s) have been asserted in accordance with the Copyright, Designs and Patents Act 1988. All rights to the book as a whole are reserved by INTECHOPEN LIMITED. The book as a whole (compilation) cannot be reproduced, distributed or used for commercial or non-commercial purposes without INTECHOPEN LIMITED's written permission. Enquiries concerning the use of the book should be directed to INTECHOPEN LIMITED rights and permissions department (permissions@intechopen.com).

Violations are liable to prosecution under the governing Copyright Law.

Individual chapters of this publication are distributed under the terms of the Creative Commons Attribution 3.0 Unported License which permits commercial use, distribution and reproduction of the individual chapters, provided the original author(s) and source publication are appropriately acknowledged. If so indicated, certain images may not be included under the Creative Commons license. In such cases users will need to obtain permission from the license holder to reproduce the material. More details and guidelines concerning content reuse and adaptation can be found at http://www.intechopen.com/copyright-policy.html.

#### Notice

Statements and opinions expressed in the chapters are these of the individual contributors and not necessarily those of the editors or publisher. No responsibility is accepted for the accuracy of information contained in the published chapters. The publisher assumes no responsibility for any damage or injury to persons or property arising out of the use of any materials, instructions, methods or ideas contained in the book.

First published in London, United Kingdom, 2020 by IntechOpen IntechOpen is the global imprint of INTECHOPEN LIMITED, registered in England and Wales, registration number: 11086078, 7th floor, 10 Lower Thames Street, London, EC3R 6AF, United Kingdom Printed in Croatia

British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library

Additional hard and PDF copies can be obtained from<EMAIL_ADDRESS>
Consumer Behavior and Marketing Edited by Matthew Reyes p. cm. Print ISBN 978-1-78923-855-6 Online ISBN 978-1-78923-856-3 eBook (PDF) ISBN 978-1-78985-468-8


**Acknowledgements**

This work was supported by Nihon University Multidisciplinary Research Grant for (2020) and supported by Research Institute of Science and Technology Nihon University College of Science and Technology Leading Research Promotion Grant. Also, the part of the work was supported by JSPS KAKENHI Grant Number JP18K04060. The authors appreciated the Nihon University Robotics Society (NUROS).


**2. Physiological role of metalloproteinases**

MMP-1 (collagenase 1) hydrolyzes collagen types I, II, III, VII, VIII, X and XI, as well as gelatin, fibronectin, vitronectin, laminin, tenascin, aggrecan, links protein, myelin basic protein and versican. MMP-2 (gellatinase) degrades collagen types I, II, III, IV, V, VII, X and XI, gelatin, elastin, fibronectin, vitronectin, laminin, entactin, tenascin, SPARC and aggrecan, links protein, galectin-3, versican, decanin and myelin basic protein. One of the most important differences between theses two metalloproteinases is the possibility of the hydrolysis of elastin and collagen type IV by MMP-2, but not by MMP-1. Researches have also focused their interest on MMP-9 which can degrade collagen types IV, V, VII, X and XIV, fibronectin, laminin, nidogen, proteoglycan link protein and versican.

For a long time metalloproteinases have been vied solely as enzymes of matrix proteins breakdown. Results of researches performed in recent years indicate that there is a group of

© 2012 Siemianowicz, licensee InTech. This is an open access chapter distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2012 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

non-matrix proteins which can be substrates for various MMPs. Metalloproteinases are involved in the activation of latent forms of effective proteins. For example, MMP-2, MMP-3 and MMP-9 can activate interleukin 1 (IL-1). They can also act on active cytokines, IL-1 undergoes subsequent degradation catalyzed by MMP-3. Metalloproteinases can alter cell surface proteins such as receptors and act on microbial peptides.

Metalloproteinases are not indiscriminately released by cells. They are secreted to or anchored to cell membrane. MT-MMPs have a specific transmembrane domain placing them in a certain position. Other metalloproteinases can be bound by specific cell-MMP interactions. This phenomenon allows an exact localization of their proteolytic activity [1,2].


Evaluation of PV-Wind Hybrid Energy System for a Small Island DOI: http://dx.doi.org/10.5772/intechopen.85221

sensitivity variables at which optimal system solutions have been obtained. Project life of system A (15 years) is less than that of system B (25 years), which is also one of the reasons for low NPC of system A.

Table 6 displays the selected size of each component for both systems. Both systems consist of one wind turbine and system converter of almost 1000 kW size. PV panel size for system B (3,157 kW) is higher than system A (2,504 kW), that is why NPC of system B is higher than system A. By selecting an appropriate model of wind turbine according to the wind conditions of Deokjeokdo island, both system architectures might be different from present cases. But, a right choice of wind

• System A: HRES with lowest overall net present cost (NPC)

Wind Solar Hybrid Renewable Energy System

• System B: HRES with lowest overall levelized cost of energy (LCOE)

Table 5 shows the basic characteristics of both of the optimized system solutions. It is to be noted that both systems have batteries as default option for storing surplus electricity. System A has the lowest overall NPC (11.3 million \$) whereas LCOE is lowest in case of system B (\$ 0.123). Table 5 also displays the values of

Average load (kWh/day) Discount rate (%) Project lifetime (years)

Variable Unit System A System B NPC Million \$ 11.3 17.61 LCOE \$/kWh 0.158 0.123 Total load scaled average kWh/day 20,000 20,000 Nominal discount rate % 8 4 Project lifetime years 15 25

Component Model Unit Size (system A) Size (system B)

Leonics MTP-413F 25 kW kW 1006 1009

HOMER cycle charging N/A N/A N/A

Battery Surrette 6 CS 25P Strings 7197 6269 Wind turbine STX 93/2000 ea. 1 1

kW 2504 3157

24,720 8 25 20,000 6 20 30,000 4 15

Breakdown of multiple solutions obtained from HOMER simulations.

Table 4.

Figure 16.

Table 5.

System converter

Dispatch strategy

Systems architecture.

Table 6.

176

Basic information about both system solutions.

PV panels Canadian Solar Max Power CS6X-

325P

Sensitivity variables.

Figure 17. Daily PV power output for both systems (a) system A (b) system B.

turbine depends on the detailed wind data analysis of local site, which is beyond the scope of current study; therefore not performed here.

of the other alternate system solutions on the basis of economic evaluations. In order to achieve this goal, Figure 18 has been prepared which shows multiple

Figure 18 shows a total of 27 optimal system solutions obtained by varying the

The present study provides a basic information about the working methodologies of a hybrid renewable energy system (HRES) consisting of wind and solar as primary energy resources. Two case studies of HRES have also been included to

First case study deals with the analysis of a small HRES consisting of wind turbines and PV panels with batteries as energy storage system (ESS). This small HRES is being installed at Deokjeokdo island in South Korea and its performance have been monitored for two consecutive years (2016 and 2017). Analysis showed that the prevailing wind direction at Deokjeokdo island is either north-east or south-west, with mean wind speed of 3.6 m/s at 10 m height. Similarly, average value of daily solar radiations was estimated to be 4.13 kWh/m2 with mean clearness index of 0.5. The total capacity of this small HRES is 6 kW; with two Darrieus

Second case study finds an optimum HRES to fulfill the yearly electricity demand of Deokjeokdo island, which corresponds to approximately 7.296 MWh/ year. Over 8760 simulations were performed to find out two optimum HRESs based on lowest NPC (system A) and lowest LCOE (system B), respectively. The overall NPC of system A was calculated to be 11.29 million USD, whereas for system B, it was 17.61 million USD. On the other hand, LCOE for system A was slightly higher than system B as it was 0.158 \$/kWh for system A and 0.123 \$/kWh for system B. Both systems can independently provide electricity to Deokjeokdo island through-

This study was supported with major project funding from the Korea Institute of Civil Engineering and Building Technology. We would also like to thank the KMA for providing long-term measured wind data at all proposed wind farm sites.

system solutions obtained by superimposing NPC over LCOE.

further clarify the economic aspects of such energy systems.

VAWTs of 1.5 kW size each and 3 kW size of PV panels.

out the year without any external assistance such as grid, etc.

The authors declare no actual or potential conflicts of interest.

values of all sensitivity variables mentioned in Table 4.

Evaluation of PV-Wind Hybrid Energy System for a Small Island

DOI: http://dx.doi.org/10.5772/intechopen.85221

4. Conclusions

Acknowledgements

Conflict of interest

179

Finally, Table 7 shows the annual amount of pollutant gases emissions due to operation of both systems.

Figure 17 displays the graphical representation of power produced by PV panels in both cases. Both the images of Figure 17 indicate that summer is the ideal season for harvesting energy from sun in South Korea, as the average day-time is almost 14–15 hours in Deokjeokdo island. The average hourly power generated by PV panels in case of system A is 425 kWh whereas this value corresponds to 536 kWh for system B.


**Chapter 8**

formation in fig liquors. Journal of the Science of Food and Agriculture. 2017; 97(15):5254-5270. DOI: 10.1002/

of Ficus carica on the activity of

[36] Das SS, Dey M, Ghosh AK.

of the leaf and bark extract of

[37] Trinchese F, Liu S, Zhang H, Hidalgo A, Schmidt SD, Yamaguchi H, et al. Inhibition of calpains improves memory and synaptic transmission in a mouse model of Alzheimer disease. The Journal of Clinical Investigation. 2008; 118(8):2796-2807. DOI: 10.1172/

[38] Richter G, Schwarz HP, Dorner F, Turecek PL. Activation and inactivation of human factor X by proteases derived from Ficus carica. British Journal of Haematology. 2002;119(4):1042-1051. DOI: 10.1046/j.1365-2141.2002.03954.x

104-107. DOI: 10.4103/ 0250-474X.89768

JCI34254

jfda.2017.03.001

enzymes related to metabolic syndrome. Journal of Food and Drug Analysis. 2018;26(1):201-210. DOI: 10.1016/j.

Determination of anthelmintic activity

Tamarindus indica Linn. Indian Journal of Pharmaceutical Sciences. 2011;73(1):

[29] Soni N, Mehta S, Satpathy G, Gupta

[30] Joseph B, Raj SJ. Pharmacognostic and phytochemical properties of Ficus carica Linn–An overview. International Journal of Pharmtech Research. 2011;

[31] Takahashi T, Okiura A, Saito K,

[32] Ammar S, del Mar Contreras M, Belguith-Hadrich O, Bouaziz M, Segura-Carretero A. New insights into the qualitative phenolic profile of Ficus carica L. fruits and leaves from Tunisia using ultra-high-performance liquid chromatography coupled to quadrupoletime-of-flight mass spectrometry and their antioxidant activity. RSC Advances. 2015;5(26):20035-20050.

phenylpropanoids in fig (Ficus carica L.) leaves. Journal of Agricultural and Food Chemistry. 2014;62(41):10076-10083.

Kohno M. Identification of

DOI: 10.1021/jf5025938

DOI: 10.1039/C4RA16746E

[33] Veberic R, Colaric M, Stampar F. Phenolic acids and flavonoids of fig fruit

mediterranean region. Food Chemistry. 2008;106(1):153-157. DOI: 10.1016/j.

[34] Santos C, Botelho G, Caldeira I, Torres A, Ferreira FM. Antioxidant activity assessment in fruit liquors and spirits: Methods comparison. Ciência e Técnica Vitivinícola. 2014;29(1):28-34.

DOI: 10.1051/ctv/20142901028

[35] Mopuri R, Ganjayi M, Meriga B, Koorbanally NA, Islam MS. The effects

(Ficus carica L.) in the northern

foodchem.2007.05.061

132

RK. Estimation of nutritional, phytochemical, antioxidant and antibacterial activity of dried fig (Ficus carica). Journal of Pharmacognosy and Phytochemistry. 2014;3(2):158-165

jsfa.8410

Modern Fruit Industry

3(1):8-12


Abstract

Although it is a very old theme, unconstrained optimization is an area which is always actual for many scientists. Today, the results of unconstrained optimization are applied in different branches of science, as well as generally in practice. Here, we present the line search techniques. Further, in this chapter we consider some unconstrained optimization methods. We try to present these methods but also to present some contemporary results in this area.

 Introduction

Optimization is a very old subject of a great interest; we can search deep into a human history to find important examples of applying optimization in the usual life of a human being, for example, the need of finding the best way to produce food yielded finding the best piece of land for producing, as well as (later on, how the time was going) the best ways of treatment of the chosen land and the chosen seedlings to get the best results.

From the very beginning of manufacturing, the manufacturers were trying to find the ways to get maximum income with minimum expenses.

There are plenty of examples of optimization processes in pharmacology (for determination of the geometry of a molecule), in meteorology, in optimization of a trajectory of a deep-water vehicle, in optimization of power management (optimization of the production of electrical power plants), etc.

Optimization presents an important tool in decision theory and analysis of physical systems.

Optimization theory is a very developed area with its wide application in science, engineering, business management, military, and space technology.

Optimization can be defined as the process of finding the best solution to a problem in a certain sense and under certain conditions.

Along with the passage of time, optimization was evolving. Optimization became an independent area of mathematics in 1940, when Dantzig presented the so-called simplex algorithm for linear programming.

The development of nonlinear programming became great after presentation of conjugate gradient methods and quasi-Newton methods in the 1950s.

#### Applied Mathematics

Today, there exist many modern optimization methods which are made to solve a variety of optimization problems. Now, they present the necessary tool for solving problems in diverse fields.

At the beginning, it is necessary to define an objective function, which, for example, could be a technical expense, profit or purity of materials, time, potential energy, etc.

The object function depends on certain characteristics of the system, which are known as variables. The goal is to find the values of those variables, for which the object function reaches its best value, which we call an extremum or an optimum.

It can happen that those variables are chosen in such a way that they satisfy certain conditions, i.e., restrictions.

The process of identifying the object function, variables, and restrictions for the given problem is called modeling.

The first and the most important step in an optimization process is the construction of the appropriate model, and this step can be the problem by itself. Namely, in the case that the model is too much simplified, it cannot be a faithful reflection of the practical problem. By the other side, if the constructed model is too complicated, then solving the problem is also too complicated.

After the construction of the appropriate model, it is necessary to apply the appropriate algorithm to solve the problem. It is no need to emphasize that there does not exist a universal algorithm for solving the set problem.

Sometimes, in the applications, the set of input parameters is bounded, i.e., the input parameters have values within the allowed space of input parameters Dx; we can write

$$x \in D\_x. \tag{1}$$

Except (1), the next conditions can also be imposed:

$$
\rho\_l(\mathbf{x}\_1, \dots, \mathbf{x}\_n) = \rho\_{\text{ol}}, l = \mathbf{1}, \dots, m\_1 \le n,\tag{2}
$$

$$
\psi\_j(\varkappa\_1, \ldots, \varkappa\_n) \le \psi\_{0j}, j = 1, \ldots, m\_2. \tag{3}
$$

Optimization task is to find the minimum (maximum) of the objective function f xð Þ ¼ f xð 1; …; xnÞ, under the conditions (1), (2), and (3).

If the object function is linear, and the functions φlðx1; …; xnÞ l ¼ 1, …, m1 and ψj ðx1; …; xnÞ j ¼ 1, …, m2 are linear, then it is about the linear programming problem, but if at least one of the mentioned functions is nonlinear, it is about the nonlinear programming problem.

Unconstrained optimization problem can be presented as

$$\min\_{\mathbf{x}\in\mathbb{R}^{n}} f(\mathbf{x}),\tag{4}$$

where f ∈Rn is a smooth function.

Problem (4) is, in fact, the unconstrained minimization problem. But, it is well known that the unconstrained minimization problem is equivalent to an unconstrained maximization problem, i.e.

$$\min f(\mathbf{x}) = -\max(-f(\mathbf{x})),\tag{5}$$

as well as

$$\max f(\mathbf{x}) = -\min(-f(\mathbf{x})).\tag{6}$$

Some Unconstrained Optimization Methods DOI: http://dx.doi.org/10.5772/intechopen.83679

Definition ∗ 1.1.1 x∗ is called a global minimizer of f if fð Þ ð Þ for all x∈Rn x ≤ f x .

The ideal situation is finding a global minimizer of f. Because of the fact that our knowledge of the function f is usually only local, the global minimizer can be very difficult to find. We usually do not have the total knowledge about f. In fact, most algorithms are able to find only a local minimizer, i.e., a point that achieves the smallest value of f in its neighborhood.

So, we could be satisfied by finding the local minimizer of the function f. We distinguish weak and strict (or strong) local minimizer.

Formal definitions of local weak and strict minimizer of the function f are the next two definitions, respectively.

Definition 1.1.2 x∗ is called a weak local minimizer of f if there exists a neighbor- ∗ hood N of x∗, such that fð Þ x ≤fð Þ x for all x∈ N.

Definition 1.1.3 x∗ is called a strict (strong) local minimizer of f if there exists a ∗ ∗ neighborhood N of x , such that fð Þ x , f xð Þ for all x∈ N.

Considering backward definitions 1.1.2 and 1.1.3, the procedure of finding local minimizer (weak or strict) does not seem such easy; it seems that we ∗ should examine all points from the neighborhood of x , and it looks like a very difficult task.

Fortunately, if the object function f satisfies some special conditions, we can solve this task in a much easier way.

For example, we can assume that the object function f is smooth or, furthermore, twice continuously differentiable. Then, we concentrate to the gradient ∇ ∗ f x as well as to the Hessian ∇2f x∗ ð Þ ð Þ.

All algorithms for unconstrained minimization require the user to start from a certain point, so-called the starting point, which we usually denote by x0. It is good to choose x0 such that it is a reasonable estimation of the solution. But, to find such estimation, a little more knowledge about the considered set of data is needed, and the systematic investigation is needed also. So, it seems much simpler to use one of the algorithms to find x0 or to take it arbitrarily.

There exist two important classes of iterative methods—line search methods and trust-region methods—made in the aim to solve the unconstrained optimization problem (4).

In this chapter, at first, we discuss different kinds of line search. Then, we consider some line search optimization methods in details, i.e., we study steepest descent method, Barzilai-Borwein gradient method, Newton method, and quasi-Newton method.

Also, we try to give some of the most recent results in these areas.

### 2. Line search

Now, let us consider the problem

$$\min\_{\mathbf{x}\in\mathbb{R}^n} f(\mathbf{x}),\tag{7}$$

where f : Rn ! R is a continuously differentiable function, bounded from below. There exists a great number of methods made in the aim to solve the problem (7).

The optimization methods based on line search utilize the next iterative scheme:

$$
\varkappa\_{k+1} = \varkappa\_k + t\_k d\_k. \tag{8}
$$

where xk is the current iterative point, xkþ1 is the next iterative point, dk is the search direction, and tk is the step size in the direction dk.

At first, we consider the monotone line search.

Now, we give the iterative scheme of this kind of search.

Algorithm 1.2.1. (Monotone line search). Assumptions: ϵ . 0, x0, k ≔ 0. Step 1. If ∥gk∥ ≤ϵ, then STOP. Step 2. Find the descent direction dk. Step 3. Find the step size tk, such that f xð k þ tkdkÞ , fð Þ xk . Step 4. Set xkþ1 ¼ xk þ tkdk. Step 5.  Denote

$$
\Phi(t) = f(\mathbf{x}\_k + t d\_k).
$$

Trying to solve the minimization problem, we are going to search for the step size t ¼ tk, in the direction dk, such that the next relation holds:

$$
\Phi(t\_k) < \Phi(\mathbf{0}).
$$

That procedure is called the monotone line search. We can search for the step size tk in such a way that the next relation holds:

$$f(\mathbf{x}\_k + t\_k d\_k) = \min\_{t \ge 0} f(\mathbf{x}\_k + t\_k d\_k), \tag{9}$$

i.e.

$$\Phi(t\_k) = \min\_{t \ge 0} \Phi(t), \tag{10}$$

or we can use the next formula:

$$t\_k = \min \left\{ t \left| \mathbf{g} \left( \mathbf{x}\_k + t d\_k \right)^T d\_k = \mathbf{0}, t \ge \mathbf{0} \right\} \right. \tag{11}$$

In this case we are talking about the exact or the optimal line search, where the parameter tk, which is received as the solution of the one-dimensional problem (10), is the optimal step size.

By the other side, instead of using the relation (9), or the relation (11), we can be satisfied by searching for such tk, which is acceptable if the next relation suits us:

$$f(\mathbf{x}\_k) - f(\mathbf{x}\_k + t\_k d\_k) \ge \delta\_k \ge \mathbf{0}.$$

Then, we are talking about the inexact or the approximate or the acceptable line search, which is very much utilized in the practice.

There are several reasons to use the inexact instead of the exact line search. One of them is that the exact line search is expensive. Further, in the cases when the iteration is far from the solution, the exact line search is not efficient. Next, in the practice, the convergence rate of many optimization methods (such as Newton or quasi-Newton) does not depend on the exact line search.

First, we are going to mention so-called basic and, by the way, very well-known inexact line searches.

Algorithm 1.2.2. (Backtracking).

1 Assumptions: xk, the descent direction dk, 0 , δ , 2 , η∈ð0; 1Þ. Step 1. t ≔ 1.

Some Unconstrained Optimization Methods DOI: http://dx.doi.org/10.5772/intechopen.83679

Step 2. While fðxk þ tdkÞ . f xð kÞ þ δ tg T k dk, t ≔ t � η. Step 3. Set tk ¼ t. Now, we describe the Armijo rule.

Rn Theorem 1.2.1. [1] Let f ∈C1 ð Þ and let dk be the descent direction. Then, there exists the nonnegative number mk, such that

$$f(\mathbf{x}\_k + \eta^{m\_k} d\_k) \le f(\mathbf{x}\_k) + c\_1 \eta^{m\_k} \mathbf{g}\_k^T d\_k,$$

where c1 ∈ ð0; 1Þ and η∈ð0; 1Þ. Next, we describe the Goldstein rule [2]. The step size tk is chosen in such a way that

$$\begin{aligned} f(\boldsymbol{\kappa}\_k + t\boldsymbol{d}\_k) &\leq f(\boldsymbol{\kappa}\_k) + \delta \operatorname{tg}\_k^T \boldsymbol{d}\_k, \\ f(\boldsymbol{\kappa}\_k + t\boldsymbol{d}\_k) &\geq f(\boldsymbol{\kappa}\_k) + (1 - \delta) \operatorname{tg}\_k^T \boldsymbol{d}\_k, \end{aligned}$$

where 0 , δ , 1 2 .

Now, Wolfe line search rules follow [3], [4]. Standard Wolfe line search conditions are

$$f(\varkappa\_k + t\_k d\_k) - f(\varkappa\_k) \le \delta t\_k \varrho\_k^T d\_k,\tag{12}$$

$$\mathbf{g}\_{k+1}^T d\_k \ge \sigma \mathbf{g}\_k^T d\_k,\tag{13}$$

where dk is a descent direction and 0 , δ≤σ , 1.

This efficient strategy means that we should accept a positive step length tk, if conditions (12)–(13) are satisfied.

Strong Wolfe line search conditions consist of (12) and the next, stronger version of (13):

$$|\mathbf{g}\_{k+1}^T d\_k| \le -\sigma \mathbf{g}\_k^T d\_k. \tag{14}$$

In the generalized Wolfe line search conditions, the absolute value in (14) is replaced by the inequalities:

$$
\sigma\_1 \mathbf{g}\_k^T d\_k \le \mathbf{g}\_{k+1}^T d\_k \le -\sigma\_2 \mathbf{g}\_k^T d\_k,\\
0 \le \delta \le \sigma\_1 \le 1, \sigma\_2 \ge 0. \tag{15}
$$

By the other side, in the approximate Wolfe line search conditions, the inequalities (15) are changed into the next ones:

$$
\sigma \mathbf{g}\_k^T d\_k \le \mathbf{g}\_{k+1}^T d\_k \le (2\delta - 1)\mathbf{g}\_k^T d\_k,\\
\mathbf{0} \le \delta \le \frac{1}{2}, \delta \le \sigma \le 1. \tag{16}
$$

The next lemma is very important.

Lemma 1.2.1. [5] Let f ∈C Rn ð Þ. Let dk be a descent direction at the point xk, and assume that the function f is bounded from below along the direction fxk þ tdkjt . 0g. Then, if 0 , δ , σ , 1, there exist the intervals inside which the step length satisfies standard Wolfe conditions and strong Wolfe conditions.

By the other side, the introduction of the non-monotone line search is motivated by the existence of the problems where the search direction does not have to be a descent direction. This can happen, for example, in stochastic optimization [6].

Next, some efficient quasi-Newton methods, for example, SR1 update, do not produce the descent direction in every iteration [5].

Further, some efficient methods like spectral are not monotone at all.

Some numerical results given in [7–11] show that non-monotone techniques are better than the monotone ones if the problem is to find the global optimal values of the object function.

Algorithms of the non-monotone line search do not insist on a descent of the object function in every step. But, even these algorithms require the reduction of the object function after a predetermined number of iterations.

The first non-monotone line search technique is presented in [12]. Namely, in [12], the problem is to find the step size which satisfies

$$f(\mathbf{x}\_k + t\_k d\_k) \le \max\_{0 \le j \le m(k)} f\left(\mathbf{x}\_{k-j}\right) + \delta t\_k \mathbf{g}\_k^T d\_k,$$

where mð Þ¼ 0 0, 0≤ m kð Þ≤ minfm kð Þþ - 1 1; Mg, for k≥ 1, δ∈ð0; 1Þ, where M is a nonnegative integer.

This strategy is in fact the generalization of Armijo line search. In the same work, the authors suppose that the search directions satisfy the next conditions for some positive constants b1 and b2:

$$\begin{aligned} \mathbf{g}\_k^T d\_k \le \mathbf{} \mathbf{g}\_1 \|\mathbf{g}\_k\|^2, \\ \|d\_k\| \le b\_2 \|\mathbf{g}\_k\|. \end{aligned}$$

The next non-monotone line search is described in [11].  The step size has to satisfy the next conditions:

$$f(\mathbf{x}\_k + \mathbf{t}\_k d\_k) \le \mathbf{C}\_k + \delta \mathbf{t}\_k \mathbf{g}\_k^T d\_k,\tag{17}$$

$$\log(\varkappa\_k + t\_k d\_k) \ge \sigma \mathbf{g}\_k^T d\_k. \tag{18}$$

The value ηk is chosen from the interval ½ηmin; ηmaxl and then

$$Q\_{k+1} = \eta\_k Q\_k + \mathbf{1},\\
\mathbf{C}\_{k+1} = \frac{\eta\_k Q\_k \mathbf{C}\_k + f(\mathbf{x}\_{k+1})}{Q\_{k+1}}.$$

Non-monotone rules which contain the sequence of nonnegative parameters f g ϵk are used firstly in [13], and they are successfully used in many other algorithms, for example, in [14]. The next property of the parameters ϵk is assumed:

$$
\epsilon\_k > 0, \quad \sum\_k \epsilon\_k = \epsilon < \infty,
$$

and the corresponding rule is

$$f(\varkappa\_k + t\_k d\_k) \le f(\varkappa\_k) + c\_1 t\_k \gpharpoonright d\_k + \epsilon\_k.$$

Now, we give the non-monotone line search algorithm, shortly NLSA, presented in [11].

#### Algorithm 1.2.3. (NLSA).

Assumptions: x0, 0≤ηmin ≤ηmax ≤ 1, 0 , δ , σ , 1 , ρ, μ . 0. Set C0 ¼ f xð Þ0 , Q0 ¼ 1, k ¼ 0.

Some Unconstrained Optimization Methods DOI: http://dx.doi.org/10.5772/intechopen.83679

Step 1. If ∥∇f xð Þk ∥ is sufficiently small, then STOP.

Step 2. Set xkþ1 ¼ xk þ tkdk, where tk satisfies either the (non-monotone) Wolfe conditions (17) and (18) or the (non-monotone) Armijo conditions: tk ¼ tkρhk , where tk . 0 is the trial step and hk is the largest integer such that (17) holds and tk ≤ μ.

Step 3. Choose ηk ∈ ½ηmin; ηmax�, and set

$$Q\_{k+1} = \eta\_k Q\_k + \mathbf{1},\\
\mathbf{C}\_{k+1} = (\eta\_k Q\_k \mathbf{C}\_k + f(\mathbf{x}\_{k+1})) / Q\_{k+1}.$$

Step 4. Set k ≔ k þ 1 and go to Step 1.

We can notice [11] that Ckþ1 is a convex combination of f xð Þ0 , fð Þ x1 , …, fð Þ xk . The parameter ηk controls the degree of non-monotonicity.

If ηk ¼ 0 for all k, then this non-monotone line search becomes monotone Wolfe or Armijo line search.

If ηk ¼ 1 for all k, then Ck ¼ Ak, where

$$A\_k = \frac{1}{k+1} \sum\_{i=0}^k f(\mathbf{x}\_i).$$

Lemma 1.2.2. [11] If ∇f xð Þk Tdk ≤ 0 for each k, then for the iterates generated by the non-monotone line search algorithm, we have f k ≤Ck ≤ Ak for each k. Moreover, if ∇fð ÞT xk dk , 0 and fð Þ x are bounded from below, then there exists tk satisfying either Wolfe or Armijo conditions of the line search update.

This study would be very incomplete unless we mention that there are many modifications of the abovementioned line searches. All these modifications are made to improve the previous results.

For example, in [15], the new inexact line search is described by the next way. ˜ °1 Let β ∈ð0; 1Þ, σ ∈ 0; ; let Bk be a symmetric positive definite matrix which 2 approximates ∇2f xk and sk ¼ � dT gTdk ð Þ k . The step size tk is the largest one in k Bkdk ˛ sk; skβ; skβ2 ; … ˝ such that

$$f(\varkappa\_k + td\_k) - f(\varkappa\_k) \le \sigma t \left[ \mathbf{g}\_k^T d\_k + \frac{1}{2} t d\_k^T B\_k d\_k \right].$$

Further, in [16], a new inexact line search rule is presented. This rule is a modified version of the classical Armijo line search rule. We describe it now.

Let g ¼ ∇f xð Þ be a Lipschitz continuous function and L the Lipschitz constant. Let Lk be an approximation of L. Set

$$\boldsymbol{\beta}\_k = -\frac{\mathbf{g}\_k^T d\_k}{L\_k \|\boldsymbol{d}\_k\|^2}.$$

Find a step size tk as the largest component in the set ˛ βk; βkρ; βkρ2… ˝ such that the inequality

$$f(\boldsymbol{\kappa}\_k + t\_k d\_k) \le f(\boldsymbol{\kappa}\_k) + \sigma t\_k \left( \boldsymbol{g}\_k^T d\_k - \frac{1}{2} t\_k \mu L\_k \left\| \boldsymbol{d}\_k \right\|^2 \right)^{\frac{1}{2}}$$

holds, where σ ∈ð0; 1Þ, μ∈ ½0; ∞Þ, and ρ∈ð0; 1Þ are given constants. Next, in [17], a new, modified Wolfe line search is given in the next way. Find tk . 0 such that

$$\begin{aligned} &f(\boldsymbol{\varkappa}\_{k} + \boldsymbol{t}\_{k}d\_{k}) - f(\boldsymbol{\varkappa}\_{k}) \leq \min\{\boldsymbol{\delta}\boldsymbol{t}\_{k}\boldsymbol{\mathcal{g}}\_{k}^{T}d\_{k}, -\boldsymbol{\gamma}t\_{k}^{2}||d\_{k}||^{2}\}, \\ &\boldsymbol{g}(\boldsymbol{\varkappa}\_{k} + \boldsymbol{t}\_{k}d\_{k})^{T}d\_{k} \geq \sigma \boldsymbol{\mathcal{g}}\_{k}^{T}d\_{k}, \end{aligned}$$

where δ∈ð0; 1Þ, σ ∈ðδ; 1Þ, and γ . 0. More recent results on this topic can be found, for example, in [18–23].

#### 2.1 Steepest descent (SD)

The classical steepest descent method which is designed by Cauchy [24] can be considered as one among the most important procedures for minimization of real-valued function defined on Rn.

Steepest descent is one of the simplest minimization methods for unconstrained optimization. Since it uses the negative gradient as its search direction, it is known also as the gradient method.

It has low computational cost and low matrix storage requirement, because it does not need the computations of the second derivatives to be solved to calculate the search direction [25].

Suppose that fð Þ x is continuously differentiable in a certain neighborhood of a point xk and also suppose that gk≜∇f xð Þk 6¼ 0.

Using Taylor expansion of the function f near xk as well as Cauchy-Schwartz inequality, one can easily prove that the greatest fall of f exists if and only if dk ¼ �gk, i.e., �gk is the steepest descent direction.

The iterative scheme of the SD method is

$$
\boldsymbol{\infty}\_{k+1} = \boldsymbol{\mathfrak{x}}\_{k} - \boldsymbol{t}\_{k} \, \mathbf{g}\_{k} \,. \tag{19}
$$

The classical steepest descent method uses the exact line search.

Now, we give the algorithm of the steepest descent method which refers to the exact as well as to the inexact line search.

Algorithm 1.2.4. (Steepest descent method, i.e., SD method).

Assumptions: 0 , ϵ ≪ 1, x0 ∈ Rn. Let k ¼ 0.

Step 1. If ∥ gk∥ ≤ε, then STOP, else set dk ¼ �gk.

Step 2. Find the step size tk, which is the solution of the problem

$$\min\_{t \ge 0} f(\mathbf{x}\_k + td\_k),\tag{20}$$

else find the step size tk by any of the inexact line search methods.

Step 3. Set xkþ1 ¼ xk þ tkdk.

Step 4. Set k ≔ k þ 1 and go to Step 1.

The classical and the oldest steepest descent step size tk, which was designed by Cauchy (in the case of the exact line search), is computed as [26]

$$t\_k = \frac{\mathbf{g}\_k^T \mathbf{g}\_k}{\mathbf{g}\_k^T \mathbf{G} \mathbf{g}\_k},$$

where gk ¼ ∇f xð Þk and G ¼ ∇2f xð Þk .

Theorem 1.2.2. [27] (Global convergence theorem of the SD method) Let f ∈C1 . Then, each accumulation point of the iterative sequence f g xk , generated by Algorithm 1.2.4, is a stationary point.

Remark 1.2.1. The steepest descent method has at least the linear convergence rate. More information about the convergence of the SD method can be found in [5, 27].

Although known as the first unconstrained optimization method, this method is still a theme considered by scientists.

Different modifications of this method are made, for example, see [25, 28–32].

In [28], the authors presented a new search direction from Cauchy's method in the form of two parameters known as Zubai'ah-Mustafa-Rivaie-Ismail method, shortly, ZMRI method:

$$d\_k = -\mathbf{g}\_k - \|\mathbf{g}\_k\| \mathbf{g}\_{k-1}.\tag{21}$$

So, in [28], a new modification of SD method is suggested using a new search direction, dk, given by (21). The numerical results are presented based on the number of iterations and CPU time. It is shown that this new method is efficient when it is compared to the classical SD.

In [25], a new scaled search direction of SD method is presented. The inspiration for this new method is the work of Andrei [33], in which the author presents and analyzes a new scaled conjugate gradient algorithm, based on an interpretation of the secant equation and on the inexact Wolfe line search conditions.

The method proposed in [25] is known as Rashidah-Rivaie-Mamat (RRM) method, and it suggests the direction dk given by the next relation:

$$d\_k = \begin{cases} -\mathcal{g}\_k, \text{if } k = 0, \\ -\theta\_k \mathcal{g}\_k - \|\mathcal{g}\_k\| \mathcal{g}\_{k-1}, \end{cases} \tag{22}$$

k�1 y where k�1 θk is a scaling parameter, θk ¼ dT ∥2 , yk�1 ¼ gk � gk�1. ∥ gk�1

Further, in [25], a comparison among RRM, ZMRI, and SD methods is made; it is shown that RRM method is better than ZMRI and SD methods.

It is interesting that the exact line search is used in [25].

In [34], the properties of steepest descent method from the literature are reviewed together with advantages and disadvantages of each step size procedure.

Namely, the step size procedures, which are compared in this paper, are:

gT 1. tk ¼ k gk : Step size method by Cauchy [24], computed by exact line search gTHk g k k (C step size).

° s; sβ; sβ2 ˛ 2. Given s . 0, β, σ ∈ ð0; 1Þ, tk ¼ max ; … such that

$$\int (\mathbf{x}\_k + \mathbf{t}\_k d\_k) \le \mathbf{f}(\mathbf{x}\_k) + \sigma t\_k \mathbf{g}\_k^T d\_k - \text{Armijo's line search} \left(\mathbf{A} \text{step size}\right).$$

3. Given β, σ ∈ ð0; 1Þ,~t0 ¼ 1, and tk ¼ β~tk such that

$$f(\mathbf{x}\_k + t\_k d\_k) \le f(\mathbf{x}\_k) + \sigma t\_k \mathbf{g}\_k^T d\_k - \text{Backtracking line search} (\mathbf{B} \text{step size}).$$

s k�1yk�1 4. tk ¼ ∥ T yk�1∥2 , (BB1), tk ¼ ∥sk�1∥2 , (BB2), sk�1 ¼ xk � xk�1 yk�1 ¼ gk � gk�1, : s T k�1yk�1 Barzilai and Borwein's formula. The convergence is R-superlinear.

T t 2 g g k�1 k k 5. tk ¼ : Elimination line search (EL step size), which esti- 2 f xð kþtkdkÞ�f xð kÞþtk�1 gTgk ð , k

mates the step size without computation of the Hessian.

The comparison is based on time execution, number of total iteration, total percentage of function, gradient and Hessian evaluation, and the most decreased value of objective function obtained.

From the numerical results, the authors conclude that the A method and BB1 method are the best methods among others.

Further, in [34], the general conclusions about the steepest descent method are given:

3. xk approaches the minimizer slowly, in fact in a zigzag way.

In [35], in the aim to achieve fast convergence and the monotone property, a new step size for the steepest descent method is suggested.

In [36], for quadratic positive definite problems, an over-relaxation has been considered. Namely, Raydan and Svaiter [36] proved that the poor behavior of the steepest descent method is due to the optimal Cauchy choice of step size and not to the choice of the search direction. These results are extended in [29] to convex, well-conditioned functions. Further, in [29], it is shown that a simple modification of the step length by means of a random variable uniformly distributed in ð0; 1�, for the strongly convex functions, represents an improvement of the classical gradient descent algorithm. Namely, in this paper, the idea is to modify the gradient descent method by introducing a relaxation of the following form:

$$\mathbf{x}\_{k+1} = \mathbf{x}\_k + \theta\_k \mathbf{t}\_k d\_k,\tag{23}$$

where θk is the relaxation parameter, a random variable uniformly distributed between 0 and 1.

In the recent years, the steepest descent method has been applied in many branches of science; one can be inspired, for example, by [37–43].

#### 2.2 Barzilai and Borwein gradient method

Remind to the fact that SD method performs poorly, converges linearly, and is badly affected by the ill-conditioning.

Also, remind to the fact that this poor behavior of SD method is due to the optimal choice of the step size and not to the choice of the steepest descent direction �gk.

Barzilai and Borwein presented [44] a two-point step size gradient method, which is well known as BB method.

The step size is derived from a two-point approximation to the secant equation. Consider the gradient iteration form:

$$
\mathbf{x}\_{k+1} = \mathbf{x}\_k - t\_k \mathbf{g}\_k.
$$

It can be rewritten as xkþ1 ¼ xk � Dk gk, where Dk ¼ tkI.

To make the matrix Dk having quasi-Newton property, the step size tk is computed in such a way that we get

$$\min \|s\_{k-1} - D\_k \mathcal{y}\_{k-1}\|.$$

Some Unconstrained Optimization Methods DOI: http://dx.doi.org/10.5772/intechopen.83679

This yields that

$$t\_k^{BB1} = \frac{s\_{k-1}^T y\_{k-1}}{y\_{k-1}^T y\_{k-1}}, \mathbf{s}\_{k-1} = \mathbf{x}\_k - \mathbf{x}\_{k-1}, y\_{k-1} = \mathbf{g}\_k - \mathbf{g}\_{k-1}.\tag{24}$$

But, using symmetry, we may minimize ∥D�1 sk�1 � yk�1 k ∥, with respect to tk, and we get:

$$t\_k^{BB2} = \frac{\left\|\mathbf{s}\_{k-1}\right\|^2}{s\_{k-1}^T y\_{k-1}}, \mathbf{s}\_{k-1} = \mathbf{x}\_k - \mathbf{x}\_{k-1}, y\_{k-1} = \mathbf{g}\_k - \mathbf{g}\_{k-1}.\tag{25}$$

Now, we give the algorithm of BB method.

Algorithm 1.2.5. (Barzilai-Borwein gradient method, i.e., BB method). Assumptions: 0 , ϵ ≪ 1, x0 ∈ Rn. Let k ¼ 0.

Step 1. If ∥ gk∥ ≤ϵ, then STOP, else set dk ¼ �gk.

Step 2. If k ¼ 0, then find the step size t0 by the line search, else compute tk using the formula (24) or (25).

Step 3. Set xkþ1 ¼ xk þ tkdk.

Step 4. Set k ≔ k þ 1 and go to Step 1.

Considering Algorithm 1.2.5, we can conclude that this method does not require any matrix computation or any line search.

The Barzilai-Borwein method is in fact the gradient method, which requires less computational work than SD method, and it speeds up the convergence of the gradient method. Barzilai and Borwein proved that BB algorithm is R�superlinearly convergent for the quadratic case.

In the general non-quadratic case, a globalization strategy based on nonmonotone line search is applied in this method.

In this general case, tk, computed by (24) or (25), may be unacceptably large or small. That is the reason why we assume that there exist the numbers t l and t r , such that

$$0 \le t^l \le t\_k \le t^r,\text{ for all } k.$$

Using the iteration

$$\boldsymbol{\infty}\_{k+1} = \boldsymbol{\mathcal{X}}\_{k} - \frac{1}{t\_{k}} \boldsymbol{\mathcal{g}}\_{k} = \boldsymbol{\mathcal{X}}\_{k} - \boldsymbol{\lambda}\_{k} \boldsymbol{\mathcal{g}}\_{k},\tag{26}$$

with

$$t\_k = \frac{s\_{k-1}^T \mathcal{Y}\_{k-1}}{s\_{k-1}^T s\_{k-1}}, \lambda\_k = \frac{1}{t\_k},$$

$$s\_k = -\frac{1}{t\_k} \mathbf{g}\_k = -\lambda\_k \mathbf{g}\_k,$$

we get

$$t\_{k+1} = \frac{s\_k^T \mathcal{Y}\_k}{s\_k^T s\_k} = \frac{-\lambda\_k \mathcal{g}\_k^T \mathcal{Y}\_k}{\lambda\_k^2 \mathcal{g}\_k^T \mathcal{g}\_k} = -\frac{\mathcal{g}\_k^T \mathcal{Y}\_k}{\lambda\_k \mathcal{g}\_k^T \mathcal{g}\_k}.$$

Now, we give the algorithm of the Barzilai-Borwein method with non-monotone line search.

Algorithm 1.2.6. (BB method with non-monotone line search).

Assumptions: 0 , ϵ ≪ 1, x0 ∈ Rn, M ≥0 is an integer, ρ∈ð0; 1Þ, δ . 0, 0 , σ1 , σ2 , 1, t l , t r . Let k ¼ 0. Step 1. If ∥ gk∥ ≤ϵ, then STOP. Step 2. If tk ≤t l , or tk ≥t r , then set tk ¼ δ. Step 3. Set λ ¼ 1 . tk Step 4. (non-monotone line search) If

$$f(\mathbf{x}\_k - \lambda \mathbf{g}\_k) \le \max\_{0 \le j \le \min(k, \mathcal{M})} f(\mathbf{x}\_{k-j}) - \rho \lambda \mathbf{g}\_k^T \mathbf{g}\_k,$$

then set

$$
\lambda\_k = \lambda, \mathfrak{x}\_{k+1} = \mathfrak{x}\_k - \lambda\_k \mathfrak{g}\_{k'},
$$

and go to Step 6.

Step 5. Choose σ ∈½σ1; σ2�, set λ ¼ σλ, and go to Step 4. gTyk Step 6. Set tkþ1 ¼ � k and k ≔ k þ 1, and return to Step 1. λk gTg k k Obviously, the above algorithm is globally convergent.

Several authors paid attention to the Barzilai-Borwein method, and they proposed some variants of this method.

In [8], the globally convergent Barzilai-Borwein method is proposed by using non-monotone line search by Grippo et al. [12]. In the same paper, Raydan proves the global convergence of the non-monotone Barzilai-Borwein method.

Further, Grippo and Sciandrone [45] propose another type of the non-monotone Barzilai-Borwein method.

Dai [7] gives the basic analysis of the non-monotone line search strategy. Moreover, in [46] numerical results are presented, using

$$t\_k = \frac{s\_{\nu(k)}^T y\_{\nu(k)}}{s\_{\nu(k)}^T s\_{\nu(k)}}.\tag{27}$$

and

$$\nu(k) = M\_{\mathfrak{c}} \cdot \ll \frac{k-1}{M\_{\mathfrak{c}}} \ll 1$$

where for r∈ R, ⌞r⌟ denotes the largest integer j such that j≤r and Mc is a positive integer. The gradient method with (27) is called the cyclic Barzilai-Borwein method. Numerical results in [46] prove that their method performs better than the Barzilai-Borwein method.

Many researchers study the gradient method for minimizing a strictly convex quadratic function, namely,

$$\min f(\mathbf{x}) = \frac{1}{2}\mathbf{x}^T A \mathbf{x} - \mathbf{b}^T \mathbf{x},\tag{28}$$

where A ∈ Rn�n is a symmetric positive definite matrix and b∈ Rn is a given vector. For an application of the Barzilai-Borwein method to the problem (28), Raydan [47] establishes global convergence, and Dai and Liao [48] prove R-linear rate of convergence. Friedlander, Martinez, Molina, and Raydan [49] propose a new gradient method with retards, in which tk is defined by

Some Unconstrained Optimization Methods DOI: http://dx.doi.org/10.5772/intechopen.83679

$$\mathfrak{a}\_{k} = \frac{\mathfrak{g}\_{\nu(k)}^{T} A^{\rho(k)+1} \mathfrak{g}\_{\nu(k)}}{\mathfrak{g}\_{\nu(k)}^{T} A^{\rho(k)} \mathfrak{g}\_{\nu(k)}}, \nu(k) \in \{k, k-1, \dots, \max\{0, k-m\}\} \tag{29}$$

˜ ° and ρð Þk ∈ q1; …; q , where m is a positive integer and q1, …, q ≥ � 2 are m m integers. In the same paper, they establish its global convergence for problem (28) and prove the Q-superlinear rate of convergence in the special case.

In [50], the authors extend the Barzilai-Borwein method, and they give extended Barzilai-Borwein method, which they denote EBB. They also establish global and Q�superlinear convergence properties of the proposed method for minimizing a strictly convex quadratic function. Furthermore, they discuss an application of their method to general objective functions. In [50], a new step size is proposed by extending (29). Namely, in this paper, following Friedlander et al. [49], a new step size is proposed as follows:

$$\begin{aligned} t\_k &= \sum\_{i=1}^l \phi\_i \frac{\mathbf{g}\_{\nu\_{i(k)}}^T A^{\rho\_i(k)+1} \mathbf{g}\_{\nu\_i(k)}}{\mathbf{g}\_{\nu\_{i(k)}}^T A^{\rho\_i(k)} \mathbf{g}\_{\nu\_i(k)}}, \\ \phi\_i &\ge 0, \sum\_{i=1}^n \phi\_i = \mathbf{1}, \\ \nu\_i(k) &\in \{k, k-1, \dots, \max\{0, k-m\}\} \end{aligned}$$

and

$$\phi\_i(k) \in \{q\_1, \ldots, q\_m\},$$

where l and m are positive integers and q1, …, q are integers. m

Also, an application of algorithm EBB to general unconstrained minimization problems (4) is considered.

Following Raydan [8], the authors [50] further combine the non-monotone line search and algorithm EBB to get the algorithm called NEBB. They also prove the global convergence of the algorithm NEBB, under some classical assumptions.

The Barzilai-Borwein method and its related methods are reviewed by Dai and Yuan [51] and Fletcher [52].

In [53], a new concept of the approximate optimal step size for gradient method is introduced and used to interpret the BB method; an efficient gradient method with the approximate optimal step size for unconstrained optimization is presented. The next definition is introduced in [53].

Definition 1.2.1. Let Φð Þt be an approximation model of fðxk � tg kÞ. A positive constant t ∗ is called approximate optimal step size associated to Φð Þt for gradient method, if t ∗ satisfies

$$t^\* = \arg\min\_{t \ge 0} \Phi(t).$$

The approximate optimal step size is different from the steepest descent step size, which will lead to the expensive computational cost. The approximate optimal step size is generally calculated easily, and it can be applied to unconstrained optimization.

Due to the effectiveness of t BB1 and the fact that t BB1 ¼ argmint . 0Φð Þt , we can k k naturally ask if more suitable approximation models can be constructed to generate more efficient approximate optimal step-sizes.

#### Applied Mathematics

This is the purpose of work [53]. Further, if the objective function f xð Þ is not close to a quadratic function on the line segment between xk�1 and xk, in this paper a conic model is developed to generate the approximate optimal step size if the conic model is suitable to be used. Otherwise, the authors consider two cases:

In [54], derivative-free iterative scheme that uses the residual vector as search direction for solving large-scale systems of nonlinear monotone equations is presented.

The Barzilai-Borwein method is widely used; some interesting results can be found in [55–57].

#### 2.3 Newton method

The basic idea of Newton method for unconstrained optimization is the iterative usage of the quadratic approximation qð Þk to the objective function f at the current ð Þk iterate xk and then minimization of such approximation q .

Let f : Rn ! R be twice continuously differentiable, xk ∈ Rn, and let the Hessian ∇2f xð Þk be positive definite.

We model ð Þk f at the current point xk by the quadratic approximation q :

$$f(\boldsymbol{\mathfrak{x}}\_{k} + \boldsymbol{s}) \approx \boldsymbol{q}^{(k)}(\boldsymbol{\mathfrak{s}}) = \boldsymbol{f}(\boldsymbol{\mathfrak{x}}\_{k}) + \nabla \boldsymbol{f}(\boldsymbol{\mathfrak{x}}\_{k})^{T} \boldsymbol{s} + \frac{1}{2} \boldsymbol{s}^{T} \nabla^{2} \boldsymbol{f}(\boldsymbol{\mathfrak{x}}\_{k}) \boldsymbol{s}, \boldsymbol{s} = \boldsymbol{\mathfrak{x}} - \boldsymbol{\mathfrak{x}}\_{k}.$$

Minimization ð Þk of q ð Þs gives the next iterative scheme:

$$\boldsymbol{\pi}\_{k+1} = \boldsymbol{\pi}\_k - \left(\nabla^2 f(\boldsymbol{\pi}\_k)\right)^{-1} \nabla f(\boldsymbol{\pi}\_k),$$

which is known as Newton formula. Denote Gk ¼ ∇2 f xk , g ð Þ k ¼ ∇f xð Þk . Then, we have a simpler form:

$$
\varkappa\_{k+1} = \varkappa\_k - G\_k^{-1} \mathbf{g}\_k. \tag{30}
$$

A Newton direction is

$$
\kappa\_k = \varkappa\_{k+1} - \varkappa\_k = -G\_k^{-1} \mathcal{g}\_k. \tag{31}
$$

We have supposed that Gk is positive definite. So, the Newton direction is a descent direction. This we can conclude from

$$\mathbf{g}\_k^T \mathbf{s}\_k = -\mathbf{g}\_k^T \mathbf{G}\_k^{-1} \mathbf{g}\_k \le \mathbf{0}.$$

Some Unconstrained Optimization Methods DOI: http://dx.doi.org/10.5772/intechopen.83679

Now, we give the algorithm of the Newton method.

Algorithm 1.2.7. (Newton method). Assumptions: ϵ . 0, x0 ∈ Rn. Let k ¼ 0. Step 1. If ∥ gk∥ ≤ϵ, then STOP. Step 2. Solve Gks ¼ �gk for sk. Step 3. Set xkþ1 ¼ xk þ sk. Step 4. k ≔ k þ 1, return to Step 1.

The next theorem shows the local convergence and the quadratic convergence rate of Newton method.

Theorem 1.2.3. [27] (Convergence theorem of Newton method) Let f ∈C2 and xk be close enough to the solution x∗ of the minimization problem with gðx∗Þ ¼ 0. If the Hessian G ∗ ð Þ x is positively definite and Gð Þ x satisfies Lipschitz condition

$$|G\_{i\bar{j}}(\infty) - G\_{i\bar{j}}(\mathcal{Y})| \le \beta \|\infty - \mathcal{Y}\|, for \text{ some } \beta, \text{for all } i, j, \text{ }$$

where Gijð Þ x is the ð Þ i; j element of G xð Þ and then for all k, Newton direction (31) is well-defined; the generated sequence xk f g converges to x∗ with a quadratic rate.

But, in spite of this quadratic rate, the Newton method is a local method: when the starting point is far away from the solution, there is a possibility that Gk is not positive definite, as well as Newton direction is not a descent direction.

So, to guarantee the global convergence, we can use Newton method with line search. We can remind to the fact that only when the step size sequence f g tk tends to 1, Newton method is convergent with the quadratic rate.

Newton iteration with line search is as follows:

$$d\_k = -G\_k^{-1} \mathbf{g}\_{k^\*} \tag{32}$$

$$
\varkappa\_{k+1} = \varkappa\_k + t\_k d\_k. \tag{33}
$$

Now, we give the algorithm.

Algorithm 1.2.8. (Newton method with line search). Assumptions: ϵ . 0, x0 ∈ Rn. Let k ¼ 0. Step 1. If ∥ gk∥ ≤ϵ, then STOP. Step 2. Solve Gkd ¼ �gk for dk. Step 3. Line search step: find tk such that

$$f(\mathbf{x}\_k + t\_k d\_k) = \min\_{t \ge 0} f(\mathbf{x}\_k + t d\_k),$$

or find tk such that (inexact) Wolfe line search rules hold.

Step 4. Set xkþ1 ¼ xk þ tkdk and k ¼ k þ 1, and go to Step 1.

The next theorems claim that Algorithm 1.2.8 with the exact line search, as well as Algorithm 1.2.8 with the inexact line search, are globally convergent.

Theorem 1.2.4. [27] Let f : Rn ! R be twice continuously differentiable on open convex set D⊂Rn. Assume that for any x0 ∈ D there exists a constant m . 0, such that f xð Þ satisfies

$$\|u^T \nabla^2 f(\mathbf{x}) u \ge m \|u\|^2, \text{for all } u \in \mathbb{R}^n, \mathbf{x} \in L(\infty\_0),\tag{34}$$

where L xð 0Þ ¼ fxj fð Þ x ≤fð Þ x0 g is the corresponding level set. Then, the sequence f g xk , generated by Algorithm 1.2.8, with the exact line search, satisfies:

Note that the next relation holds from the standard Wolfe line search:

$$f(\varkappa\_k) - f(\varkappa\_k + t\_k d\_k) \ge \overline{\eta} \|\lg\_k\|^2 \cos^2 \angle(d\_k, -\mathbb{g}\_k),\tag{35}$$

where the constant η does not depend on k.

Theorem 1.2.5. [27] Let f : Rn ! R be twice continuously differentiable on open convex set D⊂Rn. Assume that for any x0 ∈ D there exists a constant m . 0, such that f xð Þ satisfies the relation (34) on the level set Lð Þ x0 . If the line search satisfies the relation (35), then the sequence f g xk , generated by Algorithm 1.2.8, with the inexact Wolfe line search, satisfies

$$\lim\_{k \to \infty} \|\mathbf{g}\_k\| = \mathbf{0}$$

and f g xk converges to the unique minimizer of f xð Þ.

#### 2.4 Modified Newton method

The main problem in Newton method could be the fact that the Hessian Gk may be not positive definite. In that case, we are not sure that the objective function f has its minimizers; furthermore, when Gk is indefinite, the objective function f is unbounded.

So, many modified schemes are made. Now, we describe the next two methods shortly.

In [58], Goldstein and Price use the steepest descent method when Gk is not positive definite. Denoting the angle between dk and �gk by θ, as well as having in view the angle rule, θ ≤ π � μ, where μ . 0, they determine the direction dk as 2

$$d\_k = \begin{cases} -\mathbf{G}\_k^{-1} \mathbf{g}\_k, & \text{if } \cos \theta \ge \eta, \\\ -\mathbf{g}\_k, & \text{otherwise}, \end{cases}$$

where η . 0 is a given constant.

In [59], the authors present another modified Newton method. When Gk is not positive definite, Hessian Gk is changed into Gk þ νkI, where νk . 0 is chosen in such a way that Gk þ νkI is positive definite and well-conditioned. Otherwise, when Gk is positive definite, νk ¼ 0.

To consider the other modified Newton methods, such as finite difference Newton method, negative curvature direction method, Gill-Murray stable Newton method, etc., one can see [27], for example.

#### 2.5 Inexact Newton method

By the other side, because of the high cost of the exact Newton method, especially when the dimension n is large, the inexact Newton method might be a good solution. This type of method means that we only approximately solve the Newton equation.

Consider solving the nonlinear equations:

$$F(\mathbf{x}) = \mathbf{0},\tag{36}$$

where F : Rn ! Rn is assumed to have the next properties: A1 There exists x∗ such that F x∗ ð Þ¼ 0.

A ∗ 2 F is continuously differentiable in the neighborhood of x . A ∗ 3 F0 ð Þ x is nonsingular.

Remind that the basic Newton step is obtained by solving

$$F'(\mathfrak{x}\_k)\mathfrak{s}\_k = -F(\mathfrak{x}\_k)$$

and setting

xkþ1 ¼ xk þ sk:

The inexact Newton method means that we solve

$$F'(\mathbf{x}\_k)\mathbf{s}\_k = -F(\mathbf{x}\_k) + r\_k. \tag{37}$$

where

$$\|\boldsymbol{r}\_{k}\| \leq \eta\_{k} \|\boldsymbol{F}(\boldsymbol{x}\_{k})\|.\tag{38}$$

Set

$$
\varkappa\_{k+1} = \varkappa\_k + \varsigma\_k. \tag{39}
$$

Here, rk denotes the residual, and the sequence ηk f g, where 0 , ηk , 1, is the sequence which controls the inexactness.

Now, we give two theorems; the first of them claims the linear convergence, and the second claims the superlinear convergence of the inexact Newton method.

Theorem 1.2.6. [27] Let F : Rn ! Rn satisfy the assumptions A1–A3. Let the sequence f g η satisfies 0≤ηk ≤ η , t , 1. Then, for some ϵ . 0, if the starting point is k sufficiently near x∗, the sequence f g xk generated by inexact Newton's method (37)–(39) converges to x∗, and the convergence rate is linear, i.e.

$$\|\mathfrak{x}\_{k+1} - \mathfrak{x}^\*\|\_\* \le t \|\mathfrak{x}\_k - \mathfrak{x}^\*\|\_{\*\*}$$

where ∗ ∥y∥∗ ¼ ∥F0 ð Þ x y∥.

Theorem 1.2.7. [27] Let all assumptions of Theorem 1.2.6 hold. Assume that the ∗ sequence f g xk , generated by the inexact Newton method, converges to x . Then

$$\|r\_k\| = o(\|F(\mathfrak{x}\_k)\|), k \to \infty,$$

if and only if xk converges to x∗ f g superlinearly. The relation

$$\boldsymbol{\omega}\_{k+1} = \boldsymbol{\omega}\_k - \frac{\boldsymbol{f}'(\boldsymbol{\omega}\_k)}{\boldsymbol{f}'(\boldsymbol{\omega}\_k) - \boldsymbol{f}'(\boldsymbol{\omega}\_{k-1})} \cdot (\boldsymbol{\omega}\_k - \boldsymbol{\omega}\_{k-1}),\tag{40}$$

presents the secant method.

In [60], a modification of the classical secant method for solving nonlinear, univariate, and unconstrained optimization problems based on the development of the cubic approximation is presented. The iteration formula including an approximation of the third derivative of f xð Þ by using the Taylor series expansion is derived. The basic assumption on the objective function f xð Þ is that fð Þ x is a real- ∗ valued function of a single, real variable x and that f xð Þ has a minimum at x . Furthermore, in this chapter it is noted that the secant method is the simplification of Newton method. But, the order of the secant method is lower than one of the Newton methods; it is Q-superlinearly convergent, and its order is pffiffi

5þ1 p ¼ ≈1; 618. 2

This modified secant method is constructed in [60], having in view, as it is emphasized, that it is possible to construct a cubic function which agrees with f xð Þ up to the third derivatives. The third derivative of the objective function f is approximated as

$$f^{'}(\mathbf{x}) = \frac{\mathbf{3}\left\{\frac{2\left[f^{'(\mathbf{x}\_{k})} - \frac{f(\mathbf{x}\_{k}) - f(\mathbf{x}\_{k-1})}{\mathbf{x}\_{k} - \mathbf{x}\_{k-1}}\right]}{\mathbf{x}\_{k} - \mathbf{x}\_{k-1}} - f^{'}(\mathbf{x}\_{k})\right\}}{\mathbf{x}\_{k-1} - \mathbf{x}\_{k}}.$$

In [61], the authors propose an inexact Newton-like conditional gradient method for solving constrained systems of nonlinear equations. The local convergence of the new method as well as results on its rate is established by using a general majorant condition.

### 2.6 Quasi-Newton method

Consider the Newton method.

For various practical problems, the computation of Hessian may be very expensive, or difficult, or Hessian can be unavailable analytically. So, the class of so-called quasi-Newton methods is formed, such that it uses only the objective function values and the gradients of the objective function and it is close to Newton method. Quasi-Newton method is such a class of methods which does not compute Hessian, but it generates a sequence of Hessian approximations and maintains a fast rate of convergence.

So, we would like to construct Hessian approximation Bk in quasi-Newton method. Naturally, it is desirable that the sequence f g Bk possesses positive defi- 1 niteness, as well as its direction dk ¼ -B- gk should be a descent one. k

Now, let f : Rn ! R be twice continuously differentiable function on an open set D⊂Rn. Consider the quadratic approximation of f at xkþ1:

$$f(\mathbf{x}) \approx f(\boldsymbol{\omega}\_{k+1}) + \mathbf{g}\_{k+1}^T(\boldsymbol{\omega} - \boldsymbol{\omega}\_{k+1}) + \frac{\mathbf{1}}{2}(\boldsymbol{\omega} - \boldsymbol{\omega}\_{k+1})^T \mathbf{G}\_{k+1}(\boldsymbol{\omega} - \boldsymbol{\omega}\_{k+1}).$$

Finding the derivatives, we get

$$\mathbf{g}(\mathbf{x}) \approx \mathbf{g}\_{k+1} + G\_{k+1}(\mathbf{x} - \mathbf{x}\_{k+1}).$$

Setting x ¼ xk and using the standard notation: sk ¼ xkþ1 - xk, yk ¼ gkþ1 - gk, from the last relation, we get

$$G\_{k+1}^{-1} \mathcal{Y}\_k \approx \mathfrak{s}\_k. \tag{41}$$

Relation (41) transforms into the next one if f is the quadratic function:

$$G\_{k+1}^{-1} \mathcal{Y}\_k = \mathfrak{s}\_k. \tag{42}$$

Let Hk be the approximation of the inverse of Hessian. Then, we want Hk to satisfy the relation (42). In this way, we come to the quasi-Newton condition or quasi-Newton equation:

$$H\_{k+1} \mathfrak{y}\_k = \mathfrak{s}\_k. \tag{43}$$

Let Bkþ1 ¼ H�1 kþ1 be the approximation of Hessian Gkþ1. Then

$$B\_{k+1} \mathfrak{s}\_k = \mathfrak{y}\_k \tag{44}$$

is also the quasi-Newton equation. If

$$s\_k^T y\_k \ge \mathbf{0},\tag{45}$$

then the matrix Bkþ1 is positive definite. The condition (45) is known as the curvature condition.

Algorithm 1.2.9. (A general quasi-Newton method). n Assumptions: 0 ≤ϵ , 1, x0 ∈ Rn, H0 ∈ Rn� . Let k ¼ 0. Step 1. If ∥ gk∥ ≤ϵ, then STOP. Step 2. Compute dk ¼ �Hk gk. Step 3. Find tk by line search and set xkþ1 ¼ xk þ tkdk. Step 4. Update Hk into Hkþ1 such that quasi-Newton equation (43) holds. Step 5. Set k ¼ k þ 1 and go to Step 1. In Algorithm 1.2.9, usually we take H0 ¼ I, where I is an identity matrix. Sometimes, instead of Hk, we use Bk in Algorithm 1.2.9. Then, Step 2 becomes Step 2∗. Solve

$$B\_k d = -\mathbf{g}\_{k^\*} \text{ for } d\_k \dots$$

By the other side, Step 4 becomes

Step 4∗. Update Bk into Bkþ1 in such a way that quasi-Newton equation (44) holds.

#### 2.7 Symmetric rank-one (SR1) update

Let Hk be the inverse Hessian approximation of the kth iteration. We are trying to update Hk into Hkþ1, i.e.

$$H\_{k+1} = H\_k + E\_{k\nu}$$

where Ek is a matrix with a lower rank. If it is about a rank-one update, we get

$$H\_{k+1} = H\_k + \mu v^T,\tag{46}$$

where u, v∈ Rn. Using quasi-Newton equation (43), we can get

$$H\_{k+1}\mathcal{Y}\_k = \left(H\_k + \mu v^T\right)\mathcal{Y}\_k = s\_{k\nu}$$

wherefrom

$$(v^T \mathcal{Y}\_k)u = \mathfrak{s}\_k - H\_k \mathcal{Y}\_k. \tag{47}$$

Further, from (46) and (47), we have

$$H\_{k+1} = H\_k + \frac{1}{\nu^T \mathcal{Y}\_k} \left(\mathfrak{s}\_k - H\_k \mathfrak{y}\_k\right) \nu^T.$$

Having in view that the inverse Hessian approximation Hk has to be the symmetric one, we use v ¼ sk � Hkyk, so we get the symmetric rank-one update (i.e., SR1 update):

$$H\_{k+1} = H\_k + \frac{\left(\mathbf{s}\_k - H\_k \mathbf{y}\_k\right)\left(\mathbf{s}\_k - H\_k \mathbf{y}\_k\right)^T}{\left(\mathbf{s}\_k - H\_k \mathbf{y}\_k\right)^T \mathbf{y}\_k}. \tag{48}$$

Theorem 1.2.8. [27] (Property theorem of SR1 update) Let s0, s1, and sn�1 be linearly independent. Then, for quadratic function with a positive definite Hessian, SR1 method terminates at n þ 1 steps, i.e., Hn ¼ G�1 .

More information about SR1 update can be found.

#### 2.8 Davidon-Fletcher-Powell (DFP) update

There exists another type of update, which is a rank-two update. In fact, we get Hkþ1 using two symmetric, rank-one matrices:

$$H\_{k+1} = H\_k + auu^T + bvv^T,\tag{49}$$

where u, v∈ Rn and a, b are scalars which have to be determined. Using quasi-Newton equation (43), we can get

$$+H\_k y\_k + auu^T y\_k + bvv^T y\_k = \mathfrak{s}\_k. \tag{50}$$

The values of u, v are not determined in a unique way, but the good choice is

$$
\mu = s\_k, \upsilon = H\_k \mathcal{y}\_k.
$$

Now, from (50), we get:

$$a = \frac{1}{s\_k^T \mathcal{Y}\_k}, b = -\frac{1}{\mathcal{Y}\_k^T H\_k \mathcal{Y}\_k}.$$

Hence, we get the formula

$$H\_{k+1} = H\_k + \frac{s\_k s\_k^T}{s\_k^T \boldsymbol{\nu}\_k} - \frac{H\_k \boldsymbol{\nu}\_k \boldsymbol{\nu}\_k^T H\_k}{\boldsymbol{\nu}\_k^T H\_k \boldsymbol{\nu}\_k},\tag{51}$$

which is DFP update.

Theorem 1.2.9. [27] (Positive definiteness of DFP update) DFP update (51) retains T positive definiteness if and only if sk yk . 0.

Theorem 1.2.10. [27] (Quadratic termination theorem of DFP method) Let fð Þ x be a quadratic function with positive definite Hessian G. Then, if the exact line search is used, the ˛ ˝ sequence sj , generated from DFP method, satisfies, for i ¼ 0, 1, …, m, where m ≤ n � 1:

Some Unconstrained Optimization Methods DOI: http://dx.doi.org/10.5772/intechopen.83679