# Second-order optimality conditions for multiobjective optimization problems
with constraints

Nguyen Quang Huy  Department of Mathematics, Hanoi Pedagogical University 2,
Xuan Hoa, Phuc Yen, Vinh Phuc, Vietnam<EMAIL_ADDRESS><EMAIL_ADDRESS>,  Bui Trong Kien  Department of Optimization and Control Theory, Institute of
Mathematics, VAST, 18 Hoang Quoc Viet Road, Hanoi, Vietnam<EMAIL_ADDRESS>,  Gue Myung Lee  Department of Applied Mathematics, Pukyong National
University, Busan 48513, Korea<EMAIL_ADDRESS>and  Nguyen Van Tuyen∗
Department of Mathematics, Hanoi Pedagogical University 2, Xuan Hoa, Phuc Yen,
Vinh Phuc, Vietnam<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract.

In this paper, we introduce the second-order subdifferentials for functions
which are Gâteaux differentiable on an open set and whose Gâteaux derivative
mapping is locally Lipschitz. Based on properties of this kind of second-order
subdifferentials and techniques of variational analysis, we derive second-
order necessary conditions for weak Pareto efficient solutions of
multiobjective programming problems with constraints.

###### Key words and phrases:

Second-order subdifferential, second-order variations, second-order weak
directional derivative, second-order necessary optimality conditions, weak
Pareto efficient solutions

###### 2010 Mathematics Subject Classification:

49J52, 90C29, 90C46, 65K10, 49K30

∗Corresponding.

The research of Nguyen Van Tuyen was supported by the Ministry of Education
and Training of Vietnam [grant number B2018-SP2-14]

##  1\. Introduction

Let $X,Y$, and $Z$ be Banach spaces with the dual spaces $X^{*},Y^{*},$ and
$Z^{*}$, respectively. Throughout the paper we assume that the unit ball
$B^{*}\subset X^{*}$ is weak∗-sequentially compact. Let $D$ be a nonempty open
subset in $X$ and $Q$ be a closed convex set in $Z$ with nonempty interior.
Given mappings $f_{j}\colon D\to\mathbb{R}$, $H\colon D\to Y$ and $G:D\to Z$,
we consider the following constrained multiobjective programming problem

$(P)\quad\quad\begin{cases}\text{Min}_{\mathbb{R}^{m}_{+}}F(x):=(f_{1}(x),\ldots,f_{m}(x))\\\
\text{subject to}\\\ H(x)=0,\\\ G(x)\in Q.\end{cases}$

The prototype of such problem arises in control theory with state equations
and pointwise constraints. The goal of this paper is to derive second-order
necessary conditions for problem $(P)$ in term of a notion of second-order
subdifferentials for functions which are of class $C^{1,1}(D)$. Recall that a
function $\phi\colon D\to\mathbb{R}$ is said to be of class $C^{1,1}(D)$ if
its first-order Gâteaux derivative $\phi^{\prime}\colon D\to X^{*}$ is locally
Lipschitz on $D$.

By introducing generalized second-order directional derivatives and using
techniques of variational analysis, Páles and Zeidan [21] gave second-order
necessary conditions for mathematical programming problems in the form $(P)$,
i.e., when $m=1$, and some problems where the objective function is the
maximum of smooth functions depending on a parameter from a compact metric
space. To our knowledge, this result has been the best one on the second-order
necessary conditions so far. Instead of generalized second-order directional
derivatives, Georgiev and Zlateva [10] introduced the so-called second-order
Clarke subdifferentials for functions of class $C^{1,1}(D)$ in the case, where
the dual $X^{*}$ is separable. This definition is based on a result of
Christensen related to the almost everywhere differentiability of Lipschitz
functions $\psi\colon X\to Z$ with $Z$ is a Banach space which has a
Radon–Nikodym property. Then the authors obtained second-order necessary
conditions and sufficiently conditions for mathematical programming problems
in the form $(P)$ in terms of second-order Clarke subdifferentials.

The study of second-order optimality conditions for vector optimization
problems is of the concern of some mathematicians. For the papers which have
close connection to the present work, we refer the readers to [11, 15, 16] and
references therein. Let us give briefly some comments on the considered
problems and the obtained results of those papers. In [11], under the Robinson
qualification constraint conditions, the author derived second-order necessary
optimality conditions and sufficient optimality conditions for vector
optimization problems where the mappings are second-order directionally
differentiable. By using the Dubovitskii–Milyutin approach, the authors [15,
16] obtained some second-order necessary optimality conditions in terms of
second-order tangential derivatives for set-valued optimization problems. For
more discussions on the recent development of the second-order derivatives
relative to optimal conditions in nonsmooth analysis, the reader is referred
to [4, 5, 6, 17, 18, 19, 20, 23, 25, 12, 13, 14, 22, 24] and the references
therein.

In this paper we derive the second-order necessary optimality conditions for
problem $(P)$, where the Robinson qualification constraint conditions may not
be valid and the mappings may not be second-order differentiable. To do this,
we first introduce second-order subdifferentials for functions of class
$C^{1,1}(D)$ and give some properties for this kind of second-order
subdifferentials. We then utilize the Dubovitskii–Milyutin approach as well as
techniques of variational analysis of [21] to deal with the problem. The
obtained results improve and generalize the corresponding results of [10,
Theorem 2.4], [21, Theorem 6] and [22, Theorem 8.2]. We also show that our
results still hold for critical directions which may not be regular.

The rest of our paper consists of two sections. In Section 2, we present some
properties of second-order subdifferentals and some results related to
variation sets of second-order. Section 3 is destined for first- and second-
order necessary conditions for weak Pareto efficient solutions of $(P)$.

##  2\. Second-order subdifferentials and second-order variations

###  2.1. Second-order subdifferentials

Let $f\colon D\to\mathbb{R}$ be a locally Lipschitz function on $D$. Recall
that the Clarke subdifferential of $f$ at $\hat{x}\in D$ is defined by

$\partial f(\hat{x}):=\\{x^{*}\in X^{*}\;|\;\langle x^{*},h\rangle\leq
f^{\circ}(\hat{x};h),\ \ \forall h\in X\\},$

where

$f^{\circ}(\hat{x};h):=\limsup_{x\to\hat{x},\atop{t\to
0^{+}}}\frac{f(x+th)-f(x)}{t}$

is the Clarke directional derivative of $f$ at $\hat{x}$ in the direction $h$.

Denote by ${\mathcal{L}}(X\times X)$ the Banach space of all bilinear
continuous functionals $L\colon X\times X\to\mathbb{R}$ with the norm

$\|L\|:=\sup_{\|h_{1}\|=1\atop{\|h_{2}\|=1}}|L(h_{1},h_{2})|,$

and ${\mathcal{L}}(X,X^{*})$ the Banach space of all linear continuous
mappings $L\colon X\to X^{*}$ with the norm

$\|L\|:=\displaystyle\sup_{\|h\|=1}\|L(h)\|_{*}.$

It is well-known that ${\mathcal{L}}(X\times X)$ and ${\mathcal{L}}(X,X^{*})$
are isometrically isomorphic; see [1, Section 2.2.5]. So, in the sequel, we
identify ${\mathcal{L}}(X\times X)$ and ${\mathcal{L}}(X,X^{*})$. By this way,
if $f\colon D\to\mathbb{R}$ is twice Gâteaux differentiable at $\hat{x}\in D$,
then $f^{\prime\prime}(\hat{x})$ is a linear mapping from $X$ to $X^{*}$. This
suggests us to introduce the following definition.

######  Definition 2.1.

Let $f\in C^{1,1}(D)$ and $\hat{x}\in D$. The second-order subdifferential of
$f$ at $\hat{x}$ is the set-valued map

$\partial^{2}f(\hat{x})\colon X\rightrightarrows X^{*},$

which is defined by

$\partial^{2}f(\hat{x})(d):=\partial\langle
f^{\prime}(\,\cdot\,),d\rangle(\hat{x}),\ \ \forall\ d\in X.$

Note that, by the Hahn–Banach Theorem, $\partial^{2}f(\hat{x})(d)$ is always
nonempty for all $d\in X$.

The following proposition summarizes some properties of
$\partial^{2}f(\,\cdot\,)$.

######  Proposition 2.2.

Suppose that $f$ and $g$ are of class $C^{1,1}(D)$. Then the following
assertions hold:

  1. (i)

The mapping $\partial^{2}f(\hat{x}):X\rightrightarrows X^{*}$ has nonempty
convex and $w^{*}$-compact valued.

  2. (ii)

For each $d\in X$, the mapping $x\mapsto\partial^{2}f(x)(d)$ from
$(D,\|\cdot\|)$ to $X^{*}$ is local bounded and upper semicontinuous at
$\hat{x}$, that is, if $x_{n}\to\hat{x}$, $L_{n}\xrightarrow{w^{*}}{L}$ and
$L_{n}\in\partial^{2}f(x_{n})(d)$, then $L\in\partial^{2}f(\hat{x})(d)$.

  3. (iii)

If $f$ is twice continuously Gâteaux differentiable at $\hat{x}\in D$, then
$\partial^{2}f(\hat{x})=\\{f^{\prime\prime}(\hat{x})\\}$.

  4. (iv)

For any $d\in X$ and $s\in\mathbb{R}$, one has

$\displaystyle\partial^{2}f(\hat{x})(sd)$
$\displaystyle=s\partial^{2}f(\hat{x})(d);$
$\displaystyle\partial^{2}(f+g)(\hat{x})(d)$
$\displaystyle\subset\partial^{2}f(\hat{x})(d)+\partial^{2}g(\hat{x})(d).$

###### Proof.

(i) Since $f^{\prime}(\,\cdot\,)$ is locally Lipschitz around $\hat{x}$ with
constant $l>0$, the mapping

$x\mapsto\langle f^{\prime}(x),d\rangle$

is locally Lipschitz around $\hat{x}$ with constant $l\|d\|$. Hence,

$\|L\|\leq l\|d\|,\ \ \forall L\in\partial^{2}f(\hat{x})(d).$

By the Banach–Alaoglu–Bourbaki Theorem, $\partial^{2}f(\hat{x})(d)$ is a
$w^{*}$-compact set. The convexity of $\partial^{2}f(\hat{x})(d)$ is easy to
check, so omitted.

(ii) The assertion follows directly from [7, Proposition 2.1.5].

(iii) Fix $d\in X$, we have

$\displaystyle\partial\langle f^{\prime}(\,\cdot\,),d\rangle(\hat{x})$
$\displaystyle=\bigg{\\{}L\in X^{*}\,|\,\langle
L,h\rangle\leq\limsup_{x\to\hat{x}\atop{\varepsilon\to 0^{+}}}\frac{\langle
f^{\prime}(x+\varepsilon h)-f^{\prime}(x),d\rangle}{\varepsilon},\ \ \forall
h\in X\bigg{\\}}$ $\displaystyle=\\{L\,|\,\langle L,h\rangle\leq
f^{\prime\prime}(\hat{x})(d,h),\ \ \forall h\in
X\\}=\\{L\,|\,L=f^{\prime\prime}(\hat{x})(d)\\}.$

Hence, $\partial^{2}f(\hat{x})(d)=f^{\prime\prime}(\hat{x})(d)$ for all $d\in
X.$

(iv) By [7, Proposition 2.3.1], we have

$\displaystyle\partial^{2}f(\hat{x})(sd)$ $\displaystyle=\partial\langle
f^{\prime}(\,\cdot\,),(sd)\rangle(\hat{x})=\partial(s\langle
f^{\prime}(\,\cdot\,),d\rangle)(\hat{x})$ $\displaystyle=s\partial\langle
f^{\prime}(\,\cdot\,),d\rangle(\hat{x})=s\partial^{2}f(\hat{x})(d).$

The second assertion follows directly from [7, Proposition 2.3.3]. ∎

To illustrate how to compute $\partial^{2}f(\hat{x})$ we give a simple example
for the case where $X=\mathbb{R}^{2}$.

######  Example 2.3.

Let $X=\mathbb{R}^{2}$ and $f(x,y)=\int_{0}^{x}|s|ds+y^{2}$. Then

$\partial^{2}f(0,0)=\Big{\\{}\Big{(}\begin{array}[]{cc}a&0\\\ 0&2\\\
\end{array}\Big{)}\;|\;a\in[-1,1]\Big{\\}}.$

In fact, we have $f^{\prime}(x,y)=(|x|,2y)$. Hence, for any $d=(d_{1},d_{2})$,
one has

$\langle f^{\prime}(x,y),d\rangle=d_{1}|x|+2d_{2}y.$

It follows that

$\displaystyle\partial^{2}f(0,0)(d)$
$\displaystyle=(d_{1}\partial(|x|)|_{x=0},2d_{2}\partial(y)|_{y=0})=\\{(ad_{1},2d_{2})\;|\;a\in[-1,1]\\}.$

Hence

$\partial^{2}f(0,0)(d)=\Big{\\{}\Big{(}\begin{array}[]{cc}a&0\\\ 0&2\\\
\end{array}\Big{)}\Big{(}\begin{array}[]{c}d_{1}\\\ d_{2}\\\
\end{array}\Big{)}\,|\,a\in[-1,1]\Big{\\}}.$

We obtain the desired formula.

The following mean value theorem plays an important role in our paper.

######  Theorem 2.4.

Let $f\in C^{1,1}(D)$. Then, for every $a,b\in D$ with $[a,b]\subset D$, there
exist $\xi\in(a,b)$ and $L\in\partial^{2}f(\xi)(b-a)$ such that

$f(b)-f(a)-\langle f^{\prime}(a),b-a\rangle=\frac{1}{2}\langle L,b-a\rangle.$

###### Proof.

For the proof we need the following lemma.

######  Lemma 2.5 (see [10, Proposition 1.14]).

Let $I$ be an open interval containing $[0,1]$ and $\phi\in C^{1,1}(I)$. Then,
there exits $t_{0}\in(0,1)$ such that

$\phi(1)-\phi(0)-\phi^{\prime}(0)\in\frac{1}{2}\partial\phi^{\prime}(t_{0}).$
(2.1)

We now define a function $\phi(t):=f(a+th)$, $t\in[0,1]$ with $h:=b-a$. It is
clear that $\phi$ satisfies properties of the above lemma. Therefore, there
exists $t_{0}\in(0,1)$ such that (2.1) is satisfied. Since
$\phi^{\prime}(t)=\langle f^{\prime}(a+th),h\rangle$, the chain rule (see [7,
Theorem 2.3.10]) implies that

$\displaystyle\partial\phi^{\prime}(t_{0})=\partial\langle
f^{\prime}(\,\cdot\,),h\rangle(a+t_{0}h)(h)=\partial\langle
f^{\prime}(\,\cdot\,),h\rangle(\xi)(h)=\partial^{2}f(\xi)(h)(h),$

where $\xi=a+t_{0}(b-a)\in(a,b)$. Hence, there exists
$L\in\partial^{2}f(\xi)(b-a)$ such that

$\displaystyle f(b)-f(a)-\langle
f^{\prime}(a),b-a\rangle=\phi(1)-\phi(0)-\phi^{\prime}(0)=\frac{1}{2}\langle
L,b-a\rangle.$

We obtained the desired conclusion of Theorem 2.1. ∎

Let $Y$ be a Banach space and $H\colon D\to Y$ be a mapping defined on $D$. We
say that $H$ is strictly Fréchet differentiable at $\hat{x}\in D$, if there
exists a linear continuous mapping $H^{\prime}(\hat{x})\colon X\to Y$ such
that for all $\varepsilon>0$, there exists $\delta>0$ with

$\|H(u)-H(v)-\langle H^{\prime}(\hat{x}),u-v\rangle\|\leq\varepsilon\|u-v\|$

whenever $u$ and $v$ satisfy $\|u-\hat{x}\|<\delta$ and
$\|v-\hat{x}\|<\delta$. It is easy to see that

$\langle H^{\prime}(\hat{x}),d\rangle=\lim_{x\to\hat{x}\atop{\varepsilon\to
0^{+}}}\frac{H(x+\varepsilon d)-H(x)}{\varepsilon}$

holds for all $d\in X$.

According to [21], when $H$ is strictly Fréchet differentiable at $\hat{x}$
and $d\in X$, then the second-order weak directional derivative of $H$ at
$\hat{x}$ in the direction $d$ is defined by

$H^{\prime\prime}(\hat{x};d):=\bigg{\\{}y\in Y\,|\,\liminf_{\varepsilon\to
0^{+}}\Big{\|}y-2\frac{H(\hat{x}+\varepsilon d)-H(\hat{x})-\varepsilon\langle
H^{\prime}(\hat{x}),d\rangle}{\varepsilon^{2}}\Big{\|}=0\bigg{\\}}.$

In other words, using the concept of the sequential Painlevé–Kuratowski upper
limit of [2], we have

$H^{\prime\prime}(\hat{x};d)=\mathop{{\rm Lim}\,{\rm sup}}_{\varepsilon\to
0^{+}}\Big{[}2\frac{H(\hat{x}+\varepsilon d)-H(\hat{x})-\varepsilon\langle
H^{\prime}(\hat{x}),d\rangle}{\varepsilon^{2}}\Big{]}.$

This set may be empty. If it is nonempty, then we say that $H$ is twice weakly
directionally differentiable at $\hat{x}$ in the direction $d$. It is clear
that when $H$ is of class $C^{2}$, then

$H^{\prime\prime}(\hat{x};d)=\\{H^{\prime\prime}(\hat{x})(d)(d)\\}.$

Now we compare the second-order weak directional derivative with the second-
order subdifferential in the sense of Definition 2.1.

######  Proposition 2.6.

Let $H$ be a real-valued function defined on $D$ and $\hat{x}\in D$. If $H\in
C^{1,1}(D)$, then $H$ is twice weakly directionally differentiable at
$\hat{x}$ in the any direction $d\in X$ and
$H^{\prime\prime}(\hat{x};d)\subset\partial^{2}H(\hat{x})(d)(d)$.

###### Proof.

Let $d\in X$ and $\varepsilon_{n}$ be an arbitrary positive sequence
converging to $0$ as $n\to\infty$. For each $n\in\mathbb{N}$, by Theorem 2.4,
there exist $t_{n}\in(0,1)$ and
$L_{n}\in\partial^{2}H(\hat{x}+t_{n}\varepsilon_{n}d)(d)$ such that

$H(\hat{x}+\varepsilon_{n}d)-H(\hat{x})-\varepsilon_{n}\langle
H^{\prime}(\hat{x}),d\rangle=\frac{1}{2}\varepsilon_{n}^{2}\langle
L_{n},d\rangle.$

By Proposition 2.2, we can assume that $L_{n}$ converges weakly∗ to
$L\in\partial^{2}f(\hat{x})(d)$. This implies that

$\lim\limits_{n\to\infty}2\dfrac{H(\hat{x}+\varepsilon_{n}d)-H(\hat{x})-\varepsilon_{n}\langle
H^{\prime}(\hat{x}),d\rangle}{\varepsilon^{2}_{n}}=\langle L,d\rangle.$

Thus, $\langle L,d\rangle\in H^{\prime\prime}(\hat{x};d)$ and
$H^{\prime\prime}(\hat{x};d)$ is nonempty.

To prove the second assertion, fix $y\in H^{\prime\prime}(\hat{x};d)$. Then
there exists a positive sequence $\varepsilon_{n}$ converging to $0$ such that

$\lim\limits_{n\to\infty}2\dfrac{H(\hat{x}+\varepsilon_{n}d)-H(\hat{x})-\varepsilon_{n}\langle
H^{\prime}(\hat{x}),d\rangle}{\varepsilon^{2}_{n}}=y.$

For the sequence $\varepsilon_{n}$, as in the proof of the first assertion,
there is $L\in\partial^{2}f(\hat{x})(d)$ such that

$\lim\limits_{n\to\infty}2\dfrac{H(\hat{x}+\varepsilon_{n}d)-H(\hat{x})-\varepsilon_{n}\langle
H^{\prime}(\hat{x}),d\rangle}{\varepsilon^{2}_{n}}=\langle L,d\rangle.$

This implies that $y=\langle L,d\rangle$ and we therefore get
$H^{\prime\prime}(\hat{x};d)\subset\partial^{2}H(\hat{x})(d)(d)$. ∎

The following result is immediate from the definition of the second-order weak
directional derivative and Proposition 2.6.

######  Corollary 2.7.

Let $H:=(h_{1},\ldots,h_{p})\colon D\to\mathbb{R}^{p}$ be a vector-valued
function and $\hat{x}\in D$. If $h_{i}\in C^{1,1}(D)$ for all $i=1,\ldots,p$,
then $H$ is twice weakly directionally differentiable at $\hat{x}$ in the any
direction $d\in X$ and

$H^{\prime\prime}(\hat{x};d)\subset
h_{1}^{\prime\prime}(\hat{x};d)\times\ldots
h_{p}^{\prime\prime}(\hat{x};d)\subset\partial^{2}h_{1}(\hat{x})(d)(d)\times\ldots\times\partial^{2}h_{p}(\hat{x})(d)(d).$

###  2.2. Second-order variations

In this section, we recall some concepts related to second-order variations
from [8, 21].

######  Definition 2.8.

Let $f$ be a real-valued function defined on $D$. A vector $\bar{w}\in X$ is
called a second-order descent variation of $f$ at $\hat{x}\in D$ in the
direction $d$ if there exists an $\bar{\varepsilon}>0$ such that
$\hat{x}+\varepsilon d+\varepsilon^{2}(\bar{w}+w)\in D$ and

$f(\hat{x}+\varepsilon d+\varepsilon^{2}(\bar{w}+w))<f(\hat{x})$

for all $\varepsilon\in(0,\bar{\varepsilon})$ and $\|w\|<\bar{\varepsilon}$.
The set of such $\bar{w}$ is denoted by $W^{2}_{\delta}(f;\hat{x},d)$. This
set is always open.

Let $\Omega$ be a nonempty subset in $X$, $\hat{x}\in\Omega$ and $d\in X$.

######  Definition 2.9.

A vector $\bar{w}\in X$ is said to be a second-order admissible variation of
$\Omega$ at $\hat{x}$ in the direction $d$ if there exists an
$\bar{\varepsilon}>0$ such that

$\hat{x}+\varepsilon d+\varepsilon^{2}(\bar{w}+w)\in\Omega$

for all $\varepsilon\in(0,\bar{\varepsilon})$ and $\|w\|<\bar{\varepsilon}$.
We denote this set by $W_{\alpha}^{2}(\Omega;\hat{x},d)$, which is always
open.

######  Definition 2.10.

The second-order tangent variation set  of $\Omega$ at $\hat{x}$ in the
direction $d$ is the set $W_{\tau}^{2}(\Omega;\hat{x},d)$ of vectors
$\bar{w}\in X$ such that there exist sequences $\varepsilon_{n}\to 0^{+}$ and
$w_{n}\to 0$ satisfying

$\hat{x}+\varepsilon_{n}d+\varepsilon_{n}^{2}(\bar{w}+w_{n})\in\Omega\ \
\text{for all }\ \ n\in\mathbb{N}.$

######  Remark 2.11.

  1. (i)

Denote by $d_{\Omega}(x)$ the distance of $x$ from $\Omega$; then the set of
all second-order tangent variations of $\Omega$ at $\hat{x}$ in the direction
$d$ can be formulated as follows:

$W_{\tau}^{2}(\Omega;\hat{x},d)=\Big{\\{}\bar{w}\,|\,\liminf_{\varepsilon\to
0^{+}}\dfrac{d_{\Omega}(\hat{x}+\varepsilon
d+\varepsilon^{2}\bar{w})}{\varepsilon^{2}}=0\Big{\\}}.$

  2. (ii)

It is easy to check that

$W_{\alpha}^{2}(X\setminus\Omega;\hat{x},d)=X\setminus
W_{\tau}^{2}(\Omega;\hat{x},d).$

  3. (iii)

Suppose that $f$ is a real-valued function defined on $D$ and $\hat{x}\in D$.
Then we have

$W^{2}_{\delta}(f;\hat{x},d)=W^{2}_{\alpha}(\Omega;\hat{x},d)\ \ \text{for all
}\ \ d\in X,$

where $\Omega:=\\{x\in D\,|\,f(x)<f(\hat{x})\\}$.

The following result gives a sufficient condition for a vector $w$ to be a
second-order descent variation of a given $C^{1,1}$ function on $D$.

######  Proposition 2.12.

Suppose that $f\in C^{1,1}(D)$, $\hat{x}\in D$ and $d\in X$ satisfying
$\langle f^{\prime}(\hat{x}),d\rangle\leq 0$. Denote

$W_{f}=\bigg{\\{}w\in X\,|\,\langle
f^{\prime}(\hat{x}),w\rangle+\frac{1}{2}\sup_{L\in\partial^{2}f(\hat{x})(d)}\langle
L,d\rangle<0\bigg{\\}}.$

Then, $W_{f}$ is an open and convex set, and the following inclusion holds
true

$W_{f}\subseteq W^{2}_{\delta}(f;\hat{x},d).$ (2.2)

###### Proof.

Clearly, $W_{f}$ is an open and convex set. We now prove inclusion (2.2). The
proof is indirect. Assume the opposite, i.e., there exists $\bar{w}\in W_{f}$
but $\bar{w}\notin W^{2}_{\delta}(f;\hat{x},d)$. Then, for each
$n\in\mathbb{N}$, there exist $\varepsilon_{n}\in(0,\frac{1}{n})$ and
$w_{n}\in X$ with $\|w_{n}\|<\frac{1}{n}$ such that at least one of the
following relations

$\displaystyle\hat{x}+\varepsilon_{n}d+\varepsilon^{2}_{n}(\bar{w}+w_{n})\in
D,$ $\displaystyle
f(\hat{x}+\varepsilon_{n}d+\varepsilon^{2}_{n}(\bar{w}+w_{n}))<f(\hat{x}),$

does not hold. For each $n\in\mathbb{N}$, put
$x_{n}=\hat{x}+\varepsilon_{n}d+\varepsilon^{2}_{n}(\bar{w}+w_{n})$. Clearly,
the sequence $\\{x_{n}\\}$ converges to $\hat{x}$ as $n\to\infty$. From the
openness of $D$ it follows that $x_{n}\in D$ for all $n$ sufficient large.
Thus, without loss of generality, we may assume that

$f(\hat{x}+\varepsilon_{n}d+\varepsilon^{2}_{n}(\bar{w}+w_{n}))\geq
f(\hat{x}),\ \ \forall n\in\mathbb{N}.$

This implies that

$\displaystyle\langle f^{\prime}(\hat{x}),d\rangle$
$\displaystyle+\varepsilon_{n}\Big{[}\frac{f(\hat{x}+\varepsilon_{n}d+\varepsilon_{n}^{2}(\bar{w}+w_{n}))-f(\hat{x}+\varepsilon_{n}d)}{\varepsilon_{n}^{2}}\Big{]}$
$\displaystyle\ \ \ \ \ \ \ \
+\varepsilon_{n}\Big{[}\frac{f(\hat{x}+\varepsilon_{n}d)-f(\hat{x})-\varepsilon_{n}\langle
f^{\prime}(\hat{x}),d\rangle}{\varepsilon_{n}^{2}}\Big{]}\geq 0,\ \ \forall
n\in\mathbb{N}.$ (2.3)

By Theorem 2.4, for each $n\in\mathbb{N}$, there exist $t_{n}\in(0,1)$ and

$L_{n}\in\partial^{2}f(\hat{x}+t_{n}\varepsilon_{n}d)(\varepsilon_{n}d)=\varepsilon_{n}\partial^{2}f(\hat{x}+t_{n}\varepsilon_{n}d)(d)$
(2.4)

such that

$f(\hat{x}+\varepsilon_{n}d)-f(\hat{x})-\varepsilon_{n}f^{\prime}(\hat{x};d)=\frac{1}{2}\langle
L_{n},\varepsilon_{n}d\rangle,\ \ \forall n\in\mathbb{N}.$

By (2.4), $L_{n}=\varepsilon_{n}H_{n}$ for some
$H_{n}\in\partial^{2}f(\hat{x}+t_{n}\varepsilon_{n}d)(d)$ and so

$f(\hat{x}+\varepsilon_{n}d)-f(\hat{x})-\varepsilon_{n}\langle
f^{\prime}(\hat{x}),d\rangle=\frac{1}{2}\varepsilon_{n}^{2}\langle
H_{n},d\rangle.$

It follows that

$\frac{f(\hat{x}+\varepsilon_{n}d)-f(\hat{x})-\varepsilon_{n}f^{\prime}(\hat{x};d)}{\varepsilon_{n}^{2}}=\frac{1}{2}\langle
H_{n},d\rangle.$

Hence, by (2.2), we have

$\langle
f^{\prime}(\hat{x}),d\rangle+\varepsilon_{n}\Big{[}\frac{f(\hat{x}+\varepsilon_{n}d+\varepsilon_{n}^{2}(\bar{w}+w_{n}))-f(\hat{x}+\varepsilon_{n}d)}{\varepsilon_{n}^{2}}\Big{]}+\frac{1}{2}\varepsilon_{n}\langle
H_{n},d\rangle\geq 0.$ (2.5)

Since $\partial^{2}f(\,\cdot\,)(d)$ is locally bounded near $\hat{x}$, we can
assume that $H_{n}$ converges weak∗ to $H_{0}$. By the upper semicontinuity of
the mapping $\partial^{2}f(\,\cdot\,)$, we have
$H_{0}\in\partial^{2}f(\hat{x})(d)$. Besides, one has

$\lim_{n\to\infty}\Big{[}\frac{f(\hat{x}+\varepsilon_{n}d+\varepsilon_{n}^{2}(\bar{w}+w_{n}))-f(\hat{x}+\varepsilon_{n}d)}{\varepsilon_{n}^{2}}\Big{]}=\langle
f^{\prime}(\hat{x}),\bar{w}\rangle.$

Letting $n\to\infty$ in (2.5) we obtain $\langle
f^{\prime}(\hat{x}),d\rangle\geq 0.$ Combining this with assumptions of the
proposition, we get $\langle f^{\prime}(\hat{x}),d\rangle=0$. Substituting
$\langle f^{\prime}(\hat{x}),d\rangle=0$ into (2.5) and dividing two sides by
$\varepsilon_{n}>0$, we get

$\Big{[}\frac{f(\hat{x}+\varepsilon_{n}d+\varepsilon_{n}^{2}(\bar{w}+w_{n}))-f(\hat{x}+\varepsilon_{n}d)}{\varepsilon_{n}^{2}}\Big{]}+\frac{1}{2}\langle
H_{n},d\rangle\geq 0.$

Passing the limit, we obtain

$\langle f^{\prime}(\hat{x}),\bar{w}\rangle+\frac{1}{2}\langle
H_{0},d\rangle\geq 0,$

contrary to the fact that $\bar{w}\in W_{f}$. The proof is complete. ∎

The following result presents a characterization of the second-order tangent
variation set to the null-set of a set-valued mapping between two general
Banach spaces.

######  Lemma 2.13 (see [21, Theorem 5]).

Assume that $H\colon D\to Y$ is strictly Fréchet differentiable at $\hat{x}\in
D$ such that $H^{\prime}(\hat{x})\colon X\to Y$ is surjective. Let
$\Omega:=\\{x\in X\,|\,H(x)=0\\}$. Then $\bar{w}\in
W^{2}_{\tau}(\Omega;\hat{x},d)$ if and only if $\langle
H^{\prime}(\hat{x}),d\rangle=0$, $H$ is twice weakly directionally
differentiable at $\hat{x}$ in the direction $d$ and the following condition
holds:

$0\in\langle
H^{\prime}(\hat{x}),\bar{w}\rangle+\frac{1}{2}H^{\prime\prime}(\hat{x};d).$

######  Definition 2.14.

Let $C\subset X$ be a nonempty convex set and $x\in C$.

  1. (i)

The normal cone to $C$ at $x$ is the set defined by

$N(C;x):=\\{x^{*}\in X^{*}\;|\;\langle x^{*},y-x\rangle\leq 0,\ \ \forall y\in
C\\}.$

  2. (ii)

The adjoint set of $C$ is the set defined by

$C^{+}:=\\{\varphi:X\to\mathbb{R}\;|\;\varphi\ \text{is affine and}\
\varphi(x)\geq 0,\ \ \forall x\in C\\}.$

Let $Q\subset X$ be a closed convex set with nonempty interior. The interior
of $Q$ is denoted by $Q^{\circ}$. Let $\hat{x}\in Q$ and $d\in X$. We define
the following set:

$Q^{\circ}(\hat{x},d):=\bigcup_{\bar{\varepsilon}>0}\bigcap_{\varepsilon<\bar{\varepsilon}\atop{\|w\|<\bar{\varepsilon}}}\Big{[}\frac{1}{\varepsilon^{2}}(Q-\hat{x}-\varepsilon
d)+w\Big{]}.$

This set plays an important role in the description of the second-order
necessary optimality condition for problem $(P)$; see [21, 22] for more
details. It is easy to see that $Q^{\circ}(\hat{x},d)$ is an open convex set
and $Q^{\circ}(\hat{x},d)=W^{2}_{\alpha}(Q;\hat{x},d).$ The nonemptyness of
$Q^{\circ}(\hat{x},d)$ is an important fact. As shown in [21], it is necessary
in order that $d\in\overline{\rm cone}\,(Q-\hat{x})$.

The following proposition follows directly from [21, Lemma 3 and Theorem 4].

######  Lemma 2.15.

Let $Q\subset X$ be a closed convex set with nonempty interior, $\hat{x}\in
Q$, and $d\in X$. Denote $C:={\rm cone}({\rm cone}(Q^{\circ}-\hat{x})-d).$
Then

  1. (i)

$\overline{Q^{\circ}(\hat{x},d)}+C\subset Q^{\circ}(\hat{x},d)$;

  2. (ii)

$Q^{\circ}(\hat{x},d)\subset C$. If in addition, $d\in{\rm cone}(Q-\hat{x})$,
then the inclusion is equality and we therefore get
$Q^{\circ}(\hat{x},d)\neq\emptyset$;

  3. (iii)

Let $d\in\overline{\rm{cone}}\,(Q-\hat{x})$ and $\phi(\,\cdot\,):=-\langle
x^{*},\,\cdot\,\rangle+t$ be an affine function defined on $X$, where
$x^{*}\in X^{*}$, $t\in\mathbb{R}$. Then, the function $\phi$ is bounded from
below on $C$ if and only if $x^{*}\in N(Q;\hat{x})$ and $x^{*}(d)=0$.
Moreover,

$C^{+}=\left\\{\phi(\,\cdot\,)=-\langle x^{*},\,\cdot\,\rangle+t\;|\;x^{*}\in
N(Q;\hat{x}),\;\langle x^{*},d\rangle=0,\;t\geq 0\right\\}.$

The following lemma is presented in [21, Lemma 2] without proof. Here we
include the proof for completeness.

######  Lemma 2.16.

Let $\gamma\colon X\to\mathbb{R}$ be a function which is convex and upper
semicontinuous. Denote

$C:=\\{x\in X\,|\,\gamma(x)<0\\}.$

Then, $C$ is open and convex. If $C$ is nonempty, then

$C^{+}=\\{\varphi\colon X\to\mathbb{R}\;|\;\varphi\ \text{is affine and}\
\exists\mu\geq 0:\varphi(x)+\mu\gamma(x)\geq 0,\ \ \forall x\in X\\}.$

###### Proof.

The openness and the convexity of $C$ are immediate from the upper
semicontinuity and the convexity of $\gamma$. Let $\varphi$ be an affine
function defined on $X$. It is easily seen that $\varphi\in C^{+}$ if and only
if the following convex system

$\begin{cases}\varphi(x)&<0,\\\ \gamma(x)&<0,\end{cases}$

has no solution $x\in X$. By Ky Fan’s Theorem [9, Theorem 1], the
inconsistency of the above system is equivalent to that there exist
$\lambda\geq 0$, $\mu\geq 0$, not all zero, such that

$\lambda\varphi(x)+\mu\gamma(x)\geq 0,\ \ \forall x\in X.$

Under the assumption $C\neq\emptyset$, we can choose $\lambda=1$, and so the
lemma follows. ∎

######  Lemma 2.17 (see [21, Lemma 1]).

Let $N$ be a positive integer and $C_{1},C_{2},\ldots,C_{N}$ be nonempty
convex sets in $X$ such that $C_{1},\ldots,C_{N-1}$ are open. Then

$\bigcap_{i=1}^{N}C_{i}=\emptyset$

if and only if there exist affine functions
$\varphi_{1},\varphi_{2},\ldots,\varphi_{N}\colon X\to\mathbb{R}$ not all
constant such that

$\sum_{i=1}^{N}\varphi_{i}=0,\ \ \varphi_{i}|_{C_{i}}\geq 0,\ \ \forall
i=1,2,\ldots,N.$

Define the support function of a nonempty set $C\subset X$ associated with
$x^{*}\in X^{*}$ by

$\delta^{*}(x^{*};C):=\begin{cases}\sup\\{\langle x^{*},c\rangle\,|\,c\in
C\\},&\ \ \text{if}\ \ C\neq\emptyset,\\\ -\infty,&\ \ \text{if}\ \
C=\emptyset.\end{cases}$

######  Lemma 2.18 (see [21, Lemma 4]).

Let $X$ and $Y$ be Banach spaces. Let $A\colon X\to Y$ be a bounded linear
operator that maps $X$ onto $Y$ and let $K\subset Y$ be a nonempty convex set.
Denote $C:=\\{x\in X\,|\,Ax\in K\\}.$ Then,

$C^{+}=\\{\varphi\colon X\to\mathbb{R}\,|\,\varphi\mbox{ is affine and
}\exists y^{*}\in Y^{*}\,:\,\varphi(x)\geq-\langle
y^{*},Ax\rangle+\delta^{*}(y^{*};K),\;x\in X\\}.$

##  3\. Optimal conditions

We now return to problem $(P)$. Put $J:=\\{1,\ldots,m\\}$. Hereafter, we use
the notation $Q_{1}:=\\{x\in D\,|\,G(x)\in Q\\}$ and $Q_{2}:=\\{x\in
D\,|\,H(x)=0\\}$.

######  Definition 3.1.

We say that $\hat{x}\in D\cap Q_{1}\cap Q_{2}$ is a weak Pareto efficient
solution of $(P)$ if there is no $x\in D\cap Q_{1}\cap Q_{2}$ such that

$F_{j}(x)-F_{j}(\hat{x})<0,\ \ \forall j=1,\ldots,m.$

The following lemma gives a necessary condition for a weak Pareto efficient
solution of $(P)$, which will be needed in the sequel. The idea of the proof
is from [3].

######  Lemma 3.2.

If $\hat{x}$ is a weak Pareto efficient solution of $(P)$, then

$\bigg{(}\bigcap_{j=1}^{m}W_{\delta}^{2}(f_{j};\hat{x},d)\bigg{)}\cap
W_{\alpha}^{2}(Q_{1};\hat{x},d)\cap W^{2}_{\tau}(Q_{2};\hat{x},d)=\emptyset.$

###### Proof.

Arguing by contradiction, assume that there exists $\bar{w}$ in the above
intersection. Then, there is $\bar{\varepsilon}>0$ such that

$\displaystyle\hat{x}+\varepsilon d+\varepsilon^{2}(\bar{w}+w)$
$\displaystyle\in D,$ $\displaystyle f_{j}(\hat{x}+\varepsilon
d+\varepsilon^{2}(\bar{w}+w))$ $\displaystyle<f_{j}(\hat{x}),\ \ j\in J,$
$\displaystyle\hat{x}+\varepsilon d+\varepsilon^{2}(\bar{w}+w)$
$\displaystyle\in Q_{1},$

hold for all $\|w\|<\bar{\varepsilon}$ and $0<\varepsilon<\bar{\varepsilon}$.
Furthermore, since $\bar{w}\in W^{2}_{\tau}(Q_{2};\hat{x},d)$, it follows that
there exist sequences $\varepsilon_{n}>0$, $w_{n}\in Z$ converging to zero
such that

$\hat{x}+\varepsilon_{n}d+\varepsilon_{n}^{2}(\bar{w}+w_{n})\in Q_{2},\ \
\forall n\in\mathbb{N}.$

Now choose $n_{0}$ large enough such that $\varepsilon_{n}<\bar{\varepsilon}$
and $\|w_{n}\|<\bar{\varepsilon}$ for all $n>n_{0}$. Then the sequence
$x_{n}:=\hat{x}+\varepsilon_{n}d+\varepsilon_{n}^{2}(\bar{w}+w_{n})$ converges
to $\hat{x}$ and

$x_{n}\in D\cap Q_{1}\cap Q_{2}\ \ \text{and}\ \ f_{j}(x_{n})<f_{j}(\hat{x}),\
\ \forall n>n_{0},$

which contradicts the optimality of $\hat{x}$. ∎

We say that $d\in X$ is a critical direction of $(P)$ at $\hat{x}$ if

$\begin{cases}\langle f_{j}^{\prime}(\hat{x}),d\rangle\leq 0,\ \ j\in J,\\\
\langle H^{\prime}(\hat{x}),d\rangle=0,\\\
G^{\prime}(\hat{x})d\in\overline{\text{cone}}(Q-G(\hat{x})).\end{cases}$

The set of all critical direction of $(P)$ at $\hat{x}$ is denoted by
$\mathcal{C}(\hat{x})$. A direction $d$ is called a regular direction at
$\hat{x}$ if $H^{\prime\prime}(\hat{x};d)\neq\emptyset$,
$G^{\prime\prime}(\hat{x},d)\neq\emptyset$ and
$Q^{\circ}(G(\hat{x}),G^{\prime}(\hat{x})d)\neq\emptyset$.

We now state the main result of the paper.

######  Theorem 3.3.

Assume that $\hat{x}$ is a weak Pareto efficient solution of $({P})$,
$f_{j}\in C^{1,1}(D)$, $j\in J$, $H$ and $G$ are strictly differentiable at
$\hat{x}$ such that $H^{\prime}(\hat{x})(X)$ is a closed subspace of $Y$.
Then, for all critical directions $d\in{\mathcal{C}}(\hat{x})$ and convex sets
$K\subset H^{\prime\prime}(\hat{x};d)$ and $M\subset
G^{\prime\prime}(\hat{x};d)$, there exist nonnegative numbers
$\mu_{1},\ldots,\mu_{m},$ and functionals $y^{*}\in Y^{*}$, $z^{*}\in Z^{*}$,
not all zero such that the following conditions hold:

  1. (i)

the complementarity conditions

$z^{*}\in N(Q;G(\hat{x}))\quad\text{and}\quad\langle
z^{*},G^{\prime}(\hat{x})d\rangle=0.$

  2. (ii)

the first-order necessary condition

$\sum_{j=1}^{m}\mu_{j}f^{\prime}_{j}(\hat{x})+H^{\prime}(\hat{x})^{*}y^{*}+G^{\prime*}(\hat{x})z^{*}=0,$

  3. (iii)

the second-order necessary condition

$\sum_{j=1}^{m}\mu_{j}\sup_{L\in\partial^{2}f_{j}(\hat{x})(d)}\langle
L,d\rangle\geq\delta^{*}(-y^{*};K)+\delta^{*}(-z^{*};M)+2\delta^{*}(z^{*};Q^{\circ}(G(\hat{x}),G^{\prime}(\hat{x})d)).$

We first prove this theorem for the case that $G(x)=x$ for all $x\in D$, i.e.,
$Q_{1}=D\cap Q$. In this case, problem $(P)$ is denoted by $(P_{1})$ and the
obtained result is as follows.

######  Theorem 3.4.

Assume that $\hat{x}$ is a weak Pareto efficient solution of $(P_{1})$,
$f_{j}\in C^{1,1}(D)$ for all $j\in J$, $H$ is strictly differentiable at
$\hat{x}$ such that $H^{\prime}(\hat{x})(X)$ is a closed subspace of $Y$. Let
$d$ be a critical direction of $(P_{1})$ at $\hat{x}$. Assume that $K$ is a
convex subset in $H^{\prime\prime}(\hat{x};d)$. Then, there exists
$(\mu,x^{*},y^{*})\in(\mathbb{R}^{m}_{+}\times X^{*}\times
Y^{*})\setminus\\{0\\}$ such that the following conditions hold:

  1. (i)

the complementarity conditions

$x^{*}\in N(Q;\hat{x}),\ \ \text{and}\quad\langle x^{*},d\rangle=0,$ (3.1)

  2. (ii)

the first-order necessary condition

$\sum_{j=1}^{m}\mu_{j}f_{j}^{\prime}(\hat{x})+H^{\prime}(\hat{x})^{*}y^{*}+x^{*}=0,$
(3.2)

  3. (iii)

the second-order necessary condition

$\sum_{j=1}^{m}\mu_{j}\sup_{L\in\partial^{2}f_{j}(\hat{x})(d)}\langle
L,d\rangle\geq\delta^{*}(-y^{*};K)+2\delta^{*}(x^{*};Q^{\circ}(\hat{x},d)).$
(3.3)

###### Proof.

We first prove the theorem when $d$ is a regular direction of $(P_{1})$ at
$\hat{x}$. Let us consider the following possible cases.

Case 1. There exists $j_{0}\in J$ such that $W_{f_{j_{0}}}=\emptyset$, where

$W_{f_{j_{0}}}:=\bigg{\\{}w\in X\,|\,\langle
f^{\prime}_{j_{0}}(\hat{x}),w\rangle+\frac{1}{2}\sup_{L\in\partial^{2}f_{j_{0}}(\hat{x})(d)}\langle
L,d\rangle<0\bigg{\\}}.$

Then, for all $w\in X$, we have

$\langle
f^{\prime}_{j_{0}}(\hat{x}),w\rangle+\frac{1}{2}\sup_{L\in\partial^{2}f_{j_{0}}(\hat{x})(d)}\langle
L,d\rangle\geq 0.$ (3.4)

We choose $\mu_{j_{0}}=1$, $\mu_{j}=0$ for all $j\in J\backslash\\{j_{0}\\}$,
$y^{*}=0$ and $x^{*}=0$. Fixing any $x\in X$ and substituting $w=tx$ with
$t>0$ into (3.4) and then dividing two sides by $t$, we have

$\langle
f^{\prime}_{j_{0}}(\hat{x}),x\rangle+\frac{1}{2t}\sup_{L\in\partial^{2}f_{j_{0}}(\hat{x})(d)}\langle
L,d\rangle\geq 0.$

Letting $t\to+\infty$, we get $\langle f^{\prime}_{j_{0}}(\hat{x}),x\rangle=0$
for all $x\in X$. This implies that $f^{\prime}_{j_{0}}(\hat{x})=0$.
Substituting $f^{\prime}_{j_{0}}(\hat{x})=0$ into (3.4), we have

$\sup_{L\in\partial^{2}f_{j_{0}}(\hat{x})(d)}\langle L,d\rangle\geq 0.$

Hence we obtain the conclusions of the theorem.

Case 2. $H^{\prime}(\hat{x})(X)$ is a proper subspace of $Y$. Since
$H^{\prime}(\hat{x})(X)$ is closed, by the separation theorem, there exists
$y_{0}^{*}\in Y^{*}\setminus\\{0\\}$ such that $y^{*}_{0}$ is identically zero
on the range of $H^{\prime}(\hat{x})$. To obtain the desired conclusions, we
take $\mu_{j}=0$ for all $j\in J$, $x^{*}=0$, and $y^{*}=y^{*}_{0}$ or
$y^{*}=-y^{*}_{0}$.

Case 3. $W_{f_{j}}\neq\emptyset$ for all $j\in J$ and
$H^{\prime}(\hat{x})(X)=Y.$

Put

$W_{H}=\Big{\\{}w\in X\,|\,H^{\prime}(\hat{x})w\in-\frac{1}{2}K\Big{\\}}$

and

$W_{Q}=W_{\alpha}^{2}(Q_{1};\hat{x},d)=Q^{\circ}(\hat{x},d).$

It is clear that $W_{H}$ and $W_{Q}$ are nonempty and convex sets. Moreover,
$W_{Q}$ is open. Thus, the sets $W_{f_{j}}$, $j\in J$, $W_{Q}$, and $W_{H}$
are nonempty. Furthermore, they are convex and the first $m+1$ sets are open.
By Proposition 2.12 and Lemma 2.13, we have

$W_{f_{j}}\subseteq W^{2}_{\delta}(f_{j};\hat{x},d),\quad W_{H}\subseteq
W^{2}_{\tau}(Q_{2};\hat{x},d).$

From Lemma 3.2 it follows that

$\bigg{(}\bigcap_{j=1}^{m}W_{\delta}^{2}(f_{j};\hat{x},d)\bigg{)}\cap
W_{\alpha}^{2}(Q_{1};\hat{x},d)\cap W^{2}_{\tau}(Q_{2};\hat{x},d)=\emptyset.$

Hence,

$\bigg{(}\bigcap_{j=1}^{m}W_{f_{j}}\bigg{)}\cap W_{Q}\cap W_{H}=\emptyset.$

It follows from Lemma 2.17 that there exist affine functions $\varphi_{j}$,
$j\in J$, $\phi_{Q}$ and $\phi_{H}$, not all constant, such that

$\varphi_{j}\in W_{f_{j}}^{+},j\in J,\phi_{Q}\in W_{Q}^{+},\phi_{H}\in
W_{H}^{+}$

and

$\sum_{j=0}^{m}\varphi_{j}+\phi_{Q}+\phi_{H}=0.$ (3.5)

By Lemma 2.16, there exist nonnegative numbers $\mu_{j}$ such that

$\varphi_{j}(x)+\mu_{j}\Big{(}\langle
f^{\prime}_{j}(\hat{x}),x\rangle+\frac{1}{2}\sup_{L\in\partial^{2}f_{j}(\hat{x})(d)}\langle
L,d\rangle\Big{)}\geq 0,\quad\forall x\in X,j\in J.$ (3.6)

Assume that $\phi_{Q}(\cdot)=-\langle x^{*},\,\cdot\,\rangle+t$, where
$x^{*}\in X^{*}$ and $t\in\mathbb{R}$. Since $\phi_{Q}\in W_{Q}^{+}$, we have
$\phi_{Q}(x)\geq 0$ for all $x\in W_{Q}=Q^{\circ}(\hat{x},d)$. From this and
Lemma 2.15(i) it follows that

$t\geq\delta^{*}(x^{*};Q^{\circ}(\hat{x},d))\ \ \text{and}\ \ -\langle
x^{*},u+x\rangle+t\geq 0$

for all $u\in\overline{Q^{\circ}(\hat{x},d)}$ and $x\in C$. Fix
$u_{0}\in\overline{Q^{\circ}(\hat{x},d)}$, then we have

$-\langle x^{*},x\rangle+t\geq\langle x^{*},u_{0}\rangle,\ \ \forall x\in C.$

Consequently, $\phi_{Q}$ is bounded from below on $C$. By Lemma 2.15(iii),
$x^{*}\in N(Q;\hat{x})$ and $\langle x^{*},d\rangle=0$. Clearly,

$\phi_{Q}(x)=-\langle x^{*},x\rangle+t\geq-\langle
x^{*},x\rangle+\delta^{*}(x^{*};Q^{\circ}(\hat{x},d)),\ \ \forall x\in X.$
(3.7)

By Lemma 2.18, there exists $y^{*}\in Y^{*}$ such that

$\phi_{H}(x)+\langle
y^{*}H^{\prime}(\hat{x}),x\rangle\geq\frac{1}{2}\delta^{*}(-y^{*};K),\ \
\forall x\in X.$ (3.8)

Adding inequalities (3.6)–(3.8) and using (3.5), we obtain

$\displaystyle\sum_{j=1}^{m}\mu_{j}\Big{(}\langle
f^{\prime}_{j}(\hat{x}),x\rangle+\frac{1}{2}\sup_{L\in\partial^{2}f_{j}(\hat{x})(d)}\langle
L,d\rangle\Big{)}$ $\displaystyle+\langle
H^{\prime}(\hat{x})^{*}y^{*},x\rangle+\langle x^{*},x\rangle$
$\displaystyle\geq\frac{1}{2}\delta^{*}(-y^{*};K)+\delta^{*}(x^{*},Q^{\circ}(\hat{x},d))$
(3.9)

for all $x\in X$. Fixing any $z\in X$ and substituting $x=tz$, where $t>0$,
into (3) and dividing two side by $t$, we obtain

$\displaystyle\sum_{j=1}^{m}\mu_{j}\langle f^{\prime}_{j}(\hat{x}),z\rangle$
$\displaystyle+\langle H^{\prime}(\hat{x})^{*}y^{*},z\rangle+\langle
x^{*},z\rangle$
$\displaystyle\geq-\frac{1}{2t}\sum_{j=1}^{m}\mu_{j}\sup_{L\in\partial^{2}f_{j}(\hat{x})(d)}\langle
L,d\rangle+\frac{1}{2t}\Big{(}\delta^{*}(-y^{*};K)+2\delta^{*}(x^{*},Q^{\circ}(\hat{x},d))\Big{)}.$
(3.10)

Letting $t\to+\infty$ in (3), we have

$\sum_{j=1}^{m}\mu_{j}\langle f^{\prime}_{j}(\hat{x}),z\rangle+\langle
H^{\prime}(\hat{x})^{*}y^{*},z\rangle+\langle x^{*},z\rangle\geq 0,\ \ \forall
z\in X.$

It follows that

$\sum_{j=1}^{m}\mu_{j}f^{\prime}_{j}(\hat{x})+H^{\prime}(\hat{x})^{*}y^{*}+x^{*}=0.$
(3.11)

Substituting this into (3), we have

$\sum_{j=1}^{m}\mu_{j}\sup_{L\in\partial^{2}f_{j}(\hat{x})(d)}\langle
L,d\rangle\geq\delta^{*}(-y^{*};K)+2\delta^{*}(x^{*},Q^{\circ}(\hat{x},d)).$

Then, $(\mu_{1},\ldots,\mu_{m},\lambda_{k},x^{*},y^{*})$ satisfies all
conditions (3.1)–(3.3). In this case, we claim that
$(\mu_{1},\ldots,\mu_{m},x^{*})\neq 0$. Indeed, we first show that at least
one of multipliers $\mu_{1},\ldots,\mu_{m},$ $x^{*},y^{*}$ is different from
zero. If otherwise, then, since (3.6)–(3.8), $\varphi_{j},j\in J$, $\phi_{Q}$
and $\phi_{H}$ must be all constants, a contradiction. Thus, if
$(\mu_{1},\ldots,\mu_{m},x^{*})=0$, then $y^{*}\neq 0$. Substituting this into
(3.11) we have $H^{\prime}(\hat{x})^{*}y^{*}=0$. Since
$H^{\prime}(\hat{x})X=Y$, we have $y^{*}=0$, contrary to the fact that
$y^{*}\neq 0$.

We now consider the case that $d$ is a nonregular critical direction of
$(P_{1})$ at $\hat{x}$. Clearly, $\hat{d}=0$ is a critical direction at
$\hat{x}$. Moreover, it is easy to check that
$H^{\prime\prime}(\hat{x};0)=\\{0\\}$ and

$Q^{\circ}(\hat{x};0)=\text{cone}\,(Q^{\circ}-\hat{x})\neq\emptyset.$

Thus $\hat{d}=0$ is also a regular direction of $(P)$ at $\hat{x}$. Now, apply
the above proof to $\hat{d}=0$ and $K=\\{0\\}$, there exist nonnegative
numbers $\mu_{1},\ldots,\mu_{m}$ and functionals $x^{*}\in N(Q;\hat{x})$,
$y^{*}\in Y^{*}$ not all zero satisfying condition (3.2). The nonregularity of
$d$ means that either $H^{\prime\prime}(\hat{x};d)$, or $Q^{\circ}(\hat{x};d)$
is empty. Thus the left-hand side of (3.3) equals positive infinity and
condition (3.3) is trivial. The proof is complete. ∎

Proof of Theorem 3.3. By introducing a new variable $z\in Z$, we can reduce
problem $(P)$ to the following problem:

$(\widetilde{P})\quad\begin{cases}\text{Min}\,_{\mathbb{R}^{m}_{+}}\widetilde{F}(x,z):=F(x)\\\
\text{subject to}\\\ (x,z)\in X\times Q,\\\
\widetilde{H}(x,z):=(H(x),G(x)-z)=(0,0).\end{cases}$

Notice that $\widetilde{H}^{\prime}(\hat{x},\hat{z})(X\times
Z)=H^{\prime}(\hat{x})X\times(G^{\prime}(\hat{x})X-Z)=H^{\prime}(\hat{x})X\times
Z$ is a closed subspace in $Y\times Z$. By Theorem 3.4, we can find
multipliers which satisfy the desired conclusion of the theorem. $\hfill\Box$

######  Remark 3.5.

It is worth noting that Theorem 3.3 embraces a first-order condition as a
special case. Indeed, let $\hat{x}$ be a weak Pareto efficient solution of
$(P)$, and denote by $\Lambda(\hat{x})$ the set of Lagrange multipliers
$(\mu,x^{*},y^{*})\in(\mathbb{R}^{m}_{+}\times X^{*}\times
Y^{*})\setminus\\{0\\}$ which satisfy conditions (i)–(ii) of Theorem 3.3.
Applying Theorem 3.3 for $d=0$, the set of Lagrange multipliers
$\Lambda(\hat{x})$ is always nonempty.

We finish this section by presenting a corollary of Theorem 3.4 for the case
that $H=(h_{1},\ldots,h_{p})$, $G=(g_{1},\ldots,g_{k})$, and
$Q=-\mathbb{R}^{k}_{+}$. The obtained result generalizes [10, Theorem 2.4] to
the multiobjective optimization case.

######  Corollary 3.6.

Consider problem $(P_{1})$ where $Q=-\mathbb{R}^{k}_{+}$ and
$H=(h_{1},\ldots,h_{p})$, $G=(g_{1},\ldots,g_{k})$ are vector-valued functions
with $C^{1,1}(D)$ components. Assume that $\hat{x}$ is a weak Pareto efficient
solution of $(P_{1})$. Then, for every critical direction $d$, there exist
$(\mu,\lambda,\beta)\in(\mathbb{R}^{m}_{+}\times\mathbb{R}^{k}_{+}\times\mathbb{R}^{p})\setminus\\{0\\},$
$L_{i}\in\partial^{2}f_{j}(\hat{x})(d)$, $j\in J$, and
$M_{i}\in\partial^{2}g_{i}(\hat{x})(d)$, $i=1,\ldots,k$,
$K_{l}\in\partial^{2}h_{l}(\hat{x})(d)$, $l=1,\ldots,p$ such that the
following conditions hold:

  1. (i)

the complementarity conditions

$\lambda_{i}g_{i}(\hat{x})=0,\ \ i=1,\ldots,k,$

  2. (ii)

the first-order necessary condition

$\sum_{j=1}^{m}\mu_{j}f_{j}^{\prime}(\hat{x})+\sum_{i=1}^{k}\lambda_{i}g^{\prime}_{i}(\hat{x})+\sum_{l=1}^{p}\beta_{l}h^{\prime}_{l}(\hat{x})=0,$

  3. (iii)

the second-order necessary condition

$\sum_{j=1}^{m}\mu_{j}\langle L_{j},d\rangle+\sum_{i=1}^{k}\lambda_{i}\langle
M_{i},d\rangle+\sum_{l=1}^{p}\beta_{l}\langle K_{l},d\rangle\geq 0.$

###### Proof.

By Corollary 2.7, $H^{\prime\prime}(\hat{x};d)$ and
$G^{\prime\prime}(\hat{x};d)$ are nonempty. Thus, every critical direction at
$\hat{x}$ is also regular. Moreover, we have

$\displaystyle H^{\prime\prime}(\hat{x};d)$
$\displaystyle\subset\partial^{2}h_{1}(\hat{x})(d)(d)\times\ldots\times\partial^{2}h_{p}(\hat{x})(d)(d),$
$\displaystyle G^{\prime\prime}(\hat{x};d)$
$\displaystyle\subset\partial^{2}g_{1}(\hat{x})(d)(d)\times\ldots\times\partial^{2}g_{k}(\hat{x})(d)(d).$

Hence, there exist $K_{l}\in\partial^{2}h_{l}(\hat{x})(d)$, $l=1,\ldots,p$,
and $M_{i}\in\partial^{2}g_{i}(\hat{x})(d)$, $i=1,\ldots,k$, such that

$\displaystyle\left(\langle K_{1},d\rangle,\ldots,\langle
K_{p},d\rangle\right)\in H^{\prime\prime}(\hat{x};d),$
$\displaystyle\left(\langle M_{1},d\rangle,\ldots,\langle
M_{k},d\rangle\right)\in G^{\prime\prime}(\hat{x};d).$

Applying Theorem 3.3 for $d\in\mathcal{C}(\hat{x})$, $Q=-\mathbb{R}^{k}_{+}$,
$K=\left\\{\left(\langle K_{1},d\rangle,\ldots,\langle
K_{p},d\rangle\right)\right\\}$, and $M=\left\\{\left(\langle
M_{1},d\rangle,\ldots,\langle M_{k},d\rangle\right)\right\\}$, there exist
multipliers which satisfy the desired conclusion of the corollary. ∎

## References

  * [1] V. M. Alekseev, V. M. Tikhimirov and S. V. Fomin, Optimal control, Plenum Publishing Corporation, New York, 1987.
  * [2] J. -P. Aubin and H. Frankowska, Set-valued analysis, Birkhäuser, Boston, 1990.
  * [3] A. Ben-Tal and J. Zowe, A unified theory of first and second order conditions for extremum problems in topological vector spaces, Math. Programming Stud. 19 (1982), 39–76.
  * [4] J. F. Bonnans, R. Cominetti and A. Shapiro, Second order optimality conditions based on parabolic second order tangent sets, SIAM J. Optim. 9 (1999), 466–492.
  * [5] J. M. Borwein and Q. J. Zhu, Techniques of variational analysis, Springer, New York, 2005.
  * [6] N. H. Chieu, G. M. Lee and N. D. Yen, Second-order subdifferentials and optimality conditions for $C^{1}$-smooth optimization problems, Appl. Anal. Optim. 1 (2017), 461–476.
  * [7] F. H. Clarke, Optimization and nonsmooth analysis, Wiley, New York, 1983.
  * [8] A. Y. Dubovitskii and A. A. Milyutin, Second variations in extremal problems with constraints, Dokl. Akad. Nauk SSSR. 160 (1965), 18–21.
  * [9] K. Fan, I. Glicksberg and A. J. Hoffman, Systems of inequalities involving convex functions, Proc. Amer. Math. Soc. 8 (1957), 617–622.
  * [10] P. G. Georgiev and N. P. Zlateva, Second-order subdifferentials of $C^{1,1}$ functions and optimality conditions, Set-Valued Anal. 4 (1996), 101–117.
  * [11] H. Gfrerer, Second-order optimality conditions for scalar and vector optimization problems in Banach spaces, SIAM J. Control Optim. 45 (2006), 972–997.
  * [12] N. Q. Huy and N. V. Tuyen, New second-order optimality conditions for a class of differentiable optimization problems, J. Optim. Theory Appl. 171 (2016), 27–44.
  * [13] N. Q. Huy, D. S. Kim and N. V. Tuyen, New second-order Karush–Kuhn–Tucker optimality conditions for vector optimization, Appl. Math. Optim. 79 (2019), 279–307.
  * [14] H. Kawasaki, An envelope like effect of infinitely many inequality constraints on second-order necessary conditions for minimization problems, Math. Program. 41 (1988), 73–96.
  * [15] A. A. Khan and C. Tammer, Second-order optimality conditions in set-valued optimization via asymptotic derivatives, Optimization 62 (2013), 743–758.
  * [16] A. A. Khan, B. Soleimani and C. Tammer, Second-order optimality conditions in set-valued optimization with variable ordering structure, Pure Appl. Funct. Anal. 2 (2017), 305–316.
  * [17] B. T. Kien, N. V. Tuyen and J. -C. Yao, Second-order KKT optimality conditions for multi-objective optimal control problems, SIAM J. Control Optim. 56 (2018), 4069-4097.
  * [18] D. S. Kim and N. V. Tuyen, A note on second-order Karush-Kuhn-Tucker necessary optimality conditions for smooth vector optimization problems, RAIRO-Oper. Res. 52 (2018), 567–575.
  * [19] B. S. Mordukhovich, Variational Analysis and Generalized Differentiation, I: Basic Theory; II: Applications, Springer, Berlin, 2006.
  * [20] B. S. Mordukhovich, Variational Analysis and Applications, Springer, New York, 2018.
  * [21] Z. Páles and V. M. Zeidan, Nonsmooth optimum problems with constraints, SIAM J. Control Optim. 32 (1994), 1476–1502.
  * [22] Z. Páles and V. M. Zeidan, First- and second-order necessary conditions for control problems with constraints, Trans. Amer. Math. Soc. 346 (1994), 421–453.
  * [23] Z. Páles and V. M. Zeidan, Infinite dimensional generalized Jacobian: Properties and calculus rules, J. Math. Anal. Appl. 344 (2008), 55–75.
  * [24] N. V. Tuyen, N. Q. Huy and D. S. Kim, Strong second-order Karush–Kuhn–Tucker optimality conditions for vector optimization, Appl. Anal. (2018). DOI:10.1080/00036811.2018.1489956
  * [25] X. Q. Yang and V. Jeyakumar, Generalized second-order directional derivatives and optimization with $C^{1,1}$ functions, Optimization 26 (1992), 165–185.