# Existence of efficient and properly efficient solutions to problems of
constrained vector optimization

DO SANG KIM1 1Department of Applied Mathematics, Pukyong National University,
Busan 48513, Republic of Korea<EMAIL_ADDRESS>,  BORIS S. MORDUKHOVICH2,3
2Department of Mathematics, Wayne State University, Detroit, MI 48202, USA
3RUDN University, Moscow 117198, Russia<EMAIL_ADDRESS>,  TIÊ´N-SO .N
PHẠM4 4Department of Mathematics, University of Dalat, 1 Phu Dong Thien Vuong,
Dalat, Vietnam<EMAIL_ADDRESS>and  NGUYEN VAN TUYEN5,6 5School of
Mathematical Sciences, University of Electronic Science and Technology of
China, Chengdu, P.R. China  6Department of Mathematics, Hanoi Pedagogical
University 2, Xuan Hoa, Phuc Yen, Vinh Phuc, Vietnam<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract.

The paper is devoted to the existence of global optimal solutions for a
general class of nonsmooth problems of constrained vector optimization without
boundedness assumptions on constraint sets. The main attention is paid to the
two major notions of optimality in vector problems: Pareto efficiency and
proper efficiency in the sense of Geoffrion. Employing adequate tools of
variational analysis and generalized differentiation, we first establish
relationships between the notions of properness, $M$-tameness, and the
Palais–Smale conditions formulated for the restriction of the vector cost
mapping on the constraint set. These results are instrumental to derive
verifiable necessary and sufficient conditions for the existence of Pareto
efficient solutions in vector optimization. Furthermore, the developed
approach allows us to obtain new sufficient conditions for the existence of
Geoffrion-properly efficient solutions to such constrained vector problems.

###### Key words and phrases:

Existence theorems $\cdot$ Pareto efficient solutions $\cdot$ Geoffrion-
properly efficient solutions $\cdot$ $M$-tameness $\cdot$ Palais–Smale
conditions $\cdot$ Properness

###### 1991 Mathematics Subject Classification:

90C29 $\cdot$ 90C30 $\cdot$ 90C31 $\cdot$ 49J30

The first author was supported by the National Research Foundation of Korea
Grant funded by the Korean Government (NRF-2016R1A2B4011589). Research of
second author was partially supported by the USA National Science Foundation
under grant DMS-1512846 and by the USA Air Force Office of Scientific Research
under grant #15RT0462. The third author was partially supported by Vietnam
National Foundation for Science and Technology Development (NAFOSTED) under
grant #101.04-2016.05.

##  1\. Introduction

This paper concerns some fundamental issues of global vector optimization that
are revolved around the existence of efficient and properly efficient
solutions under unbounded constraints. Such issues have been addressed in many
publications; see, e.g., the books [14, 17, 21] and the papers [1, 2, 5, 7, 8,
9, 10, 11] with the references therein. We offer here a new approach to these
topics that allows us to derive significantly new existence theorems for a
general class of problems in vector optimization. This approach is mainly
based on advanced tools of variational analysis and generalized
differentiation that provides essential improvements of known results even in
the case of problems with smooth data.

The basic problem under consideration is formulated as follows:

$\text{\rm
Min}_{\,\mathbb{R}^{m}_{+}}\big{\\{}f(x)\,\big{|}\,x\in\Omega\big{\\}},$ (VP)

where $f\colon\mathbb{R}^{n}\to\mathbb{R}^{m}$ is a locally Lipschitz mapping,
where $\Omega\subset\mathbb{R}^{n}$ is a nonempty and closed (not necessarily
bounded) set, and where “minimization” is understood in conventional terms of
vector optimization that are specified below.

In the case of unconstrained problems (VP) with $\Omega=\mathbb{R}^{n}$,
existence theorems for weak Pareto/weak efficient and the so-called relative
Pareto (while not Pareto efficient) solutions were obtained in [1, 2, 19] by
using appropriate set-valued extensions of the Ekeland variational principle
under the following major assumptions:

  * $\bullet$

$f$ is quasibounded from below, i.e., there exists a set
$M\subset\mathbb{R}^{m}$ such that

$\displaystyle f(\mathbb{R}^{n})\subset M+\mathbb{R}^{m}_{+}.$

  * $\bullet$

$f$ satisfies a certain Palais–Smale condition.

Somewhat related results for weak Pareto minimizers were obtained [10] under
more restrictive assumptions. As discussed in [16], such assumptions are
rather limited. To improve them, powerful methods of semialgebraic geometry
and polynomial optimization were invoked in [16]. In this way the equivalence
between the following conditions was proved therein when
$\Omega=\mathbb{R}^{n}$ and $f$ is polynomial in (VP); see below for the exact
definitions:

  * $\bullet$

$f$ is proper at the sublevel $\bar{y}$.

  * $\bullet$

$f$ satisfies the Palais–Smale condition at the sublevel $\bar{y}.$

  * $\bullet$

$f$ satisfies the weak Palais–Smale condition at the sublevel $\bar{y}.$

  * $\bullet$

$f$ is $M$-tame at the sublevel $\bar{y}.$

As consequences of these results, some sufficient conditions for the existence
of Pareto efficient solutions of the unconstrained polynomial problem (VP)
were given in [16].

The main contributions of this paper are significantly different from [16].
First of all, we study the constrained problem (VP) with an arbitrary closed
constraint set $\Omega$ and without any polynomial requirement on $f$, which
is now replaced by local Lipschitz continuity. To proceed, we do not use
methods of semialgebraic geometry but employ instead tools of variational
analysis and generalized differentiation. Our major results are as follows:

(a) Assuming that the image set $f(\Omega)$ has a bounded section at some
$\bar{y}\in f(\Omega)$, which is indeed necessary for the existence of Pareto
efficient solutions to (VP), we show that the following statements are
equivalent:

  * •

the restriction $f|_{\Omega}$ of $f$ on $\Omega$ is proper at the sublevel
$\bar{y}$.

  * •

the restriction $f|_{\Omega}$ satisfies the Palais–Smale condition at the
sublevel $\bar{y}.$

  * •

the restriction $f|_{\Omega}$ satisfies the weak Palais–Smale condition at the
sublevel $\bar{y}.$

  * •

the restriction $f|_{\Omega}$ is $M$-tame at the sublevel $\bar{y}.$

(b) Based on these results, we obtain necessary and sufficient conditions for
the existence of Pareto efficient solutions to problem (VP). As a byproduct of
our approach, new sufficient conditions for the existence of Geoffrion-
properly efficient solutions to (VP) are also derived.

The rest of the paper is organized as follows. In Section 2 we recall some
definitions and preliminary results from variational analysis and generalized
differentiation. Section 3 is devoted to establishing relationships between
properness, Palais–Smale conditions, and $M$-tameness. In Section 4 we prove
the existence of Pareto efficient and Geoffrion-properly efficient solutions
to the vector optimization problem (VP). The concluding Section 5 contains
discussions of open problems to address in our future research.

##  2\. Preliminaries

Our notation is terminology are standard in variational analysis and vector
optimization; see, e.g., the books [14, 18, 20]. Recall that for any number
$n\in\mathbb{N}:=\\{1,2,\ldots\\}$ we denote $x:=(x_{1},\ldots,x_{n})$ and
equip the space $\mathbb{R}^{n}$ with the usual scalar product
$\langle\cdot,\cdot\rangle$ and the Euclidean norm $\|\cdot\|$. The closed
unit ball in $\mathbb{R}^{n}$ is denoted by $\mathbb{B}^{n}$.

###  2.1. Definitions of optimal solutions

Let

$\displaystyle\mathbb{R}^{m}_{+}$ $\displaystyle:=$
$\displaystyle\big{\\{}y:=(y_{1},\ldots,y_{m})\in\mathbb{R}^{m}\,\big{|}\,\,y_{i}\geq
0,\;i=1,\ldots,m\big{\\}},$ $\displaystyle\mathrm{int}\,\mathbb{R}^{m}_{+}$
$\displaystyle:=$
$\displaystyle\big{\\{}y:=(y_{1},\ldots,y_{m})\in\mathbb{R}^{m}\,\big{|}\,\,y_{i}>0,\;i=1,\ldots,m\big{\\}}.$

Then $a\leqq b$ means $b-a\in\mathbb{R}^{m}_{+}$, $a\leq b$ means
$b-a\in\mathbb{R}^{m}_{+}\setminus\\{0\\}$, and $a<b$ means
$b-a\in\text{int}\,\mathbb{R}^{m}_{+}$ for any vectors $a,b\in\mathbb{R}^{m}$.

######  Definition 2.1.

Given $\bar{x}\in\Omega$, we say that

  1. (i)

$\bar{x}$ is a Pareto efficient solution to (VP) if there is no $x\in\Omega$
such that

$\displaystyle f(x)$ $\displaystyle\leqq$ $\displaystyle
f(\bar{x})\quad\textrm{ and }\quad f(x)\neq f(\bar{x}).$

  2. (ii)

$\bar{x}$ is a Geoffrion-properly efficient solution to (VP) if it is a Pareto
efficient solution and there is a real number $M>0$ such that whenever
$i\in\\{1,\ldots,m\\}$ and $x\in\Omega$ satisfying $f_{i}(x)<f_{i}(\bar{x})$
there exists an index $j\in\\{1,\ldots,m\\}$ with $f_{j}(\bar{x})<f_{j}(x)$
and

$\displaystyle\frac{f_{i}(\bar{x})-f_{i}(x)}{f_{j}(x)-f_{j}(\bar{x})}$
$\displaystyle\leq$ $\displaystyle M.$

It follows from the definitions that every Geoffrion-properly efficient
solution is a Pareto efficient solution to (VP) but not vice versa; see e.g.,
Example 4.1 below.

###  2.2. Normals and subdifferentials

Here we recall the notions of the normal cones to closed sets and the
subdifferential of real-valued functions used in this paper. The reader is
referred to [18, 20] for more details.

######  Definition 2.2.

Consider a set $\Omega\subset\mathbb{R}^{n}$ and a point $\bar{x}\in\Omega.$

  1. (i)

The regular normal cone (known also as the prenormal or Fréchet normal cone)
$\widehat{N}(\bar{x};\Omega)$ to $\Omega$ at $\bar{x}$ consists of all vectors
$v\in\mathbb{R}^{n}$ satisfying

$\displaystyle\langle v,x-\bar{x}\rangle$ $\displaystyle\leq$ $\displaystyle
o(\|x-\bar{x}\|)\quad\textrm{ as }\quad x\to\bar{x}\quad\textrm{ with }\quad
x\in\Omega.$

  2. (ii)

The limiting normal cone (known also as the basic or Mordukhovich normal cone)
$N(\bar{x};\Omega)$ to $\Omega$ at $\bar{x}$ consists of all vectors
$v\in\mathbb{R}^{n}$ such that there are sequences $x^{k}\to\bar{x}$ with
$x^{k}\in\Omega$ and $v^{k}\rightarrow v$ with
$v^{k}\in\widehat{N}(x^{k};\Omega)$ as $k\to\infty$.

######  Definition 2.3.

Consider a function $\phi\colon\mathbb{R}^{n}\to\mathbb{R}$ and a point
$\bar{x}\in\mathbb{R}^{n}$. The (limiting) subdifferential of $\phi$ at
$\bar{x}$ is defined by

$\displaystyle\partial\phi(\bar{x})$ $\displaystyle:=$
$\displaystyle\big{\\{}v\in\mathbb{R}^{n}\;\big{|}\;(v,-1)\in
N\big{(}(\bar{x},\phi(\bar{x}));\mathrm{epi}\,\phi\big{)}\big{\\}}$

via the limiting normal cone to the epigraph $\mathrm{epi}\,\phi$ of $\phi$
given by

$\displaystyle\mathrm{epi}\,\phi$ $\displaystyle:=$
$\displaystyle\big{\\{}(x,y)\in\mathbb{R}^{n}\times\mathbb{R}\,\big{|}\,\phi(x)\leq
y\big{\\}}.$

In [18, 19, 20] the reader can find equivalent analytic descriptions of the
subdifferential $\partial\phi(\bar{x})$ and comprehensive studies of it and
related constructions. In the case of convex sets and functions the above
normal cone and subdifferential notions reduce to the corresponding concepts
of convex analysis. Furthermore, we have
$\partial\phi(\bar{x})=\\{\nabla\phi(\bar{x})\\}$ if $\phi$ is strictly
differentiable at $\bar{x}$; in particular, when it is smooth around this
point.

Next we present several known statements, which play significant roles in the
proofs of the main results. The first lemma is the classical subdifferential
formula of convex analysis.

######  Lemma 2.1.

For each $\bar{x}\in\mathbb{R}^{n}$ we have

$\displaystyle\partial\left(\|\,\cdot\,-\bar{x}\|\right)(x)$ $\displaystyle=$
$\displaystyle\begin{cases}\displaystyle\frac{x-\bar{x}}{\|x-\bar{x}\|}&\textrm{
if }\;x\neq\bar{x},\\\ \mathbb{B}^{n}&\text{ otherwise.}\end{cases}$

The following major results of subdifferential calculus and necessary
optimality conditions for scalar nonsmooth optimization are used below in our
derivation of the existence theorems for (VP) even in the case of problems
with smooth initial data.

######  Lemma 2.2 (see [18, Theorem 3.36]).

Let the functions $\phi_{i}\colon\mathbb{R}^{n}\rightarrow{\mathbb{R}}$,
$i=1,\ldots,m$, be locally Lipschitzian around $\bar{x}\in\mathbb{R}^{n}$.
Then we have the subdifferential sum rule

$\displaystyle\partial(\phi_{1}+\cdots+\phi_{m})(\bar{x})$
$\displaystyle\subset$
$\displaystyle\partial\phi_{1}(\bar{x})+\cdots+\partial\phi_{m}(\bar{x}).$

######  Lemma 2.3 (see [18, Theorem 3.46]).

Let $\phi_{1},\ldots,\phi_{m}\colon{\mathbb{R}}^{n}\rightarrow{\mathbb{R}}$ be
locally Lipschitzian around $\bar{x}\in\mathbb{R}^{n}$. Then the maximum
function

$\displaystyle\phi(x):=\max_{1\leq i\leq m}\phi_{i}(x),\quad
x\in\mathbb{R}^{n},$

is locally Lipschitzian around $\bar{x}$, and we have the inclusion

$\displaystyle\partial\phi(\bar{x})$ $\displaystyle\subset$
$\displaystyle\left\\{\displaystyle\sum_{i\in
I(\bar{x})}\alpha_{i}\partial\phi_{i}(\bar{x})\;\Big{|}\;\alpha_{i}\geq
0\quad\textrm{ and }\quad\displaystyle\sum_{i\in
I(\bar{x})}\alpha_{i}=1\right\\},$

where the active index set is defined by
$I(\bar{x}):=\\{i\;|\;\phi_{i}(\bar{x})=\phi(\bar{x})\\}$.

######  Lemma 2.4 (see[18, Theorem 5.21(iii)]).

Let the function $\phi_{i}\colon\mathbb{R}^{n}\rightarrow{\mathbb{R}}$,
$i=0,\ldots,m$, be locally Lipschitzian $\bar{x}\in\Omega$, and let the set
$\Omega\subset\mathbb{R}^{n}$ be locally closed around this point. If
$\bar{x}$ is a local minimizer of the function $\phi_{0}$ on the set

$\displaystyle\big{\\{}x\in\Omega\,\big{|}\,\phi_{i}(x)\leq
0,\;i=1,\ldots,m\big{\\}},$

then there exist nonnegative numbers $\lambda_{i},\;i=0,\ldots,m$, with
$\sum_{i=1}^{m}\lambda_{i}=1$ such that

$\displaystyle
0\in\displaystyle\sum_{i=0}^{m}\lambda_{i}\partial\phi_{i}(\bar{x})+N(\bar{x};\Omega),$
$\displaystyle\lambda_{i}\phi_{i}(\bar{x})=0,\;i=1,\ldots,m.$

##  3\. Properness, Palais–Smale conditions, and M-Tameness

Let $f:=(f_{1},\ldots,f_{m})\colon\mathbb{R}^{n}\to\mathbb{R}^{m}$ be a
locally Lipschitz mapping, and let $\Omega\subset\mathbb{R}^{n}$ be a nonempty
and closed set. In this section we establish close relationships between the
properness, Palais–Smale conditions, and $M$-tameness for the restriction
$f|_{\Omega}$ of $f$ on $\Omega$. To proceed, the following notion is needed.

######  Definition 3.1.

Let $A$ be a subset in $\mathbb{R}^{m}$, and let $y\in\mathbb{R}^{m}$. The set
$A\cap(y-\mathbb{R}^{m}_{+})$ is called a section of $A$ at $y$ and is denoted
by $[A]_{y}.$ The section $[A]_{y}$ is said to be bounded if there exists a
vector $a\in\mathbb{R}^{m}$ such that

$\displaystyle[A]_{y}$ $\displaystyle\subset$ $\displaystyle
a+\mathbb{R}^{m}_{+}.$

It is easy to observe that for a Pareto efficient solution $\bar{x}$ to (VP)
we have

$\displaystyle[f(\Omega)]_{f(\bar{x})}$ $\displaystyle=$
$\displaystyle\big{\\{}f(\bar{x})\big{\\}}.$

Thus the condition that $f(\Omega)$ admits at least one bounded section is
necessary for the existence of Pareto efficient solutions to (VP).

Next we introduce the notions of properness for the restricted cost mapping,
which are instrumental to prove the existence of optimal solutions to (VP).

######  Definition 3.2.

We say that:

  1. (i)

The restriction $f|_{\Omega}$ of $f$ on $\Omega$ is proper at sublevel
$y\in\mathbb{R}^{m}$ if

$\forall\,\\{x^{k}\\}\;\subset\;\Omega,\;\|x^{k}\|\to\infty,\;f(x^{k})\;\leqq\;y\Longrightarrow\|f(x^{k})\|\to\infty\;\mbox{
as }\;k\to\infty.$

  2. (ii)

The restriction $f|_{\Omega}$ is proper if it is proper at every sublevel
$y\in\mathbb{R}^{m}$.

For each $\bar{y}\in\big{(}\mathbb{R}\cup\\{\infty\\}\big{)}^{m}$, consider
the sets

$\displaystyle\widetilde{K}_{\infty,\leqq\bar{y}}\,(f,\Omega)$
$\displaystyle:=$
$\displaystyle\big{\\{}y\in\mathbb{R}^{m}\big{|}\,\exists\,\\{x^{k}\\}\subset\Omega,\;f(x^{k})\leqq\bar{y},\;\|x^{k}\|\to\infty,f(x^{k})\to
y,\;\mbox{ and}$ $\displaystyle\qquad\qquad\qquad\nu(x^{k})\to 0\;\mbox{ as
}\,k\to\infty\\},$ $\displaystyle{K}_{\infty,\leqq\bar{y}}\,(f,\Omega)$
$\displaystyle:=$
$\displaystyle\big{\\{}y\in\mathbb{R}^{m}\,\big{|}\,\exists\,\\{x^{k}\\}\subset\Omega,\;f(x^{k})\leqq\bar{y},\;\|x^{k}\|\to\infty,\;f(x^{k})\to
y,\;\mbox{ and}$ $\displaystyle\qquad\qquad\qquad\|x^{k}\|\nu(x^{k})\to
0\;\mbox{ as }\;k\to\infty\\},$

where $\nu\colon\mathbb{R}^{n}\to\mathbb{R}$ is the (extended) Rabier function
defined by

$\displaystyle\nu(x):=\inf\Big{\\{}\Big{\|}\sum_{i=1}^{m}\lambda_{i}v^{i}+w\Big{\|}\;\Big{|}\;v^{i}\in\partial
f_{i}(x)\cup\partial(-f_{i})(x),\,w\in
N(x;\Omega),\,\lambda\in\mathbb{R}_{+}^{m},\,\displaystyle\sum_{i=1}^{m}\lambda_{i}=1\Big{\\}}.$
(1)

Following [12, Chapter 2], we also consider the set

$\displaystyle T_{\infty,\leqq\bar{y}}\,(f,\Omega)$ $\displaystyle:=$
$\displaystyle\big{\\{}y\in\mathbb{R}^{m}\;\big{|}\;\exists\,\\{x^{k}\\}\subset\Gamma(f,\Omega),\;f(x^{k})\leqq\bar{y},\;\|x^{k}\|\to\infty,\;\mbox{
and }$ $\displaystyle\qquad\qquad\qquad f(x^{k})\to y\;\mbox{ as
}\;k\to\infty\\},$

defined via the construction

$\displaystyle\Gamma(f,\Omega)\ :=\ \Big{\\{}x\in\Omega$
$\displaystyle\Big{|}$ $\displaystyle\exists v^{i}\in\partial
f_{i}(x)\cup\partial(-f_{i})(x)\ \textrm{ for }\ i=1,\ldots,m,$
$\displaystyle\exists(\lambda,\mu)\in\mathbb{R}_{+}^{m}\times\mathbb{R}\
\textrm{ with }\ \displaystyle\sum_{i=1}^{m}\lambda_{i}+|\mu|=1\textrm{ such
that }$ $\displaystyle 0\ \in\displaystyle\sum_{i=1}^{m}\lambda_{i}v^{i}+\mu
x+N(x;\Omega)\Big{\\}}.$

######  Remark 3.1.

It follows from the proofs given below that the results of Theorems 3.1, 4.2
and Corollaries 3.1, 4.1 hold true if in the definitions of the function $\nu$
and the set $\Gamma(f,\Omega)$ we delete the terms $\partial(-f_{i})(x)$. We
include this in the original definitions for simplicity and for the
unification with other results of the paper.

When $\bar{y}=(\infty,\ldots,\infty)$, we simplify the notation by writing
$\widetilde{K}_{\infty}(f,\Omega),$ ${K}_{\infty}(f,\Omega),$ and
$T_{\infty}(f,\Omega)$ instead of
$\widetilde{K}_{\infty,\leqq\bar{y}}(f,\Omega),$
$K_{\infty,\leqq\bar{y}}(f,\Omega),$ and $T_{\infty,\leqq\bar{y}}(f,\Omega)$,
respectively.

It follows from the definitions that the properness of $f|_{\Omega}$ at
sublevel $\bar{y}\in\mathbb{R}^{m}$ yields

$\displaystyle T_{\infty,\leqq\bar{y}}(f,\Omega)$ $\displaystyle=$
$\displaystyle\widetilde{K}_{\infty,\leqq\bar{y}}\,(f,\Omega)\ =\
{K}_{\infty,\leqq\bar{y}}\,(f,\Omega)\ =\ \emptyset.$

The converse does not hold in general. Indeed, let $\Omega:=\mathbb{R}^{2}$,
and let $f\colon\mathbb{R}^{2}\to\mathbb{R}$ be a real-valued function defined
by $f(x_{1},x_{2}):=x_{1}+x_{2}$. It is easy to check that

$\displaystyle
T_{\infty,\leqq\bar{y}}\,(f,\Omega)=\widetilde{K}_{\infty,\leqq\bar{y}}\,(f,\Omega)={K}_{\infty,\leqq\bar{y}}\,(f,\Omega)=\emptyset\quad\textrm{
for all }\quad\bar{y}\in\mathbb{R},$

while $f$ is not proper at every sublevel. Nevertheless, we have the following
rather surprising result the proof of which is based on variational arguments
and subdifferential calculus.

######  Theorem 3.1.

Assume that there exists $\bar{y}\in f(\Omega)$ such that the section
$[f(\Omega)]_{\bar{y}}$ is bounded. Then the following statements are
equivalent:

  1. (i)

$f|_{\Omega}$ is proper at the sublevel $\bar{y}$.

  2. (ii)

$f|_{\Omega}$ satisfies the Palais–Smale condition at $\bar{y}$, i.e.,
$\widetilde{K}_{\infty,\leqq\bar{y}}\,(f,\Omega)=\emptyset$.

  3. (iii)

$f|_{\Omega}$ satisfies the weak Palais–Smale condition at $\bar{y}$, i.e.,
${K}_{\infty,\leqq\bar{y}}\,(f,\Omega)=\emptyset$.

  4. (iv)

$f|_{\Omega}$ is M-tame at the sublevel $\bar{y}$, i.e.,
$T_{\infty,\leqq\bar{y}}\,(f,\Omega)=\emptyset$.

Furthermore, the set $[f(\Omega)]_{\bar{y}}$ is nonempty and compact provided
that one of the above equivalent conditions is satisfied.

Proof. Note first that implications (i)$\Rightarrow$(ii),
(ii)$\Rightarrow$(iii), and (i)$\Rightarrow$(iv) are obvious.

To prove (iii)$\Rightarrow$(i), we argue by contradiction and assume that $f$
is not proper at the sublevel $\bar{y}$. Since the section
$Y:=[f(\Omega)]_{\bar{y}}$ is bounded, there exists
$\widehat{y}\in\mathbb{R}^{m}$ with $\widehat{y}\not\in Y$ and
$\widehat{y}\leqq\bar{y}$. Consider the nonempty set

$\displaystyle X$ $\displaystyle:=$ $\displaystyle
f^{-1}(Y)\cap\Omega=\big{\\{}x\in\Omega\,\big{|}\,f(x)\leqq\bar{y}\big{\\}}\neq\emptyset$
(2)

and define the function $\phi\colon\mathbb{R}^{n}\rightarrow\mathbb{R}$ by

$\displaystyle\phi(x)$ $\displaystyle:=$
$\displaystyle\max_{i=1,\ldots,m}|f_{i}(x)-\widehat{y}_{i}|.$ (3)

It is clear that the function $\phi$ is nonnegative, locally Lipschitzian and
satisfies the condition $\phi(x)>0$ for all $x\in X$. Since $f$ is not proper
at the sublevel $\bar{y}$, we see that the number

$\displaystyle c$ $\displaystyle:=$ $\displaystyle\liminf\limits_{x\in
X,\,\|x\|\to\infty}\phi(x)$ (4)

is finite. For each $R>0$ consider the quantity

$\displaystyle\mathfrak{m}(R)$ $\displaystyle:=$
$\displaystyle\inf\limits_{x\in X,\,\|x\|\geq R}\phi(x)$

and observe that $\mathfrak{m}$ is a nondecreasing and nonnegative function
with $\lim\limits_{R\to\infty}\mathfrak{m}(R)=c.$ Thus for each
$k\in\mathbb{N}$ there exists $R_{k}>k$ satisfying

$\displaystyle\mathfrak{m}(R)$ $\displaystyle\geq$ $\displaystyle
c-\frac{1}{2k}\quad\textrm{whenever}\quad R\geq R_{k}.$

Choose now $x^{k}\in X$ with $\|x^{k}\|>2R_{k}$ and such that

$\displaystyle\phi(x^{k})$ $\displaystyle<$
$\displaystyle\mathfrak{m}(2R_{k})+\frac{1}{2k}.$

We clearly have the chain of inequalities

$\displaystyle\mathfrak{m}(2R_{k})\ \leq\ \phi(x^{k})$ $\displaystyle<$
$\displaystyle\mathfrak{m}(2R_{k})+\frac{1}{2k}$ $\displaystyle\leq$
$\displaystyle c+\frac{1}{2k}$ $\displaystyle\leq$
$\displaystyle\mathfrak{m}(R_{k})+\frac{1}{k}$ $\displaystyle=$
$\displaystyle\inf\limits_{x\in X,\,\|x\|\geq R_{k}}\phi(x)+\frac{1}{k}.$

We are now in a position to apply the Ekeland variational principle [6] (see,
e.g., [18, Theorem 2.26]) to the function $\phi$ on the set $\\{x\in
X\,|\,\|x\|\geq R_{k}\\}$ with the parameters $\varepsilon:=\frac{1}{3k}$ and
$\lambda:=\dfrac{\|x^{k}\|}{2}$ therein. Note that in the finite-dimensional
setting under consideration this result and other variational principles can
be proved easily; see [19, Theorem 2.12]. In this way we find $u^{k}\in X$
with $\|u^{k}\|\geq R_{k}$ satisfying the following conditions:

  * (a)

$\mathfrak{m}(R_{k})\leq\phi(u^{k})\leq\phi(x^{k}),$

  * (b)

$\|u^{k}-x^{k}\|\leq\lambda$, and

  * (c)

$\phi(x)+\dfrac{\varepsilon}{\lambda}\|x-u^{k}\|\geq\phi(u^{k})$ for all $x\in
X$ with $\|x\|\geq R_{k}.$

It follows from (a) that $\phi(u^{k})\to c$ as $k\to\infty$, while (b) yields

$\displaystyle R_{k}$ $\displaystyle<$
$\displaystyle\displaystyle\dfrac{\|x^{k}\|}{2}\ \leq\ \|u^{k}\|\ \leq\
\dfrac{3}{2}\|x^{k}\|$

and implies, in particular, that $\|u^{k}\|\to\infty$ as $k\to\infty$.
Applying the necessary optimality conditions from Lemma 2.4 to the nonsmooth
scalar optimization problem in (c) allows us to find
$(\kappa,\beta)\in\mathbb{R}_{+}\times\mathbb{R}^{m}_{+}$ for which

$\displaystyle
0\in\kappa\,\partial\left[\phi(\cdot)+\dfrac{\varepsilon}{\lambda}\|\cdot-u^{k}\|\right](u^{k})+\displaystyle\sum_{i=1}^{m}\beta_{i}\partial
f_{i}(u^{k})+N(u^{k};\Omega),$ (5)
$\displaystyle\beta_{i}\big{(}f_{i}(u^{k})-\bar{y}_{i}\big{)}=0\ \textrm{ for
}\ i=1,\ldots,m,\quad\textrm{ and }\quad
1=\kappa+\displaystyle\sum_{i=1}^{m}\beta_{i}.$

The subdifferential sum rule from Lemma 2.2 and the calculation of Lemma 2.1
give us

$\displaystyle\partial\left[\phi(\cdot)+\displaystyle\dfrac{\varepsilon}{\lambda}\|\cdot-u^{k}\|\right](u^{k})$
$\displaystyle\subset$
$\displaystyle\partial\phi(u^{k})+\dfrac{\varepsilon}{\lambda}\mathbb{B}^{n}.$
(6)

In order to evaluate the subdifferential $\partial\phi(u^{k})$ of the maximum
function (3) in our setting, define the index sets

$\displaystyle I_{1}^{\pm}$ $\displaystyle:=$
$\displaystyle\big{\\{}i\,\big{|}\,f_{i}(u^{k})-\widehat{y}_{i}=\pm\phi(u^{k})\big{\\}},$
$\displaystyle I_{2}$ $\displaystyle:=$
$\displaystyle\big{\\{}i\,\big{|}\,f_{i}(u^{k})-\bar{y}_{i}=0\big{\\}}.$

Since $\phi(u^{k})>0$, we get $I_{1}^{-}\cap I_{1}^{+}=\emptyset$.
Furthermore, the condition $\widehat{y}\leq\bar{y}$ ensures that
$I_{1}^{-}\cap I_{2}=\emptyset$. Then Lemma 2.3 tells us that

$\displaystyle\partial\phi(u^{k})$ $\displaystyle\subset$
$\displaystyle\left\\{\displaystyle\sum_{i\in I_{1}^{+}}\alpha_{i}\partial
f_{i}(u^{k})+\sum_{i\in
I_{1}^{-}}\alpha_{i}\partial(-f_{i})(u^{k})\,\Big{|}\,\alpha_{i}\geq
0,\;\displaystyle\sum_{i\in I_{1}^{+}\cup I_{1}^{-}}\alpha_{i}=1\right\\}.$

This, together with (5) and (6), implies that there exist numbers
$\alpha_{i}\geq 0$ for $i\in I_{1}^{+}\cup I_{1}^{-}$ with $\sum_{i\in
I_{1}^{+}\cup I_{1}^{-}}\alpha_{i}=1$ such that

$\displaystyle 0$ $\displaystyle\in$ $\displaystyle\displaystyle\sum_{i\in
I_{1}^{+}}\kappa\alpha_{i}\partial f_{i}(u^{k})+\sum_{i\in
I_{1}^{-}}\kappa\alpha_{i}\partial(-f_{i})(u^{k})+\sum_{i=1}^{m}\beta_{i}\partial
f_{i}(u^{k})+N(u^{k};\Omega)+\kappa\dfrac{\varepsilon}{\lambda}\mathbb{B}^{n}.$

For each $i\in I^{+}_{1}\cup I^{-}_{1}\cup I_{2}$, we denote

$\displaystyle\lambda_{i}$ $\displaystyle:=$
$\displaystyle\begin{cases}\kappa\alpha_{i}&\textrm{ if }\;i\in
I^{+}_{1}\setminus I_{2},\\\ \kappa\alpha_{i}+\beta_{i}&\textrm{ if }\;i\in
I^{+}_{1}\cap I_{2},\\\ \beta_{i}&\textrm{ if }\;i\in I_{2}\setminus
I^{+}_{1},\\\ \kappa\alpha_{i}&\textrm{ if }\;i\in I^{-}_{1},\\\ 0&\textrm{
otherwise,}\end{cases}$

and arrive at the following relationships

$\displaystyle 0\ \in\ \displaystyle\sum_{i\not\in
I_{1}^{-}}\lambda_{i}\partial f_{i}(u^{k})+\sum_{i\in
I_{1}^{-}}\lambda_{i}\partial(-f_{i})(u^{k})+N(u^{k};\Omega)+\kappa\dfrac{\varepsilon}{\lambda}\mathbb{B}^{n},$
$\displaystyle\lambda_{i}\geq 0\textrm{ for }i=1,\ldots,m,\quad\textrm{and
}\quad\displaystyle\sum_{i=1}^{m}\lambda_{i}\ =\ \sum_{i\in I^{+}_{1}\cup
I^{-}_{1}}\kappa\alpha_{i}+\sum_{i\in I_{2}}\beta_{i}\ =\
\kappa+\displaystyle\sum_{i\in I_{2}}\beta_{i}\ =\ 1.$

Observe that if we choose $\widehat{y}$ so that $\widehat{y}<y$ for all $y\in
Y,$ then $I_{1}^{-}=\emptyset;$ cf. Remark 3.1. It follows from the definition
of the Rabier function $\nu$ that

$\displaystyle\nu(u^{k})$ $\displaystyle\leq$
$\displaystyle\kappa\frac{\varepsilon}{\lambda}\ \leq\
\frac{\varepsilon}{\lambda}\ =\ \frac{2}{3k\|x^{k}\|}\ \leq\
\frac{1}{k\|u^{k}\|}.$

Consequently, we get the estimate

$\displaystyle\|u^{k}\|\nu(u^{k})$ $\displaystyle\leq$
$\displaystyle\displaystyle\frac{1}{k}\;\mbox{ for each }\;k\in\mathbb{N},$

and therefore $\|u^{k}\|\nu(u^{k})\to 0$ as $k\to\infty.$

On the other hand, it follows from the boundedness of the section
$[f(\Omega)]_{\bar{y}}$ and the inclusion
$\\{f(u^{k})\\}\subset[f(\Omega)]_{\bar{y}}$ that the sequence
$\\{f(u^{k})\\}$ has an accumulation point, say $y\in\mathbb{R}^{m}$. Thus
$y\in K_{\infty,\leqq\bar{y}}\,(f,\Omega),$ a contradiction that verifies
implication (iii)$\Rightarrow$(i).

Next we prove (iv)$\Rightarrow$(i). Assume on the contrary that $f$ is not
proper at the sublevel $\bar{y}$. Then there exists a sequence
$\\{x^{k}\\}\subset\Omega$ such that $\|x^{k}\|\to\infty$ as $k\to\infty,$
$f(x^{k})\leqq\bar{y},$ and the sequence of images $\\{f(x^{k})\\}$ is
bounded.

Since the section $Y:=[f(\Omega)]_{\bar{y}}$ is bounded, there exists
$\widehat{y}\in\mathbb{R}^{m}$ with $\widehat{y}\not\in Y$ and
$\widehat{y}\leq\bar{y}.$ As above, consider the set $X$ from (2), the maximum
function $\phi(x)$ from (3), and then conclude that the number $c$ defined in
(4) is finite. For each $k\in\mathbb{N}$ we form the following scalar
nonsmooth optimization problem:

minimize $\displaystyle\phi(x)$ subject to $\displaystyle x\in X\quad{\rm
and}\quad\|x\|^{2}-\|x^{k}\|^{2}=0.$

Since the constraint set here is nonempty and compact, this problem admits an
optimal solution denoted by $v^{k}$. The usage of necessary optimality
conditions from Lemma 2.4 give us a triple
$(\kappa,\beta,\mu)\in\mathbb{R}_{+}\times\mathbb{R}^{m}_{+}\times\mathbb{R}$
satisfying the relationships

$\displaystyle
0\in\kappa\,\partial\phi(v^{k})+\displaystyle\sum_{i=1}^{m}\beta_{i}\partial\big{(}f_{i}(\cdot)-\bar{y}_{i}\big{)}(v^{k})+2\mu
v^{k}+N(v^{k};\Omega),$
$\displaystyle\beta_{i}\big{(}f_{i}(v^{k})-\bar{y}_{i}\big{)}=0\ \textrm{ for
}\;i=1,\ldots,m\quad\textrm{ and }\quad
1=\kappa+\displaystyle\sum_{i=1}^{m}\beta_{i}+|\mu|.$

Proceeding now as in the proof of the previous implication which taking into
account the subdifferential sum rule and the subdifferential calculation for
the maximum function together with the modified form of the cost function, we
arrive at the conditions

$\displaystyle 0\ \in\ \displaystyle\sum_{i\not\in
I_{1}^{-}}\lambda_{i}\partial f_{i}(v^{k})+\sum_{i\in
I_{1}^{-}}\lambda_{i}\partial(-f_{i})(v^{k})+2\mu v^{k}+N(v^{k};\Omega),$
$\displaystyle\displaystyle\sum_{i=1}^{m}\lambda_{i}+2|\mu|\ =\ \sum_{i\in
I^{+}_{1}\cup I^{-}_{1}}\kappa\alpha_{i}+\sum_{i\in I_{2}}\beta_{i}+2|\mu|\ =\
\kappa+\sum_{i\in I_{2}}\beta_{i}+2|\mu|\ =\ 1+|\mu|\ \geq\ 1$

with the same index sets and the expressions for $\lambda_{i}$ as above. We
clearly get $v^{k}\in\Gamma(f,\Omega)$.

Thus we constructed the sequence $\\{v^{k}\\}$ with the following properties:

  1. (a)

$\\{v^{k}\\}\ \subset\ \Gamma(f,\Omega)$;

  2. (b)

$\|v^{k}\|\ =\ \|x^{k}\|\to\infty$ as $k\to\infty$;

  3. (c)

$\phi(v^{k})\ \leq\ \phi(x^{k})$ for all $k\in\mathbb{N}$;

  4. (d)

$f(v^{k})\ \leqq\ \bar{y}$ for all $k\in\mathbb{N}$.

It follows from the boundedness of the section $[f(\Omega)]_{\bar{y}}$ and the
inclusion $\\{f(v^{k})\\}\subset[f(\Omega)]_{\bar{y}}$ that the sequence
$\\{f(v^{k})\\}$ has an accumulation point $y\in\mathbb{R}^{m}$. Therefore
$y\in T_{\infty,\leqq\bar{y}}\,(f,\Omega)$, a contradiction. This completes
the proof of the equivalence between all the properties (i)–(iv).

Let us finally verify the last statement of the theorem. Suppose that (i)
holds and then show that the set $[f(\Omega)]_{\bar{y}}$ is closed and hence
it is compact. To proceed, take an arbitrary sequence
$\\{y^{k}\\}\subset[f(\Omega)]_{\bar{y}}$ converging to $y\in\mathbb{R}^{m}$
and find a sequence $\\{x^{k}\\}\subset\Omega$ such that
$f(x^{k})=y^{k}\leqq\bar{y}$ for all $k\in\mathbb{N}$. Since
$\displaystyle\lim_{k\to\infty}f(x^{k})=\displaystyle\lim_{k\to\infty}y^{k}=y$,
it follows from (i) that the sequence $\\{x^{k}\\}$ is bounded. Thus
$\\{x^{k}\\}$ has an accumulation point $x$, which belongs to $\Omega$ due to
the closedness of this set. The continuity of $f$ implies that $y=f(x)$, and
consequently we have that $y\in f(\Omega)$. Noting that $y\leqq{\bar{y}}$
gives us the inclusion $y\in[f(\Omega)]_{\bar{y}}$, which therefore completes
the proof of the theorem. $\hfill\triangle$

The results of Theorem 3.1 significantly extend the recent ones from [16],
where such an equivalence is established in the case of
$\Omega=\mathbb{R}^{n}$ and polynomial mappings $f$ by using methods of
semialgebraic geometry. The proof of [16] is based on the inclusion

$\displaystyle T_{\infty,\bar{y}}(f,\mathbb{R}^{n})$ $\displaystyle\subset$
$\displaystyle K_{\infty,\bar{y}}(f,\mathbb{R}^{n})$ (7)

valid when $f$ is polynomial. The following example shows that if $f$ is not
polynomial, then (7) fails. Thus the approach of [16] cannot be applied to our
general setting, while the new approach of variational analysis allows to
treat (VP) in full generality.

######  Example 3.1.

Let $\bar{y}=0$, and let $f\colon\mathbb{R}\to\mathbb{R}$ be defined by
$f(x):=\sin x$. We claim that $0\in
T_{\infty,\bar{y}}(f,\mathbb{R}^{n})\setminus
K_{\infty,\bar{y}}(f,\mathbb{R}^{n})$ and so
$T_{\infty,\bar{y}}(f,\mathbb{R}^{n})\nsubseteq
K_{\infty,\bar{y}}(f,\mathbb{R}^{n}).$ Indeed, let $x^{k}=2k\pi$ for all
$k\in\mathbb{N}$. It is easily seen that $\Gamma(f,\mathbb{R})=\mathbb{R}$ and
hence $\\{x^{k}\\}\subset\Gamma(f,\mathbb{R})$. Since $x^{k}\to\infty$ and
$f(x^{k})\to 0$ as $k\to\infty$, we have that $0\in
T_{\infty,\bar{y}}(f,\mathbb{R}^{n})$. To show now that $0\notin
K_{\infty,\bar{y}}(f,\mathbb{R}^{n})$, assume the contrary and then find a
sequence $\\{u^{k}\\}\subset\mathbb{R}$ such that $f(u^{k})\leqq 0$,
$u^{k}\to\infty$, $f(u^{k})\to 0$, and $u^{k}\nabla f(u^{k})\to 0$ as
$k\to\infty$. This implies that $\sin u^{k}\to 0$ and $u^{k}\cos u^{k}\to 0$
as $k\to\infty$. Using $u^{k}\to\infty$ and $u^{k}\cos u^{k}\to 0$, we get
that $\cos u^{k}\to 0$ as $k\to\infty$. Consequently,
$\sin^{2}u^{k}+\cos^{2}u^{k}\to 0$ as $k\to\infty$, which contradicts the fact
that $\sin^{2}u^{k}+\cos^{2}u^{k}=1$ for all $k\in\mathbb{N}$ and thus
verifies the failure of inclusion (7).

We conclude this section with an immediate consequence of Theorem 3.1; cf.
[12, Theorem 2.5] for the case where $f$ is polynomial and
$\Omega=\mathbb{R}^{m}$.

######  Corollary 3.1.

Assume that every section of the set $f(\Omega)$ is bounded. Then the
following assertions are equivalent:

  1. (i)

$f|_{\Omega}$ is proper.

  2. (ii)

$f|_{\Omega}$ satisfies the Palais–Smale condition:
$\widetilde{K}_{\infty}\,(f,\Omega)=\emptyset$.

  3. (iii)

$f|_{\Omega}$ satisfies the weak Palais–Smale condition:
${K}_{\infty}\,(f,\Omega)=\emptyset$.

  4. (iv)

$f|_{\Omega}$ is M-tame: $T_{\infty}\,(f,\Omega)=\emptyset$.

Furthermore, every section of the set $f(\Omega)$ is compact provided that one
of the above equivalent conditions is satisfied.

##  4\. Existence of optimal solutions

This section contains our main results on the existence of optimal solutions
to constrained vector optimization problem (VP) in the general nonsmooth
setting. We start with deriving verifiable necessary and sufficient conditions
for the existence of Pareto efficient solutions.

###  4.1. Existence of Pareto efficient solutions

Given $\bar{y}\in(\mathbb{R}\cup\\{\infty\\})^{m}$, denote

$\displaystyle{K}_{0,\leqq\bar{y}}\,(f,\Omega)$ $\displaystyle:=$
$\displaystyle\big{\\{}f(x)\in\mathbb{R}^{m}\,\big{|}\,x\in\Omega,\;f(x)\leqq\bar{y},\;\nu(x)=0\big{\\}},$

where $\nu(x)$ is the Rabier function defined in (1). The motivation behind
this definition comes from the observation that if $\bar{x}$ is a Pareto
efficient solution to problem (VP), then $\nu(\bar{x})=0$ and so
$f(\bar{x})\in{K}_{0,\leqq\bar{y}}\,(f,\Omega)$ with $\bar{y}=f(\bar{x})$.

######  Theorem 4.1.

The following assertions are equivalent:

  1. (i)

Problem (VP) admits a Pareto efficient solution.

  2. (ii)

There exists a vector $\bar{y}\in f(\Omega)$ such that the section
$[f(\Omega)]_{\bar{y}}$ is bounded and the inclusion
$\widetilde{K}_{\infty,\leqq\bar{y}}\,(f,\Omega)\subset{K}_{0,\leqq\bar{y}}\,(f,\Omega)$
holds.

  3. (iii)

There exists a vector $\bar{y}\in f(\Omega)$ such that the section
$[f(\Omega)]_{\bar{y}}$ is bounded and the inclusion
${K}_{\infty,\leqq\bar{y}}\,(f,\Omega)\subset{K}_{0,\leqq\bar{y}}\,(f,\Omega)$
holds.

  4. (iv)

There exists a vector $\bar{y}\in f(\Omega)$ such that the section
$[f(\Omega)]_{\bar{y}}$ is bounded and the inclusion
$T_{\infty,\leqq\bar{y}}\,(f,\Omega)\subset{K}_{0,\leqq\bar{y}}\,(f,\Omega)$
holds.

Proof. First we justify in parallel implications (i)$\Rightarrow$(ii),
(i)$\Rightarrow$(iii), and (i)$\Rightarrow$ iv). To this end, let
$\bar{x}\in\Omega$ be a Pareto efficient solution to (VP), and let
$\bar{y}:=f(\bar{x})$. As mentioned above, the section $[f(\Omega)]_{\bar{y}}$
is just $\\{\bar{y}\\}$ while containing in this case the sets
${K}_{0,\leqq\bar{y}}\,(f,\Omega),$
$\widetilde{K}_{\infty,\leqq\bar{y}}\,(f,\Omega),$
${K}_{\infty,\leqq\bar{y}}\,(f,\Omega),$ and
${T}_{\infty,\leqq\bar{y}}\,(f,\Omega).$ Furthermore, the necessary optimality
conditions from Lemma 2.4 ensure the existence of
$\lambda\in\mathbb{R}^{m}_{+}$ such that $\sum_{i=1}^{m}\lambda_{i}=1$ and

$\displaystyle 0$ $\displaystyle\in$
$\displaystyle\displaystyle\sum_{i=1}^{m}\lambda_{i}\partial
f_{i}(\bar{x})+N(\bar{x};\Omega).$

By definition (1) we have $\nu(\bar{x})=0$, and therefore
$\bar{y}\in{K}_{0,\leqq\bar{y}}\,(f,\Omega)$. Thus the conditions in (ii),
(iii), and (iv) follow immediately from these facts giving us necessary
conditions for the existence of Pareto efficient solutions to (VP).

Next we verify implications (ii)$\Rightarrow$(i), (iii)$\Rightarrow$(i), and
(iv)$\Rightarrow$(i), which justify the sufficiency of conditions (ii)–(iv)
for the existence of Pareto efficient solutions to (VP). Let $\bar{y}\in
f(\Omega)$ be such that the section $Y:=[f(\Omega)]_{\bar{y}}$ is bounded.
Then the closure $\overline{Y}$ of $Y$ is a nonempty compact set. Fix a vector
$\lambda\in\mathrm{int}\,\mathbb{R}_{+}^{m}$ and consider the scalar
optimization problem

$\displaystyle\min_{y\in\overline{Y}}\langle\lambda,y\rangle.$

which has an optimal solution $\widehat{y}\in\overline{Y}$.

Assume we have proved that $\Omega\cap f^{-1}(\widehat{y})\neq\emptyset$. Take
arbitrarily $\widehat{x}\in\Omega\cap f^{-1}(\widehat{y})$ and show that
$\widehat{x}$ is a Pareto efficient solution to problem (VP). Arguing by
contradiction, suppose that there exists $x\in\Omega$ such that

$\displaystyle f(x)$ $\displaystyle\leqq$ $\displaystyle
f(\widehat{x})\quad\textrm{ and }\quad f(x)\neq f(\widehat{x}).$

Componentwise it can be equivalently written as

$\displaystyle f_{i}(x)$ $\displaystyle\leq$ $\displaystyle
f_{i}(\widehat{x})\ \textrm{ for }\ i=1,\ldots,n\quad\textrm{ and }\quad
f_{j}(x)\ <\ f_{j}(\widehat{x})\ \textrm{ for some }j.$

Hence in the case of $f(x)\in Y$ we arrive at the contradiction by

$\displaystyle\langle\lambda,f(x)\rangle$ $\displaystyle<$
$\displaystyle\langle\lambda,f(\widehat{x})\rangle=\langle\lambda,\widehat{y}\rangle.$

If otherwise $f(x)\not\in Y,$ we have that $f_{i}(x)>\bar{y}_{i}$ for some
$i\in\\{1,\ldots n\\}$, and so

$\displaystyle f_{i}(\widehat{x})$ $\displaystyle=$
$\displaystyle\widehat{y}_{i}\ \leq\ \bar{y}_{i}\ <\ f_{i}(x)\ \leq\
f_{i}(\widehat{x}),$

which is also a contradiction.

It remains to show that the set $\Omega\cap f^{-1}(\widehat{y})$ is nonempty
provided that either
${K}_{\infty,\leqq\bar{y}}\,(f,\Omega)\subset{K}_{0,\leqq\bar{y}}\,(f,\Omega)$
or
${T}_{\infty,\leqq\bar{y}}\,(f,\Omega)\subset{K}_{0,\leqq\bar{y}}\,(f,\Omega),$
with taking into account that
$\widetilde{K}_{\infty,\leqq\bar{y}}\,(f,\Omega)\subset{K}_{\infty,\leqq\bar{y}}\,(f,\Omega).$
Suppose on the contrary that this claim fails. Denote

$\displaystyle X$ $\displaystyle:=$ $\displaystyle\Omega\cap f^{-1}(Y)\ =\
\big{\\{}x\in\Omega\,\big{|}\,f(x)\leqq\bar{y}\big{\\}}\ \neq\ \emptyset,$

and consider the maximum function
$\phi\colon\mathbb{R}^{n}\rightarrow\mathbb{R}$ defined by (3) with its
properties mentioned above. Furthermore, it follows from
$\widehat{y}\in\overline{Y}$ that

$\displaystyle\inf\limits_{x\in X}\phi(x)$ $\displaystyle=$ $\displaystyle 0.$

There are two cases to be considered.

#### Case 1:

${K}_{\infty,\leqq\bar{y}}\,(f,\Omega)\subset{K}_{0,\leqq\bar{y}}\,(f,\Omega)$.

By using arguments similar to those employed to establish implication
(iii)$\Rightarrow$ (i) of Theorem 3.1 we find a sequence $\\{u^{k}\\}\subset
X$ satisfying the limiting relationships

$\displaystyle\|u^{k}\|\to\infty,\quad\phi(u^{k})\to 0,\quad\textrm{ and
}\quad\|u^{k}\|\nu(u^{k})\to 0\quad\textrm{ as }\quad k\to\infty.$

In particular, $f(u^{k})\to\widehat{y}$ as $k\to\infty$, which yields
$\widehat{y}\in{K}_{\infty,\leqq\bar{y}}\,(f,\Omega).$ This implies, by taking
into account the imposed assumption, that
$\widehat{y}\in{K}_{0,\leqq\bar{y}}\,(f,\Omega)$. Thus we arrive at
$\widehat{y}=f(\widehat{x})$ for some $\widehat{x}\in\Omega,$ a contradiction.

#### Case 2:

${T}_{\infty,\leqq\bar{y}}\,(f,\Omega)\subset{K}_{0,\leqq\bar{y}}\,(f,\Omega)$.

Invoking arguments similar to those used to prove implication
(iv)$\Rightarrow$(i) of Theorem 3.1, we find a sequence $\\{v^{k}\\}\subset
X\cap\Gamma(f,\Omega)$ satisfying the relationships

$\displaystyle\|v^{k}\|\to\infty\quad\textrm{ and }\quad\phi(v^{k})\to
0\quad\textrm{ as }\quad k\to\infty.$

In particular, we get $f(v^{k})\to\widehat{y}$ as $k\to\infty.$ This gives us
by definition that $\widehat{y}\in{T}_{\infty,\leqq\bar{y}}\,(f,\Omega)$,
which yields together with the assumption made that
$\widehat{y}\in{K}_{0,\leqq\bar{y}}\,(f,\Omega)$. Therefore
$\widehat{y}=f(\widehat{x})$ for some $\widehat{x}\in\Omega,$ a contradiction,
which completes the proof of the theorem. $\hfill\triangle$

In this way we arrive at the verifiable necessary and sufficient conditions
for the existence of Pareto efficient solutions in constrained vector
optimization with nonsmooth data.

######  Corollary 4.1.

Assume that there is $\bar{y}\in f(\Omega)$ such that the section
$[f(\Omega)]_{\bar{y}}$ is bounded. Then the problem (VP) admits a Pareto
solution provided that one of the following equivalent conditions holds:

  1. (i)

$f|_{\Omega}$ is proper at the sublevel $\bar{y}$.

  2. (ii)

$f|_{\Omega}$ satisfies the Palais–Smale condition at the sublevel $\bar{y}$:
$\widetilde{K}_{\infty,\leqq\bar{y}}\,(f,\Omega)=\emptyset$.

  3. (iii)

$f|_{\Omega}$ satisfies the weak Palais–Smale condition at the sublevel
$\bar{y}$: ${K}_{\infty,\leqq\bar{y}}\,(f,\Omega)=\emptyset$.

  4. (iv)

$f|_{\Omega}$ is M-tame at the sublevel $\bar{y}$:
$T_{\infty,\leqq\bar{y}}\,(f,\Omega)=\emptyset$.

Proof. This is a direct consequence of Theorems 3.1 and 4.1. $\hfill\triangle$

###  4.2. Existence of Geoffrion-properly efficient solutions

The last part of this paper is devoted to the existence of Geoffrion-properly
efficient solutions to problem (VP). First we show that the equivalent
conditions of Corollary 4.1 do not guarantee the existence of a Geoffrion-
properly solution to this problem.

######  Example 4.1.

Let $\Omega:=\\{x\in\mathbb{R}\,|\,-x\leqq 0\\}$, and let
$f\colon\mathbb{R}\to\mathbb{R}^{2}$ be a polynomial mapping defined by
$f(x)=(f_{1}(x),f_{2}(x)):=(-x^{2},x)$. It is easy to see that every section
of $f(\Omega)$ is bounded and that $f|_{\Omega}$ is proper. Corollary 4.1
tells us that problem (VP) admits a Pareto efficient solution. In fact, it is
not hard to check that the whole set $\Omega$ consists of Pareto efficient
solutions to (VP).

We claim that the problem (VP) has no Geoffrion-properly efficient solutions.
Indeed, let $\bar{x}$ be an arbitrary element of $\Omega$. We have to show
that for all $M>0$ there exists an index $i\in\\{1,2\\}$ and some $x\in\Omega$
with $f_{i}(x)<f_{i}(\bar{x})$ such that

$\displaystyle\frac{f_{i}(\bar{x})-f_{i}(x)}{f_{j}(x)-f_{j}(\bar{x})}$
$\displaystyle>$ $\displaystyle M$

whenever $j\in\\{1,2\\}$ with $f_{j}(\bar{x})<f_{j}(x)$. To proceed, pick
$x>\max\\{M,\bar{x}\\}$, $i=1$, and $j=2$. Then we have
$f_{1}(x)<f_{1}(\bar{x}),$ $f_{2}(\bar{x})<f_{2}(x),$ and

$\displaystyle\frac{f_{i}(\bar{x})-f_{i}(x)}{f_{j}(x)-f_{j}(\bar{x})}$
$\displaystyle=$ $\displaystyle x+\bar{x}\ >\ M,$

which verifies the claim.

As shown in [13, Theorem 5.2], a necessary condition for the existence of
Geoffrion-properly efficient solutions to (VP) is

$\displaystyle[f(\Omega)+\mathbb{R}^{m}_{+}]^{\oplus}\cap(-\mathbb{R}^{m}_{+})$
$\displaystyle=$ $\displaystyle\\{0\\}.$

By using [21, Lemma 3.2.4], it can be equivalently rewritten as

$\displaystyle[f(\Omega)]^{\oplus}\cap(-\mathbb{R}^{m}_{+})$ $\displaystyle=$
$\displaystyle\\{0\\},$ (8)

where, given $Y\subset\mathbb{R}^{m}$, the symbol $[Y]^{\oplus}$ stands for
the recession cone of $Y$ defined by

$\displaystyle[Y]^{\oplus}:=\big{\\{}d\in\mathbb{R}^{m}\,\big{|}\,\exists\,\\{(t_{k},y^{k})\\}\subset\mathbb{R}_{+}\times
Y\;\text{ such that }\;t_{k}\to 0\;\textrm{ and }\;t_{k}y^{k}\to d\big{\\}}.$

The next result provides sufficient conditions for the existence of Geoffrion-
properly efficient solutions to the constrained vector optimization problem
(VP).

######  Theorem 4.2.

Under the validity of the necessary optimality condition (8), the following
equivalent conditions are sufficient for the existence of a Geoffrion-properly
efficient solution to the vector optimization problem (VP):

  1. (i)

$f|_{\Omega}$ is proper;

  2. (ii)

$f|_{\Omega}$ satisfies the Palais–Smale condition;

  3. (iii)

$f|_{\Omega}$ satisfies the weak Palais–Smale condition;

  4. (iv)

$f|_{\Omega}$ is M-tame.

Proof. Let us first check that conditions (i)–(iv) are indeed equivalent in
the setting under consideration. By using Corollary 3.1, it suffices to show
that every section of $f(\Omega)$ is bounded. Supposing the contrary gives us
a point $y\in\mathbb{R}^{m}$ for which the section $[f(\Omega)]_{y}$ is
unbounded. Then we find a sequence $\\{x^{k}\\}\subset\Omega$ such that
$f(x^{k})\leqq y$ whenever $k\in\mathbb{N}$ and $\|f(x^{k})\|\to\infty$ as
$k\to\infty$. Denoting

$\displaystyle t_{k}=\frac{1}{\|f(x^{k})\|}\quad\textrm{ and }\quad
d^{k}:=t_{k}f(x^{k}),\quad k\in\mathbb{N},$

we have $t^{k}\to 0$ as $k\to\infty$ and $\|d^{k}\|=1$ for all
$k\in\mathbb{N}$. Without loss of generality, assume that the sequence
$\\{d^{k}\\}$ converges to some $d\in\mathbb{R}^{m}$ with $\|d\|=1$. It
follows from the definition that $d\in[f(\Omega)]^{\oplus}$. Since
$f(x^{k})\leqq y$ for all $k\in\mathbb{N}$, we arrive at
$d\in-\mathbb{R}^{m}_{+}$, a contradiction.

Now we are ready to prove that the set of Geoffrion-properly efficient
solutions to (VP) is nonempty. Invoking [3, Theorem 3.2] and [13, Theorems 2.1
and 5.1], it suffices to show that the set $f(\Omega)+\mathbb{R}^{m}_{+}$ is
closed. To proceed, we deduce from Corollary 3.1 that every section of
$f(\Omega)$ is compact. Pick an arbitrary sequence $\\{a^{k}\\}\subset
f(\Omega)+\mathbb{R}^{m}_{+}$ that converges to some $a\in\mathbb{R}^{m}$ and
find sequences $\\{y^{k}\\}\subset f(\Omega)$ and
$\\{d^{k}\\}\subset\mathbb{R}^{m}_{+}$ such that $a^{k}=y^{k}+d^{k}$ for all
$k\in\mathbb{N}$. Since the sequence $\\{a^{k}\\}$ is convergent, there is
$\bar{a}\in\mathbb{R}^{m}$ with $a^{k}\leqq\bar{a}$ for all $k\in\mathbb{N}$.
It clearly follows that $y^{k}\leqq\bar{a}$ whenever $k\in\mathbb{N}$, and
thus $\\{y^{k}\\}\subset[f(\Omega)]_{\bar{a}}$. The compactness of
$[f(\Omega)]_{\bar{a}}$ gives us a subsequence of $\\{y^{k}\\}$, which
converges to some $y\in[f(\Omega)]_{\bar{a}}$. This implies that $\\{d^{k}\\}$
is also convergent to some $d\in\mathbb{R}^{m}_{+}$, and therefore

$\displaystyle a$ $\displaystyle=$ $\displaystyle
y+d\in[f(\Omega)]_{\bar{a}}+\mathbb{R}^{m}_{+}\subset
f(\Omega)+\mathbb{R}^{m}_{+},$

which completes the proof of the theorem.$\hfill\triangle$

We end this section with the following remarks clarifying relationships of the
obtained results with the existence other types of properly efficient
solutions to (VP).

######  Remark 4.1.

(i) It has been realized in vector optimization (see, e.g., [14]) that in the
setting under consideration the concept of Geoffrion-properly efficient
solutions agrees with the notions of properly efficient solutions in the
senses of Benson [3] and Henig [13], and that every Geoffrion-properly
efficient solution is also properly efficient in the sense of Borwein [4].
Thus Theorem 4.2 also provides sufficient conditions for the existence of
properly efficient solutions in the senses of Benson, Henig, and Borwein.

(ii) It follows from [11, Theorem 5.1] that problems (VP) admits a Henig-
properly efficient solution if $\Omega=\mathbb{R}^{n}$ and the objective
mapping $f\colon\mathbb{R}^{n}\to\mathbb{R}^{m}$ is bounded from below and
satisfies the Palais–Smale condition. Recall that $f$ is bounded from below if
there exists a vector $a\in\mathbb{R}^{m}$ such that

$\displaystyle f(\mathbb{R}^{n})$ $\displaystyle\subset$ $\displaystyle
a+\mathbb{R}^{m}_{+}.$

The above definition readily implies that

$\displaystyle\big{[}f(\mathbb{R}^{n})\big{]}^{\oplus}$ $\displaystyle\subset$
$\displaystyle\big{[}a+\mathbb{R}^{m}_{+}\big{]}^{\oplus}\ =\
\mathbb{R}^{m}_{+},$

and therefore we get
$[f(\mathbb{R}^{n})]^{\oplus}\cap(-\mathbb{R}^{m}_{+})=\\{0\\}$. The converse
is true when $m=1$ but fails in general. Thus our Theorem 4.2 essentially
improves and extends the result of [11, Theorem 5.1]. To illustrate that we
get a proper improvement even for unconstrained problems on $\mathbb{R}$ with
polynomial objectives, consider problem (VP) with $\Omega:=\mathbb{R}$ and the
cost mapping $f\colon\mathbb{R}\to\mathbb{R}^{2}$ defined by
$f(x):=(x,x^{2})$. It is easy to check that

$\displaystyle\big{[}f(\mathbb{R})\big{]}^{\oplus}\cap\big{(}-\mathbb{R}^{2}_{+}\big{)}=\\{0\\}$

and that $f$ satisfies the Palais–Smale condition. Theorem 4.2 tells us that
this problem admits a Geoffrion-properly efficient solution. However, the
mapping $f$ under consideration is not bounded from below on $\mathbb{R}$, and
hence the result of [11] is not applicable in this case.

##  5\. Conclusions

This paper demonstrates that developing a novel approach of variational
analysis and generalized differentiation to the existence of global optimal
solutions to constrained problems of vector optimization allows us to derive
truly new results in this area in both smooth and nonsmooth settings. In this
way we show that the developed variational approach leads us to verifiable
necessary optimality conditions for the existence of Pareto efficient
solutions as well as sufficient conditions for the existence of properly
efficient solutions to general constrained problems with locally Lipschitzian
cost mappings. In particular, the obtained results dramatically improves the
very recent existence theorems of Pareto efficient solutions established in
[16] for unconstrained problems with polynomial cost mappings by using
techniques of semialgebraic geometry and polynomial optimization.

We see the following natural directions of future developments of the
variational approach to the existence theorems in problems of multiobjective
optimization.

1\. Avoiding the Lipschitz continuity assumption on cost mappings by
considering vector optimization problems with merely continuous and also order
semicontinuous cost mappings that frequently arise in applications. According
to the scheme implemented above, this requires further investigations of
fundamental issues of generalized differential calculus and necessary
optimality conditions dealing with non-Lipschitzian mappings and the like.

2\. Studying optimization and equilibrium problems with set-valued cost
mappings, which are at the core of most recent developments in multiobjective
optimization and practical applications to various models in economics,
finance, behavioral sciences, etc.; see, e,g., the monographs [15, 19] and the
references therein.

3\. Considering vector and set-valued optimization problems in infinite-
dimensional spaces. This would open the gate to cover, in particular, various
dynamical equilibrium models arising in macroeconomic, mechanics, and systems
control governed by constrained evolution equations, inclusions, variational
conditions, etc; see, e.g., [18, 19, 22]. Variation principles and appropriate
tools of generalized differentiation provide powerful machinery to
successfully proceed in the theoretical developments in this direction with
subsequent applications.

### Acknowledgments

The last version of this paper was done while the third author and the fourth
author were visiting Department of Applied Mathematics, Pukyong National
University, Busan, Republic of Korea in April 2018. These authors would like
to thank the department for hospitality and support during their stay.

## References

  * [1] T. Q. Bao and B. S. Mordukhovich, Variational principles for set-valued mappings with applications to multiobjective optimization, Control Cybern., 36 (2007), 531–562.
  * [2] T. Q. Bao and B. S. Mordukhovich, Relative Pareto minimizers for multiobjective problems: existence and optimality conditions, Math. Program., 122 (2010), 101–138.
  * [3] H. P. Benson, An improved definition of proper efficiency for vector maximization with respect to cones, J. Math. Anal. Appl., 71 (1979), 232–241.
  * [4] J. Borwein, Proper efficient points for maximizations with respect to cones, SIAM J. Control Optim., 15 (1977), 57–63.
  * [5] J. M. Borwein, On the existence of Pareto efficient points, Math. Oper. Res., 8 (1983), 64–73.
  * [6] I. Ekeland, Nonconvex minimization problems, Bull. Amer. Math. Soc., 1 (1979), 443–474.
  * [7] F. Flores-Bazán, Ideal, weakly efficient solutions for vector optimzation problems, Math. Program., 93 (2002), 453–475.
  * [8] A. M. Geoffrion, Proper efficiency and the theory of vector maximization, J. Math. Anal. Appl., 22 (1968), 618–630.
  * [9] C. Gutiérrez, R. López and V. Novo, Existence and boundedness of solutions in infinite-dimensional vector optimization problems, J. Optim. Theory Appl., 162 (2014), 515–547.
  * [10] T. X. D. Ha, Variants of the Ekeland variational principle for a set-valued map involving the Clarke normal cone, J. Math. Anal. Appl., 316 (2006), 346–356.
  * [11] T. X. D. Ha, The Ekeland variational principle for Henig proper minimizers and super minimizers, J. Math. Anal. Appl., 364 (2010), 156–170.
  * [12] H. V. Hà and T. S. Phạm, Genericity in Polynomial Optimization, World Scientific Publishing, Singapore, 2017.
  * [13] M. I. Henig, Proper efficiency with respect to cones, J. Optim. Theory Appl., 36 (1982), 387–407.
  * [14] J. Jahn, Vector Optimization: Theory, Applications, and Extensions, Springer, Berlin, 2004.
  * [15] A. A. Khan, C. Tammer and C. Zălinescu, Set-Valued Optimization. An Introduction with Applications, Springer, Berlin, 2015.
  * [16] D. S. Kim, T. S. Phạm and N. V. Tuyen, On the existence of Pareto solutions for polynomial vector optimization problems, to appear in Math. Program. (2018); https://doi.org/10.1007/s10107-018-1271-7.
  * [17] D. T. Luc, Theory of Vector Optimization, Lecture Notes in Econom. Math. Systems, Vol. 319, Springer, Berlin, 1989.
  * [18] B. S. Mordukhovich, Variational Analysis and Generalized Differentiation, I: Basic Theory; II: Applications, Springer, Berlin, 2006.
  * [19] B. S. Mordukhovich, Variational Analysis and Applications, Springer, New York, 2018.
  * [20] R. T. Rockafellar and R. J-B. Wets, Variational Analysis, Springer, Berlin, 1998.
  * [21] Y. Sawaragi, H. Nakayama and T. Tanino, Theory of Multiobjective Optimization, Academic Press, Orlando, FL, 1985.
  * [22] Y. B. Xiao, N. J. Huang and Y. J. Cho, A class of generalized evolution variational inequalities in Banach spaces, Appl. Math. Lett., 25 (2012), 914–920.