On approximate quasi Pareto solutions in nonsmooth semi-infinite interval-valued vector optimization problems

Nguyen Huy Hunga, Hoang Ngoc Tuana, and Nguyen Van TuyenaCONTACT Nguyen Van Tuyen. Email<EMAIL_ADDRESS><EMAIL_ADDRESS>aDepartment of Mathematics, Hanoi Pedagogical University 2, Xuan Hoa, Phuc Yen, Vinh Phuc, Vietnam

This paper deals with approximate solutions of a nonsmooth semi-infinite programming with multiple interval-valued objective functions.  We first introduce four types of approximate quasi Pareto solutions of the considered problem by considering the lower-upper interval order relation and then apply some advanced tools of variational analysis and generalized differentiation to establish necessary optimality conditions for these approximate solutions. Sufficient conditions for approximate quasi Pareto solutions of such a problem are also provided by means of introducing the concepts of approximate (strictly) pseudo-quasi generalized convex functions defined in terms of the limiting subdifferential of locally Lipschitz functions. Finally, a Mond–Weir type dual model in approximate form is formulated, and weak, strong and converse-like duality relations are proposed.

KKT optimality conditions; Duality relations; Limiting/Mordukhovich
subdifferential; Approximate quasi Pareto solutions; Nonsmooth semi-infinite interval-valued vector optimization

90C29; 90C46; 90C70; 90C34;  49J52

§ INTRODUCTION

In this paper, we are interested in approximate solutions of the following semi-infinite programming with multiple interval-valued objective functions:

\begin{align}\label{problem}\tag{SIVP}
LU-&\mathrm{Min}\, f(x):=(f_1(x), \ldots, f_m(x))
\\
&\text{s. t.}\ \ x\in \mathcal{F}:=\{x\in \Omega\,:\, g_t(x)\leq 0, t\in T\}, \notag
\end{align}

where $f_i\colon \mathbb{R}^n\to \mathcal{K}_c$, $i\in I:=\{1, \ldots, m\}$, are interval-valued functions defined by $f_i(x)=[f_i^L(x), f_i^U(x)]$, $f_i^L,$ $f_i^U\colon\mathbb{R}^n\to \mathbb{R}$  are locally Lipschitz functions satisfying $f_i^L(x)\leq f_i^U(x)$ for all $x\in\mathbb{R}^n$ and $i\in I$, $\mathcal{K}_c$ is the class of all closed and bounded intervals in $\mathbb{R}$, i.e.,

$$\mathcal{K}_c=\{[a^L, a^U]\,:\, a^L, a^U\in\mathbb{R}, a^L\leq a^U\},$$

$g_t\colon\mathbb{R}^n\to \mathbb{R}$, $t\in T$, are locally Lipschitz functions, $T$ is an arbitrary  set (possibly infinite), and $\Omega$ is a nonempty and closed subset of $\mathbb{R}^n$. Set $g_T:=(g_t)_{t\in T}$.

An interval-valued optimization problem is one of the deterministic optimization models to deal with the uncertain (incomplete) data. In the literature, there are three main approaches to model constrained optimization with uncertainty, say stochastic programming approach, fuzzy  programming approach, and interval-valued programming approach; see, e.g., [2, 3, 1, 5, 6, 4]. Many methodologies have been developed to solve these problems. However, it should be noted here that the usual way is to transform stochastic and fuzzy optimization problems into the conventional optimization problems; frequently, these problems are very complicated. Consequently, stochastic and fuzzy optimization problems are not easy to be solved. In interval-valued optimization, the coefficients of objective and constraint functions are taken as closed intervals. Hence, the interval-valued optimization problem will be easier to be solved than a stochastic or fuzzy optimization one. That is the main reason why the interval-valued optimization problems have recently received increasing interest in optimization community; see, e.g., [7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 22, 19, 20, 21, 6] and the references therein.

In [8], Ishibuchi and Tanaka  introduced the lower-upper (LU) interval order relation and reformulated optimization problems with interval-valued objective functions as vector optimization problems using the order relation.  Thereafter, many authors have  investigated optimality conditions  of Karush–Kuhn-Tucker-type (KKT) and duality for optimization problems with one or multiple interval-valued objective functions  and finitely many  constraints; see, e.g., [7, 11, 13, 14, 15, 16, 22, 19, 20, 21, 6].

The semi-infinite optimization problems play a very important role in optimization theory and their models cover optimal control, approximation theory,  semi-definite programming and numerous engineering problems, etc. However, in contrast with the case of optimization problems with finitely many  constraints, there have only been several papers dealing with KKT optimality conditions and duality for semi-infinite interval-valued optimization problems. For papers of this topic, we refer the reader to [9, 10, 12, 17, 18] and references given therein. In [10], Kumar et al. established optimality conditions and duality theorems for  interval-valued programming problems with infinitely many constraints. Then, in [9, 12, 17, 18], the authors presented optimality conditions and duality theorems for Pareto optimal solutions with respect to $LU$ interval order relation of a semi-infinite interval-valued vector optimization problem   under the convexity of the objective functions and constraints. To the best of our
knowledge, so far there have been no papers investigating optimality conditions and duality relations for approximate Pareto solutions of such a problem.  It should be noted here that the study of approximate solutions is very important in optimization  because, from the computational point of view, numerical algorithms usually generate only approximate solutions if we stop them after a finite number of steps. Furthermore, the solution set may be empty in the general noncompact case, whereas approximate solutions exist under very weak assumptions; see e.g.,  [30, 25, 22, 29, 27, 31, 28, 26, 23, 24].

Motivated by the above observations, in this paper, we introduce four kinds of approximate quasi Pareto solutions with respect to $LU$ interval order relation for problems of the form (<ref>). Then we employ the Mordukhovich/limiting subdifferential and the Mordukhovich/limiting normal cone (cf. [32]) to examine  KKT optimality conditions and duality relations for these approximate solutions of problem (<ref>).

The paper is organized as follows. Section <ref> provides some basic definitions from variational analysis, interval analysis and several auxiliary results. In Section <ref>, we introduce four kinds of approximate quasi Pareto solutions of problem (<ref>) and establish KKT-type necessary conditions for these approximate solutions.  With the help of approximate generalized convex functions defined in terms of the Mordukhovich/limiting subdifferential and the Mordukhovich normal cone, we provide sufficient conditions for approximate quasi Pareto solutions of the considered problem. Section <ref> is devoted to presenting duality relations for approximate quasi Pareto solutions. The conclusions
are presented in  the final section.

§  PRELIMINARIES

We use the following notation and terminology. Fix $n \in {\mathbb{N}}:=\{1, 2, \ldots\}$. The space $\mathbb{R}^n$ is equipped with the usual scalar product and  Euclidean norm. The closed unit ball of $\mathbb{R}^n$ is denoted by $\mathbb{B}_{\mathbb{R}^n}$.  We denote the nonnegative orthant in $\mathbb{R}^n$ by  $\mathbb{R}^n_+$.  The topological closure is denoted  by  $\mathrm{cl}\,{S}$.

Given $\bar x\in  \mbox{cl}\,S$. The set

\begin{equation*}
N(\bar x; S):=\{z^*\in \mathbb{R}^n:\exists
x^k\stackrel{S}\longrightarrow \bar x, \varepsilon_k\to 0^+, z^*_k\to z^*,
z^*_k\in {\widehat N_{\varepsilon
_k}}(x^k; S),\ \ \forall k \in\mathbb{N}\},
\end{equation*}

is called the Mordukhovich/limiting normal cone  of $S$ at $\bar x$, where

\begin{equation*}
\widehat N_\varepsilon  (x; S):= \bigg\{ {z^*  \in {\mathbb{R}^n} \;:\;\limsup_{u\overset{S} \rightarrow x}
\frac{{\langle z^* , u - x\rangle }}{{\parallel u - x\parallel }} \leq \varepsilon } \bigg\}
\end{equation*}

is the set of  $\varepsilon$-normals of $S$  at $x$ and  $u\xrightarrow {{S}} x$ means that $u \rightarrow x$ and $u \in S$.

Let  $\varphi \colon \mathbb{R}^n \to  \overline{\mathbb{R}}$ be an extended-real-valued function. The epigraph  and  domain of $\varphi$ are denoted, respectively, by

\begin{align*}
\mbox{epi }\varphi&:=\{(x, \alpha)\in\mathbb{R}^n\times\mathbb{R} \,:\,  \alpha\geq \varphi(x) \},
\\
\mbox{dom }\varphi &:= \{x\in \mathbb{R}^n \,:\, \ \ |\varphi(x)|<+\infty \}.
\end{align*}

Let $\bar x\in \mbox{dom }\varphi$. The set

\begin{align*}
\partial \varphi (\bar x):=\{x^*\in \mathbb{R}^n \,:\, (x^*, -1)\in N((\bar x, \varphi (\bar x)); \mbox{epi }\varphi )\},
\end{align*}

is called the Mordukhovich/limiting subdifferential  of $\varphi$ at $\bar x$. If $\bar x\notin \mbox{dom }\varphi$, then we put $\partial \varphi (\bar x)=\emptyset$.

We now summarize some properties of  the Mordukhovich subdifferential that will be used in the next section.
Let $\varphi_l\colon\mathbb{R}^n\to\overline{\mathbb{R}}$, $l=1, \ldots, p$, $p\geq 2$, be lower semicontinuous around $\bar x$ and let all but one of these
functions be locally Lipschitz around $\bar x$. Then we have the following inclusion

\begin{equation*}
\partial (\varphi_1+\ldots+\varphi_p) (\bar x)\subset \partial  \varphi_1 (\bar x) +\ldots+\partial \varphi_p (\bar x).
\end{equation*}

Let $\varphi_l\colon\mathbb{R}^n\to\overline{\mathbb{R}}$, $l=1, \ldots, p$,  be  locally Lipschitz around $\bar x$. Then the function
$ \phi(\cdot):=\max\{\varphi_l(\cdot):l=1, \ldots, p\}$
is also locally Lipschitz around $\bar x$ and one has

\begin{equation*}
\partial \phi(\bar x)\subset \bigcup\bigg\{\partial\bigg(\sum_{l=1}^{p}\lambda_l\varphi_l\bigg)(\bar x)\;:\; (\lambda_1, \ldots, \lambda_p)\in\Lambda(\bar x)\bigg\},
\end{equation*}

where $\Lambda(\bar x):=\big\{(\lambda_1, \ldots, \lambda_p)\;:\; \lambda_l\geq 0, \sum_{l=1}^{p}\lambda_l=1, \lambda_l[\varphi_l(\bar x)-\phi(\bar x)]=0\big\}.$

Let   $\varphi\colon\mathbb{R}^n\to\overline{\mathbb{R}}$  be finite at $\bar x$. If $\varphi$ has a local minimum at $\bar x$, then $ 0\in\partial\varphi(\bar x).$

Next we recall some definitions and facts in interval analysis. Let $A=[a^L, a^U]$ and $B=[b^L, b^U]$ be two intervals in $\mathcal{K}_c$. Then, by definition, we have

* $A+B:=\{a+b\,:\, a\in A, b\in B\}=[a^L+b^L, a^U+b^U]$;

* $A-B:=\{a-b\,:\, a\in A, b\in B\}=[a^L-b^U, a^U-b^L]$;

* For each $k\in\mathbb{R}$,

\begin{equation*}kA:=\{ka\,:\, a\in A\}=
\begin{cases}
[ka^L, ka^U]\ \ \text{if}\ \ k\geq 0,
\\
[ka^U, ka^L]\ \ \text{if}\ \ k < 0,
\end{cases}
\end{equation*}

see, e.g., [33, 34, 35]  for more details.
It should be note that if $a^L=a^U$, then $A=[a, a]=a$ is a real number.

Let $A=[a^L, a^U]$ and $B=[b^L, b^U]$ be two intervals in $\mathcal{K}_c$. We say that:

* $A\leq_{LU} B$ if $a^L\leq b^L$ and $a^U\leq b^U$.

* $A<_{LU} B$ if $A\leq_{LU} B$ and $A\neq B$, or, equivalently, $A<_{LU} B$ if

\begin{cases}
\\
a^U\leq b^U,
\end{cases}
or    			$
\begin{cases}
a^L\leq b^L
\\
a^U< b^U,
\end{cases}
or    			$
\begin{cases}
a^L< b^L
\\
a^U< b^U.
\end{cases}

* $A<^s_{LU} B$ if $a^L<b^L$ and $a^U<b^U$.

We also define $A\leq_{LU} B$ (resp., $A<_{LU} B$, $A<^s_{LU} B$) if and only if 	$B\geq_{LU} A$ (resp., $B>_{LU} A$, $B>^s_{LU} A$).

§ APPROXIMATE OPTIMALITY CONDITIONS

Let $\mathbb{R}_+^{|T|}$ denote the set of all functions $\mu\colon T\to\mathbb{R}_+$ taking values $\mu_t:=\mu(t)=0$ for all $t\in T$ except for finitely many points. The active constraint multipliers set at $\bar x\in \Omega$ is defined by

\begin{equation*}
A(\bar x):=\left\{\mu\in \mathbb{R}_+^{|T|}\;:\; \mu_tg_t(\bar x)=0, \ \ \forall t\in T\right\}.
\end{equation*}

For each $\mu\in A(\bar x)$, put $T(\mu):=\{t\in T\;:\; \mu_t\neq 0\}$.

We now introduce approximate solutions of (<ref>) with respect to $LU$ interval order relation.  Let $\epsilon_i^L$,  $\epsilon_i^U$, $i\in I$, be   real numbers satisfying $0\leq \epsilon_i^L\leq \epsilon_i^U$ for all $i\in I$  and put   $\mathcal{E}:=(\mathcal{E}_1, \ldots, \mathcal{E}_m)$, where $\mathcal{E}_i:=[\epsilon_i^L, \epsilon_i^U]$.
Let $\bar x\in \mathcal{F}$. We say that:

* $\bar x$ is a type-1 $\mathcal{E}$-quasi Pareto solution of (<ref>), denoted by $\bar x\in \mathcal{E}$-$\mathcal{S}_1^q\eqref{problem}$, if there is no $x\in \mathcal{F}$ such that

\begin{equation*}
\begin{cases}
f_i(x)\leq_{LU} f_i(\bar x)-\mathcal{E}_i\|x-\bar x\|,\ \ &\forall i\in I,
\\
f_k(x)<_{LU} f_k(\bar x)-\mathcal{E}_k\|x-\bar x\|,\ \ &\text{for at least one}\ \ k\in I.
\end{cases}
\end{equation*}

* $\bar x$ is a type-2 $\mathcal{E}$-quasi Pareto solution of (<ref>), denoted by $\bar x\in \mathcal{E}$-$\mathcal{S}_2^q\eqref{problem}$, if there is no $x\in \mathcal{F}$ such that

\begin{equation*}
\begin{cases}
f_i(x)\leq_{LU} f_i(\bar x)-\mathcal{E}_i\|x-\bar x\|,\ \ &\forall i\in I,
\\
f_k(x)<^s_{LU} f_k(\bar x)-\mathcal{E}_k\|x-\bar x\|,\ \ &\text{for at least one}\ \ k\in I.
\end{cases}
\end{equation*}

* $\bar x$ is a type-1 $\mathcal{E}$-quasi-weakly Pareto solution of (<ref>), denoted by $\bar x\in \mathcal{E}$-$\mathcal{S}_1^{qw}\eqref{problem}$, if there is no $x\in \mathcal{F}$ such that

\begin{equation*}
f_i(x)<_{LU} f_i(\bar x)-\mathcal{E}_i\|x-\bar x\|,\ \ \forall i\in I.
\end{equation*}

* $\bar x$ is a type-2 $\mathcal{E}$-quasi-weakly Pareto solution of (<ref>), denoted by $\bar x\in \mathcal{E}$-$\mathcal{S}_2^{qw}\eqref{problem}$, if there is no $x\in \mathcal{F}$ such that

\begin{equation*}
f_i(x)<^s_{LU} f_i(\bar x)-\mathcal{E}_i\|x-\bar x\|,\ \ \forall i\in I.
\end{equation*}

It should be note that, if $\mathcal{E}=0$, i.e., $\epsilon^L_i=\epsilon^U_i=0$, $i\in I$, then the notion of a type-1 $\mathcal{E}$-quasi Pareto solution (resp.,  a type-2 $\mathcal{E}$-quasi Pareto solution,  a type-1 $\mathcal{E}$-quasi-weakly Pareto solution,  a type-2 $\mathcal{E}$-quasi-weakly Pareto solution)  defined above coincides with the one of a type-1 Pareto solution (resp., a type-2 Pareto solution, a type-1  weakly Pareto solution, a type-2 weakly Pareto solution); see, e.g., [18, 11, 20].

The following relations are immediate from the definition of approximate optimal solutions.

* $\mathcal{E}$-$\mathcal{S}_1^q\eqref{problem} \subset \mathcal{E}$-$\mathcal{S}_2^q\eqref{problem} \subset \mathcal{E}$-$\mathcal{S}_2^{qw}\eqref{problem}$.

* $\mathcal{E}$-$\mathcal{S}_1^q\eqref{problem} \subset \mathcal{E}$-$\mathcal{S}_1^{qw}\eqref{problem} \subset \mathcal{E}$-$\mathcal{S}_2^{qw}\eqref{problem}$.

Next, we demonstrate that the  concepts of approximate optimal solutions in Definition <ref> are different. Let us consider the following   examples.

To obtain the necessary optimality conditions of KKT-type for approximate quasi Pareto solutions of (<ref>), we consider the following constraint qualification condition.
Let $\bar x\in\mathcal{F}$. We say that $\bar x$ satisfies the limiting constraint qualification if the following condition holds

\begin{equation}\label{LCQ}\tag{LCQ}
N(\bar x; \mathcal{F})\subseteq \bigcup_{\mu\in A(\bar x)}\left[\sum_{t\in T}\mu_t\partial g_t(\bar x)\right]+N(\bar x; \Omega).
\end{equation}

It is worth to mention that the constraint qualification (<ref>) has been widely used in the literature and it covers almost the existing constraint qualifications of the Mangasarian–Fromovitz and the Farkas–Minkowski types; see e.g., [32, 36, 37, 27, 38].

Let $\bar x\in\mathcal{F}$ and assume that $\bar x$ satisfies the (<ref>). If $\bar x\in \mathcal{E}$-$\mathcal{S}_2^{qw}\eqref{problem}$, then there exist $\lambda^L, \lambda^U\in \mathbb{R}^m_+$ with $\sum_{i\in I}(\lambda^L_i+\lambda^U_i)=1$, and $\mu\in A(\bar x)$ such that

\begin{equation}\label{Necessary-condition}
0\in \sum_{i\in I} \left[\lambda^L_i\partial f^L_i(\bar x)+\lambda^U_i\partial f_i^U(\bar x)\right] +\sum_{t\in T}\mu_t\partial g_t(\bar x)+\sum_{i\in I}\left(\lambda^L_i\epsilon^U_i+\lambda^U_i\epsilon^L_i\right)\mathbb{B}_{\mathbb{R}^n}+N(\bar x; \Omega).
\end{equation}

Since $\bar x\in \mathcal{E}$-$\mathcal{S}_2^{qw}\eqref{problem}$, there is no $x\in\mathcal{F}$ such that $f_i(x)<^s_{LU} f_i(\bar x)-\mathcal{E}_i$, $\forall i\in I$, or, equivalently,

\begin{equation*}
f_i^L(x)< f_i^L(\bar x)-\epsilon_i^U\|x-\bar x\| \ \ \text{and}\ \ f_i^U(x)< f_i^U(\bar x)-\epsilon_i^L\|x-\bar x\|, \ \ \forall i\in I.
\end{equation*}

This means that for each $x\in\mathcal{F}$, there exists $i\in I$ such that

\begin{equation}\label{equ-1}
f_i^L(x)\geq f_i^L(\bar x)-\epsilon_i^U\|x-\bar x\| \ \ \text{or}\ \ f_i^U(x)\geq f_i^U(\bar x)-\epsilon_i^L\|x-\bar x\|.
\end{equation}

For each $x\in \mathbb{R}^n$, put

$$\varphi(x):=\max_{i\in I}\left\{f_i^L(x)- f_i^L(\bar x)+\epsilon_i^U\|x-\bar x\|, f_i^U(x)-f_i^U(\bar x)+\epsilon_i^L\|x-\bar x\|\right\}.$$

It follows from (<ref>) that $\varphi(x)\geq 0$ for all $x\in\mathcal{F}$. Clearly, $\varphi(\bar x)=0$. Hence, $\bar x$ is a minimizer of  $\varphi$ on $\mathcal{F}$, or, equivalently, $\bar x$ is an optimal solution of the following unconstrained optimization problem

\begin{equation*}
\text{minimizer}\ \ \varphi(x)+\delta(x; {\mathcal{F}}), x\in\mathbb{R}^n,
\end{equation*}

where $\delta(\cdot,; {\mathcal{F}})$ is the indicator function of $\mathcal{F}$ and defined by $\delta(x; {\mathcal{F}})=0$ if $x\in\mathcal{F}$ and $\delta(x; {\mathcal{F}})=+\infty$ if $x\notin\mathcal{F}$. By Proposition <ref>, we have

\begin{equation*}
0\in \partial (\varphi+\delta(\cdot\,; {\mathcal{F}}))(\bar x).
\end{equation*}

Since functions $f^L_i$, $f^U_i$, $i\in I$, and $\|\cdot-\bar x\|$ are locally Lipschitz, $\varphi$ is   locally Lipschitz. Clearly, $\delta(\cdot\,; {\mathcal{F}})$ is lower
semicontinuous around $\bar x$. Hence it follows from Proposition <ref> and the fact that $\partial \delta(\cdot\,; {\mathcal{F}})(\bar x)=N(\bar x; \mathcal{F})$ (see, e.g., <cit.>) that

\begin{equation}\label{equ:2}
0\in \partial \varphi(\bar x)+N(\bar x; \mathcal{F}).
\end{equation}

By Proposition <ref> and the fact that $\partial(\|\cdot-\bar x\|)(\bar x)=\mathbb{B}_{\mathbb{R}^n}$ (see <cit.>)), we obtain

\begin{align*}
\partial \varphi(\bar x) \subset \Big\{\sum_{i\in I}\lambda_i^L [\partial f_i^L(\bar x)+\epsilon_i^U\mathbb{B}_{\mathbb{R}^n}]+\sum_{i\in I}\lambda_i^U [\partial f_i^U(\bar x)+\epsilon_i^L\mathbb{B}_{\mathbb{R}^n}]\;:\; &\lambda_i^L, \lambda_i^U\geq 0, i\in I,
\\
&\sum_{i\in I}(\lambda_i^L+\lambda_i^U)=1\Big\}
\\
\subset \Big\{\sum_{i\in I}\left[\lambda^L_i\partial f^L_i(\bar x)+\lambda^U_i\partial_i^U(\bar x)\right] +\sum_{i\in I}\left(\lambda^L_i\epsilon^U_i+\lambda^U_i\epsilon^L_i\right)\mathbb{B}_{\mathbb{R}^n}\;:\; &\lambda_i^L, \lambda_i^U\geq 0, i\in I,
\\
&\sum_{i\in I}(\lambda_i^L+\lambda_i^U)=1\Big\}.
\end{align*}

Combining this, (<ref>) and the (<ref>), we get the assertion.

The following example is to illustrate Theorem <ref>.

Let  $f:=(f_1, f_2)$, where the functions $f_1, f_2\colon\mathbb{R}^2\to \mathcal{K}_c$ are defined by

$$f_1(x)=f_2(x)=[x_1^2+(x_1x_2-1)^2, 2x_1^2+(x_1x_2-1)^2], \ \ \forall x=(x_1, x_2)\in\mathbb{R}^2,$$

and let $g_t\colon\mathbb{R}^2\to \mathbb{R}$ be given by

$$g_t(x)=-t|x_1|-t|x_2|, \ \ \forall x=(x_1, x_2)\in\mathbb{R}^2, t\in T:=[0,1].$$

Consider problem (<ref>) with $\Omega=\mathbb{R}^2$. Clearly, the feasible set of (<ref>) is  $\mathcal{F}=\mathbb{R}^2$.

Let $\mathcal{E}=(\mathcal{E}_1, \mathcal{E}_2)$, where $\mathcal{E}_1=\mathcal{E}_2=[1, 2]$. We see that $\bar x=(0, 0)\in \mathcal{E}$-$\mathcal{S}_1^q\eqref{problem}$ and so by Remark <ref>,  $\bar x\in \mathcal{E}$-$\mathcal{S}_2^{qw}\eqref{problem}$. Indeed, suppose on the contrary that $\bar x\notin \mathcal{E}$-$\mathcal{S}_1^q\eqref{problem}$,then there exists $x\in\mathbb{R}^2$ such that

$$f_1(x)<_{LU} f_1(\bar x)-\mathcal{E}_1\|x-\bar x\|,$$

or, equivalently,

\begin{equation}\label{equa:1}
\begin{cases}
x_1^2+(x_1x_2-1)^2\leq 1-2\sqrt{x_1^2+x_2^2},
\\
2x_1^2+(x_1x_2-1)^2\leq 1-\sqrt{x_1^2+x_2^2},
\end{cases}
\end{equation}

with at least one strict inequality. Since (<ref>), we have

\begin{equation*}
\begin{cases}
x_1^2+x_1^2x^2_2+ 2 \sqrt{x_1^2+x_2^2}-2x_1x_2\leq 0
\\
2x_1^2+x_1^2x^2_2+  \sqrt{x_1^2+x_2^2}-2x_1x_2\leq 0
\end{cases}
\Leftrightarrow
\begin{cases}
\\
\end{cases}
\end{equation*}

Hence, there is no strict inequality in (<ref>), a contradiction.

Since $N(x; \mathcal{F})=N(x; \Omega)=\partial g_0(x)=\{0\}$, the (<ref>) is satisfied at every $x\in\mathbb{R}^2$. Hence, by Theorem <ref>, there exist $\lambda^L, \lambda^U\in \mathbb{R}^2_+$ with $\sum_{i\in I}(\lambda^L_i+\lambda^U_i)=1$, and $\mu\in A(\bar x)$ satisfying condition (<ref>).

We note here that the exact solution set of the problem is empty. Indeed, let $x^*$ be an arbitrary point in $\mathbb{R}^2$. Then, $0<f_i^L(x^*)\leq f_i^U(x^*)$ for all $i\in I$. Let $x^k=(\frac{1}{k}, k)$, $k\in\mathbb{N}$.  Then, for all $i\in I$, we have

\begin{align*}
\\
\end{align*}

Hence, there exists $k_0\in\mathbb{N}$ such that

\begin{align*}
f_i^L(x^k)< f_i^L(x^*),
\\
f_i^U(x^k)< f_i^U(x^*),
\end{align*}

for all $k\geq k_0$ and $i\in I$. This means that $x^*$ is not a type-2 weakly Pareto solution of (<ref>), as required.

[axis lines=center,
axis on top,

3[domain=-3:3,y domain=-3:3,colormap/viridis,surf]
3[domain=-3:3,y domain=-3:3,red,mesh]

Plot of $f^L_i$ and $f^U_i$ in Example <ref>.

Next we present sufficient conditions for approximate quasi Pareto solutions of (<ref>).  In order to obtain these sufficient conditions, we need to introduce concepts of (strictly) generalized convexity  at a given point for a family of locally Lipschitz functions. The first definition is inspired from  [37], while the second one is motivated from  [40].

* We say that $(f, g_T)$ is generalized  convex on $\Omega$ at $\bar x\in\Omega$ if for any $x\in\Omega$, $z_i^{*L}\in\partial f_i^L(\bar x)$,  $z_i^{*U}\in\partial f_i^U(\bar x)$, $i\in I$, and $x^*_t\in\partial g_t(\bar x)$, $t\in T$, there exists $\nu\in [N(\bar x; \Omega)]^\circ$ satisfying

\begin{equation}\label{equa:9}
\begin{split}
&f_i^L(x) -f_i^L(\bar x)\geq \langle z_i^{*L}, \nu\rangle, \ \ \forall  i\in I,
\\
&f_i^U(x) -f_i^U(\bar x)\geq \langle z_i^{*U}, \nu\rangle, \ \ \forall  i\in I,
\\
&g_t(x)- g_t(\bar x) \geq \langle x^*_t, \nu\rangle,\ \ \forall t\in T\\
&\text{and}\ \ \langle b^*,\nu\rangle\leq \|x-\bar x\|,\ \ \forall b^*\in\mathbb{B}_{\mathbb{R}^n}.
\end{split}
\end{equation}

* We say that $(f, g_T)$ is strictly generalized  convex on $\Omega$ at $\bar x\in\Omega$ if for any $x\in\Omega\setminus\{\bar x\}$, $z_i^{*L}\in\partial f_i^L(\bar x)$,  $z_i^{*U}\in\partial f_i^U(\bar x)$, $i\in I$, and $x^*_t\in\partial g_t(\bar x)$, $t\in T$, there exists $\nu\in [N(\bar x; \Omega)]^\circ$ satisfying

\begin{equation*}
\begin{split}
&f_i^L(x) -f_i^L(\bar x)> \langle z_i^{*L}, \nu\rangle, \ \ \forall  i\in I,
\\
&f_i^U(x) -f_i^U(\bar x)> \langle z_i^{*U}, \nu\rangle, \ \ \forall  i\in I,
\\
&g_t(x)- g_t(\bar x) \geq \langle x^*_t, \nu\rangle,\ \ \forall t\in T\\
&\text{and}\ \ \langle b^*,\nu\rangle\leq \|x-\bar x\|,\ \ \forall b^*\in\mathbb{B}_{\mathbb{R}^n}.
\end{split}
\end{equation*}

We see that if $\Omega$ is convex and $f^L_i$, $f^U_i$, $i\in I$, and $g_t$, $t\in T$, are convex (resp. strictly convex), then $(f, g_T)$ is generalized convex (resp. strictly generalized convex) on $\Omega$ at any $\bar x\in \Omega$ with $\nu=x-\bar x$. Moreover, there exist examples that show the class of generalized convex functions is properly larger than the one of convex functions; see, e.g., <cit.> and <cit.>.

* We say that $(f, g_T)$ is $\mathcal{E}$-pseudo-quasi generalized  convex on $\Omega$ at $\bar x\in\Omega$ if for any $x\in\Omega$, $z_i^{*L}\in\partial f_i^L(\bar x)$,  $z_i^{*U}\in\partial f_i^U(\bar x)$, $i\in I$, and $x^*_t\in\partial g_t(\bar x)$, $t\in T$, there exists $\nu\in [N(\bar x; \Omega)]^\circ$ satisfying

\begin{equation}\label{equ:3}
\begin{split}
&\langle z_i^{*L}, \nu\rangle +\epsilon_i^U\|x-\bar x\|\geq 0 \Rightarrow f_i^L(x) \geq f_i^L(\bar x)-\epsilon_i^U\|x-\bar x\|, \ \ \forall  i\in I,
\\
&\langle z_i^{*U}, \nu\rangle +\epsilon_i^L\|x-\bar x\|\geq 0 \Rightarrow f_i^U(x) \geq f_i^U(\bar x)-\epsilon_i^L\|x-\bar x\|, \ \  \forall i\in I,
\\
&g_t(x)\leq g_t(\bar x) \Rightarrow \langle x^*_t, \nu\rangle\leq 0,\ \ \forall t\in T\\
&\text{and}\ \ \langle b^*,\nu\rangle\leq \|x-\bar x\|,\ \ \forall b^*\in\mathbb{B}_{\mathbb{R}^n}.
\end{split}
\end{equation}

* We say that $(f, g_T)$ is strictly $\mathcal{E}$-pseudo-quasi generalized  convex on $\Omega$ at $\bar x\in\Omega$ if for any $x\in\Omega\setminus\{\bar x\}$, $z_i^{*L}\in\partial f_i^L(\bar x)$,  $z_i^{*U}\in\partial f_i^U(\bar x)$, $i\in I$, and $x^*_t\in\partial g_t(\bar x)$, $t\in T$, there exists $\nu\in [N(\bar x; \Omega)]^\circ$ satisfying

\begin{equation}\label{equa:8}
\begin{split}
&\langle z_i^{*L}, \nu\rangle +\epsilon_i^U\|x-\bar x\|\geq 0 \Rightarrow f_i^L(x) > f_i^L(\bar x)-\epsilon_i^U\|x-\bar x\|, \ \ \forall  i\in I,
\\
&\langle z_i^{*U}, \nu\rangle +\epsilon_i^L\|x-\bar x\|\geq 0 \Rightarrow f_i^U(x) > f_i^U(\bar x)-\epsilon_i^L\|x-\bar x\|,\ \  \forall i\in I,
\\
&g_t(x)\leq g_t(\bar x) \Rightarrow \langle x^*_t, \nu\rangle\leq 0, \ \ \forall t\in T,
\\
&\text{and}\ \ \langle b^*,\nu\rangle\leq \|x-\bar x\|,\ \ \forall b^*\in\mathbb{B}_{\mathbb{R}^n}.
\end{split}
\end{equation}

By definition, it is easy to see that if  $(f, g_T)$ is  (strictly) generalized  convex on $\Omega$ at $\bar x\in\Omega$, then for any $\mathcal{E}=(\mathcal{E}_1, \ldots, \mathcal{E}_m)$, where $\mathcal{E}_i=[\epsilon_i^L, \epsilon_i^u]$, $0\leq\epsilon_i^L\leq \epsilon_i^U$, $i\in I$,   $(f, g_T)$ is (strictly) $\mathcal{E}$-pseudo-quasi generalized  convex on $\Omega$ at $\bar x\in\Omega$. Furthermore, the class of  (strictly) $\mathcal{E}$-pseudo-quasi generalized  convex functions is  properly wider than the one of (strictly) generalized convex functions. To see this, let us consider the following example.

Let $m=1$, $f\colon \mathbb{R}\to\mathcal{K}_c$ be defined by $f(x)=[f^L(x), f^U(x)]$, where

\begin{equation*}
\begin{cases}
\dfrac{x}{2},  & x\geq 0,
\smallskip
\\
\dfrac{2x}{3},  & x<0,
\end{cases}
\ \ \ \
\begin{cases}
\dfrac{2x}{3},  & x\geq 0,
\smallskip
\\
\dfrac{x}{2},  & x<0,
\end{cases}
\end{equation*}

and $g_t(x)=-tx$ for all $x\in\mathbb{R}$ and $t\in T:=[0, 1]$.	 Take $\Omega=\mathbb{R}$ and $\mathcal{E}=[\frac{2}{3}, \frac{3}{4}]$. We show that $(f, g_T)$ is strictly $\mathcal{E}$-pseudo-quasi generalized  convex at $\bar x=0$ but not  generalized  convex at this point. Indeed, it is easy to see that

\begin{equation*}
f^L(x)>f^L(\bar x)-\frac{3}{4}|x|, f^U(x)>f^U(\bar x)-\frac{2}{3}|x|, \ \ \forall x\in\mathbb{R}\setminus\{0\}.
\end{equation*}

Hence, by letting $\nu=0$, we see that all conditions in (<ref>) are satisfied for all  $z^{*L}\in\partial f^L(\bar x)$,  $z^{*U}\in\partial f^U(\bar x)$, $x^*_t\in\partial g_t(\bar x)$, $t\in T$, and $b^*\in\mathbb{B}_{\mathbb{R}^n}$.

We now show that $(f, g_T)$ is  not generalized  convex at $\bar x$ and so it is not strictly generalized  convex at this point. We have $\partial f^L(\bar x)=\partial f^U(\bar x)=\{\frac{1}{2}, \frac{2}{3}\}$ and $\partial g_t(\bar x)=\{-t\}$, $\forall t\in T$. Let $x=1$, $z^{*L}=z^{*U}=\frac{2}{3}$, $x^{*}_t=-t$, and assume that there exists $\nu\in\mathbb{R}$ satisfying  (<ref>). Then

\begin{align*}
\frac{1}{2}&\geq \frac{2\nu}{3},
\\
\frac{2}{3}&\geq \frac{2\nu}{3},
\\
-t&\geq -t\nu, \ \ \forall t\in [0, 1].
\end{align*}

Hence, $1 \leq \nu\leq \frac{3}{4},$ a contradiction.

[axis lines=center,
axis on top,
[domain=0:5,y domain=-5:5,red]
[domain=-5:0,y domain=-5:5,blue]
[domain=-5:0,y domain=-5:5,red]

[domain=0:5,y domain=-5:5,blue]

Plot of $f^L$ and $f^U$ in Example <ref>.

Let $\bar x\in\mathcal{F}$ and assume that there exist $\lambda^L, \lambda^U\in \mathbb{R}^m_+$ with $\sum_{i\in I}(\lambda_i^L+\lambda_i^U)=1$, and $\mu\in A(\bar x)$ satisfying (<ref>).

* If $(f, g_T)$ is $\mathcal{E}$-pseudo-quasi generalized convex on $\Omega$ at $\bar x$, then $\bar x\in \mathcal{E}$-$\mathcal{S}_2^{qw}\eqref{problem}$.

* If $(f, g_T)$ is strictly $\mathcal{E}$-pseudo-quasi generalized convex on $\Omega$ at $\bar x$, then $\bar x\in \mathcal{E}$-$\mathcal{S}_1^{q}\eqref{problem}$ and we therefore get $\bar x\in \mathcal{E}$-$\mathcal{S}_2^{q}\eqref{problem}$ and $\bar x\in \mathcal{E}$-$\mathcal{S}_1^{qw}\eqref{problem}$.

Since $\bar x$ satisfies (<ref>) with respect to $(\lambda^L, \Lambda^U, \mu)$, there exist $z_i^{*L}\in \partial f_i^L(\bar x)$, $z_i^{*U}\in \partial f_i^U(\bar x)$, $i\in I$, $x_t^*\in \partial g_t(\bar x)$, $t\in T$, $b^*\in \mathbb{B}_{\mathbb{R}^n}$, and $\omega^*\in N(\bar x; \Omega)$ such that

\begin{equation*}
\sum_{i\in I}[\lambda_i^L z_i^{*L}+ \lambda_i^U z_i^{*U}]+\sum_{t\in T}\mu_t x^*_t+\sum_{i\in I}(\lambda_i^L\epsilon_i^U+\lambda_i^U\epsilon_i^L)b^*+\omega^*=0,
\end{equation*}

or, equivalently,

\begin{equation}\label{equ:5}
\sum_{i\in I}[\lambda_i^L z_i^{*L}+ \lambda_i^U z_i^{*U}]+\sum_{t\in T}\mu_t x^*_t+\sum_{i\in I}(\lambda_i^L\epsilon_i^U+\lambda_i^U\epsilon_i^L)b^*=-\omega^*.
\end{equation}

We first justify (i). Suppose on the contrary that $\bar x$ is not a type-2 $\mathcal{E}$-quasi-weakly Pareto solution of (<ref>), then there exists $x\in\mathcal{F}$ such that
$$f_i(x)<^s_{LU} f_i(\bar x)-\mathcal{E}_i\|x-\bar x\|,\ \  \forall i\in I,$$

or, equivalently,

\begin{equation}\label{equ:4}
f_i^L(x)< f_i^L(\bar x)-\epsilon_i^U\|x-\bar x\| \ \ \text{and}\ \ f_i^U(x)< f_i^U(\bar x)-\epsilon_i^L\|x-\bar x\|, \ \ \forall i\in I.
\end{equation}

By the $\mathcal{E}$-pseudo-quasi generalized convexity of $(f, g_T)$, there is $\nu\in [N(\bar x; \Omega)]^\circ$ satisfying (<ref>). Thus, we deduce from (<ref>) and (<ref>) that

\begin{equation*}
\langle z_i^{*L}, \nu\rangle +\epsilon_i^U\|x-\bar x\|< 0\ \ \text{and}\ \ \langle z_i^{*U}, \nu\rangle +\epsilon_i^L\|x-\bar x\|< 0.
\end{equation*}

For each $t\in T(\mu)$, we have $g_t(\bar x)=0$. Hence,
$g_t(x)\leq g_t(\bar x)$ for all $t\in T(\mu)$. This, together with (<ref>), gives

\begin{equation*}
\langle x^*_t, \nu\rangle\leq 0\ \ \text{for all}\ \ t\in T(\mu).
\end{equation*}

Since  $\nu\in [N(\bar x; \Omega)]^\circ$, (<ref>), and (<ref>), we obtain

\begin{align*}
0&\leq \langle -\omega^*,\nu\rangle=\sum_{i\in I}[\lambda_i^L \langle z_i^{*L}, \nu\rangle+ \lambda_i^U \langle z_i^{*U}, \nu\rangle]+\sum_{t\in T}\mu_t\langle x^*_t, \nu\rangle+\sum_{i\in I}(\lambda_i^L\epsilon_i^U+\lambda_i^U\epsilon_i^L)\langle b^*,\nu\rangle
\\
&=\sum_{i\in I}[\lambda_i^L \langle z_i^{*L}, \nu\rangle+ \lambda_i^U \langle z_i^{*U}, \nu\rangle]+\sum_{t\in T(\mu)}\mu_t\langle x^*_t, \nu\rangle+\sum_{i\in I}(\lambda_i^L\epsilon_i^U+\lambda_i^U\epsilon_i^L)\langle b^*,\nu\rangle
\\
&<-\sum_{i\in I}[\lambda_i^L\epsilon_i^U\|x-\bar x\|+\lambda_i^U\epsilon_i^L\|x-\bar x\|]+\sum_{i\in I}(\lambda_i^L\epsilon_i^U+\lambda_i^U\epsilon_i^L)\|x-\bar x\|=0,
\end{align*}

a contradiction. The proof of (i) is completed.

We now prove (ii). Suppose on the contrary that $\bar x$ is not a type-1 $\mathcal{E}$-quasi  Pareto solution of (<ref>), then there exists $x\in\mathcal{F}$ such that

$$f_i(x)\leq_{LU} f_i(\bar x)-\mathcal{E}_i\|x-\bar x\|,\ \ \forall i\in I,$$

where at least one of the inequalities is strict. This implies that $x\neq \bar x$. Therefore, by the strictly $\mathcal{E}$-pseudo-quasi generalized convexity of $(f, g_T)$ at $\bar x$, (<ref>), and by the same argument as in the proof of  part (i), we can deduce the contradiction. Hence, $\bar x$ is  a type-1 $\mathcal{E}$-quasi  Pareto solution of (<ref>).  The proof is completed.

* By Remark <ref> and <cit.>,  the condition (<ref>) alone is not sufficient to 	guarantee that $\bar x$ is a $\mathcal{E}$-quasi (-weakly) Pareto solution of (<ref>) if the (strict) $\mathcal{E}$-pseudo generalized convexity of 	$(f, g_T)$ on $\Omega$ at $\bar x$ is violated.

* If $(f, g_T)$ is $\mathcal{E}$-pseudo-quasi generalized convex on $\Omega$ at $\bar x\in \mathcal{F}$ and there exist $\lambda^L, \lambda^U\in\mathbb{R}^m_+$ with $\lambda_i^L>0, \lambda_i^U>0$, $\forall i\in I$, $\sum_{i\in I}(\lambda_i^L+\lambda_i^U)=1$, and $\mu\in A(\bar x)$ satisfying (<ref>), then $\bar x\in \mathcal{E}$-$\mathcal{S}_1^{q}\eqref{problem}$.

* Since the class of (strictly) $\mathcal{E}$-pseudo-quasi generalized convex functions is properly wider than the class of (strictly) generalized convex functions, our results in Theorem <ref> generalize and improve the corresponding results in [27, 38, 30, 22].  To see this, let us consider the following simple example.

Let $m=1$, $f\colon \mathbb{R}\to\mathcal{K}_c$ be defined by $f(x)=[f^L(x), f^U(x)]$, where

\begin{equation*}
\begin{cases}
\dfrac{x}{2},  & x\geq 0,
\smallskip
\\
\dfrac{2x}{3},  & x<0.
\end{cases}
\end{equation*}

Consider problem (<ref>) with $g_t(x)=-tx$ for all $x\in\mathbb{R}$ and $t\in T:=[0, 1]$, $\Omega=\mathbb{R}$ and $\mathcal{E}=[\frac{2}{3}, \frac{2}{3}]$. Actually, in this case problem (<ref>) is a  semi-infinite programming problem.  Analysis similar to that in Example  <ref> shows that $(f, g_T)$ is strictly $\mathcal{E}$-pseudo-quasi convex at $\bar x=0$ but not generalized convex at this point. It is easy to see that $\bar x$ satisfies condition (<ref>); e.g., $\lambda^L=\lambda^U=\frac{1}{2}$, $\mu_{\frac{1}{2}}=1$,  and $\mu_t=0$ for all $t\in T\setminus\{\frac{1}{2}\}$. Hence, by Theorem <ref>, $\bar x\in \mathcal{E}$-$\mathcal{S}_1^{q}\eqref{problem}$. However, since  $(f, g_T)$ is not  generalized convex at $\bar x$, <cit.>, <cit.>, <cit.>, and <cit.> cannot be applied for this example.

§ APPROXIMATE DUALITY THEOREMS

Let $\mathcal{A}:=(A_1, \ldots, A_m)$ and $\mathcal{B}:=(B_1, \ldots, B_m)$, where $A_i$, $B_i$, $i\in I$, are intervals in $\mathcal{K}_c$. In what follows,   we use the following notations for convenience.

\begin{align*}
\mathcal{A}&\preceq_{LU}\mathcal{B} \Leftrightarrow
\begin{cases}
A_i\leq_{LU} B_i, \ \ \forall i\in I,
\\
A_k<_{LU} B_k,  \ \ \text{for at least one}\ \ k\in I.
\end{cases}
\\
\mathcal{A}&\npreceq_{LU}\mathcal{B} \ \ \text{is the negation of}\ \ \mathcal{A}\preceq_{LU}.
\\
\mathcal{A}&\prec^s_{LU}\mathcal{B}  \Leftrightarrow A_i<^s_{LU} B_i, \ \ \forall i\in I.
\\
\mathcal{A}&\nprec^s_{LU}\mathcal{B} \ \ \text{is the negation of}\ \ \mathcal{A}\prec^s_{LU}\mathcal{B}.
\end{align*}

For $y\in\mathbb{R}^n$, $(\lambda^L, \lambda^U)\in\mathbb{R}^m_+\times\mathbb{R}^m_+\setminus\{(0,0)\}$, and $\mu\in \mathbb{R}^{|T|}_+$, put

\begin{align*}
\mathcal{L}(y, \lambda^L,\lambda^U, \mu):=f(y)=\left([f_1^L(y), f_1^U(y)], \ldots, [f_m^L(y), f_m^U(y)]\right).
\end{align*}

In connection with the primal problem (<ref>), we consider the following dual problem in the sense of Mond–Weir (stated in an approximate form):

\begin{align}
\label{Dual-problem}\tag{SIVD$_{MW}$}
&\max_{y\in\Omega}\ \  \mathcal{L}(y, \lambda^L,\lambda^U, \mu)
\\
&\text{s. t.}\ \ (y, \lambda^L,\lambda^U, \mu)\in\mathcal{F}_{MW},\notag
\end{align}

where the feasible set is defined by

\begin{align*}
&\mathcal{F}_{MW}:=\big\{(y, \lambda^L, \lambda^U, \mu)\in \Omega\times\mathbb{R}^m_+ \times\mathbb{R}^m_+\times\mathbb{R}^{|T|}_+\,:\, 0\in \sum_{i\in I}[\lambda_i^L\partial f_i^L(y)+\lambda_i^U\partial f_i^U(y)]+
\\
&\sum_{t\in T}\mu_t\partial g_t(y)+\sum_{i\in I}(\lambda_i^L\epsilon_i^U+\lambda_i^U\epsilon_i^L)\mathbb{B}_{\mathbb{R}^n}+N(y; \Omega), \mu_tg_t(y)\geq 0, t\in T,  \sum_{i\in I}(\lambda_i^L+\lambda_i^U)=1\big\}.
\end{align*}

It should be noticed that approximate  quasi Pareto solutions of the dual problem (<ref>)  are defined similarly as in Definition <ref> by replacing $\leq_{LU}$, $<_{LU}$, and $<^s_{LU}$ by  $\geq_{LU}$, $>_{LU}$, and $>^s_{LU}$, respectively. For example, we say that $(\bar y, \bar  \lambda^L,\bar \lambda^U, \bar \mu)\in\mathcal{F}_{MW}$ is a type-2 $\mathcal{E}$-quasi weakly Pareto solution of (<ref>) if there is no $(y, \lambda^L, \lambda^U, \mu)\in \mathcal{F}_{MW}$ such that

\begin{equation*}
\mathcal{L}_i(y, \lambda^L,\lambda^U, \mu)-\mathcal{E}_i\|y-\bar y\|>^s_{LU} \mathcal{L}_i(\bar y, \bar \lambda^L,\bar\lambda^U, \bar\mu),\ \ \forall i\in I,
\end{equation*}

or, equivalently,

\begin{equation*}
\mathcal{L}(\bar y, \bar \lambda^L,\bar\lambda^U, \bar\mu)\nprec^s_{LU}\mathcal{L}_i(y, \lambda^L,\lambda^U, \mu)-\mathcal{E}_i\|y-\bar y\|,\ \ \forall (y, \lambda^L, \lambda^U, \mu)\in \mathcal{F}_{MW}.
\end{equation*}

The following theorem describes weak duality relations for approximate quasi Pareto solutions between the primal problem (<ref>) and the dual problem (<ref>).
Let $x\in \mathcal{F}$ and $(y, \lambda^L, \lambda^U, \mu)\in\mathcal{F}_{MW}$.

* If $(f, g_T)$ is $\mathcal{E}$-pseudo-quasi generalized convex on $\Omega$ at $y$, then

\begin{equation*}
f(x) \nprec^s_{LU} \mathcal{L}(y, \lambda^L, \lambda^U, \mu) -\mathcal{E}\|x-y\|.
\end{equation*}

* If $(f, g_T)$ is strictly $\mathcal{E}$-pseudo-quasi generalized convex on $\Omega$ at $y$, then

\begin{equation*}
f(x)\npreceq_{LU} \mathcal{L}(y, \lambda^L, \lambda^U, \mu)-\mathcal{E}\|x-y\|.
\end{equation*}

Since $x\in\mathcal{F}$ and $(y, \lambda^L, \lambda^U, \mu)\in\mathcal{F}_{MW}$,  we have $x, y\in\Omega$,

\begin{equation}\label{equ:7}
g_t(x)\leq 0, \ \ \mu_tg_t(y)\geq 0, \ \ \forall t\in T,
\end{equation}

\begin{equation*}
0\in \sum_{i\in I}[\lambda_i^L\partial f_i^L(y)+\lambda_i^U\partial f_i^U(y)]+\sum_{t\in T}\mu_t\partial g_t(y)+\sum_{i\in I}(\lambda_i^L\epsilon_i^U+\lambda_i^U\epsilon_i^L)\mathbb{B}_{\mathbb{R}^n}+N(y; \Omega).
\end{equation*}

Hence, there exist $z_i^{*L}\in \partial f_i^L(y)$, $z_i^{*U}\in \partial f_i^U(y)$, $i\in I$, $x^*_t\in\partial g_t(y)$, $t\in T$, $b^*\in\mathbb{B}_{\mathbb{R}^n}$, and $\omega^*\in N(y; \Omega)$ such that

\begin{equation}\label{equ:6}
\sum_{i\in I}[\lambda_i^L z_i^{*L}+ \lambda_i^U z_i^{*U}]+\sum_{t\in T}\mu_t x^*_t+\sum_{i\in I}(\lambda_i^L\epsilon_i^U+\lambda_i^U\epsilon_i^L)b^*=-\omega^*.
\end{equation}

We first justify (i). Assume to the contrary that

\begin{equation*}
f(x)\prec^s_{LU} \mathcal{L}(y, \lambda^L, \lambda^U, \mu)-\mathcal{E}\|x-y\|.
\end{equation*}

This means that

\begin{equation*}
f_i(x)\prec^s_{LU} \mathcal{L}_i(y, \lambda^L, \lambda^U, \mu) -\mathcal{E}_i\|x-y\|, \ \ \forall i\in I,
\end{equation*}

or, equivalently,

\begin{align*}
\begin{cases}
\\
\end{cases}
\end{align*}

for all $i\in I$. By (<ref>) and $\mu_t\geq 0$ for all $t\in T$, we have

\begin{equation*}
g_t(x)\leq 0\leq g_t(y)
\end{equation*}

for all $t\in T(\mu)$. Since $(f, g_T)$ is  $\mathcal{E}$-pseudo-quasi generalized convex  on $\Omega$ at $y$, there exists $\nu\in [N(y, \Omega)]^\circ$ such that

\begin{align*}
&\langle z_i^{*L}, \nu\rangle +\epsilon_i^U\|x-y\|<0, \ \ \forall i\in I,
\\
&\langle z_i^{*U}, \nu\rangle +\epsilon_i^L\|x-y\|<0, \ \ \forall i\in I,
\\
&\langle x^*_t, \nu\rangle \leq 0, \ \  \forall t\in T(\mu),
\\
&\text{and}\ \ \langle b^*,\nu\rangle\leq \|x-y\|,\ \ \forall b^*\in\mathbb{B}_{\mathbb{R}^n}.
\end{align*}

Since $\sum_{i\in I}(\lambda_i^L+\lambda_i^U)=1$, one has

\begin{align*}
\sum_{i\in I}\big[\langle \lambda_i^Lz_i^{*L}&+\lambda_i^Uz_i^{*U}, \nu\rangle\big]+\sum_{t\in T}\langle x^*_t, \nu\rangle+\sum_{i\in I}(\lambda_i^L\epsilon_i^U+\lambda_i^U\epsilon_i^L)\langle b^*, \nu\rangle \leq
\\
&\sum_{i\in I}\big[\langle \lambda_i^Lz_i^{*L}+\lambda_i^Uz_i^{*U}, \nu\rangle\big]+\sum_{t\in T}\langle x^*_t, \nu\rangle+\sum_{i\in I}(\lambda_i^L\epsilon_i^U+\lambda_i^U\epsilon_i^L)\|x-y\|<0.
\end{align*}

On the other hand, since  $\nu\in [N(y, \Omega)]^\circ$, $\omega^*\in N(y; \Omega)$, and (<ref>), we have

\begin{equation*}
0\leq \langle -\omega^*, \nu\rangle= \sum_{i\in I}\big[\langle \lambda_i^Lz_i^{*L}+\lambda_i^Uz_i^{*U}, \nu\rangle\big]+\sum_{t\in T}\langle x^*_t, \nu\rangle+\sum_{i\in I}(\lambda_i^L\epsilon_i^U+\lambda_i^U\epsilon_i^L)\langle b^*, \nu\rangle <0,
\end{equation*}

a contradiction, which completes the proof of (i).

We now prove (ii). Suppose on the contrary that

\begin{equation*}
f(x)\preceq_{LU} \mathcal{L}(y, \lambda^L, \lambda^U, \mu)-\mathcal{E}\|x-y\|.
\end{equation*}

This implies that

\begin{equation*}
\begin{cases}
f_i(x)\leq_{LU} f_i(y)-\mathcal{E}_i\|x-y\|, \ \ \forall i\in I,
\\
f_k(x)<_{LU} f_k(y)-\mathcal{E}_k\|x-y\|, \ \ \text{for at least one} \ \ k\in I.
\end{cases}
\end{equation*}

Hence, $x\neq y$.  Therefore, by the strictly $\mathcal{E}$-pseudo-quasi generalized convexity of $(f, g_T)$ at $y$, (<ref>), and by the same argument as in the proof of  part (i), we can get the contradiction. The proof is completed.

The following example shows that the approximate pseudo-quasi generalized convexity of $(f,g_T)$ on $\Omega$ used in Theorem <ref> cannot be omitted.

Let $m=1$, $f\colon \mathbb{R}\to\mathcal{K}_c$ be defined by $f(x)=[f^L(x), f^U(x)]$, where

$$f^L(x)=\frac{1}{3}x^3 \ \ \text{and}\ \ f^U(x)=\frac{1}{3}x^3+1.$$

Consider problem (<ref>) with $g_t(x)=-tx$ for all $x\in\mathbb{R}$ and $t\in T:=[0, 1]$, $\Omega=\mathbb{R}$ and $\mathcal{E}=[\frac{1}{5}, \frac{1}{4}]$.  Let $\bar y=1$, $\bar \lambda^L=\bar \lambda^U=\frac{1}{2}$, $\bar \mu_t=0$, $\forall t\in [0, 1)$, and $\bar \mu_1=1$. It is easy to see that $(\bar y, \bar\lambda^L,\bar \lambda^U, \bar \mu)\in \mathcal{F}_{MW}$. However, for $\bar x=0\in\mathcal{F}$, we have

\begin{align*}
f(\bar x)=[0, 1]&<^s_{LU} \mathcal{L}(\bar y, \bar\lambda^L, \bar \lambda^U, \bar{\mu})
\\
&=\bigg[\frac{1}{3}, \frac{4}{3}\bigg]-\bigg[\frac{1}{5}, \frac{1}{4}\bigg]=\bigg[\frac{1}{12}, \frac{17}{15}\bigg].
\end{align*}

This means that the conclusion of Theorem <ref> is no longer true. The reason is that the $\mathcal{E}$-pseudo-quasi generalized convexity of $(f,g_T)$ on $\Omega$ at $\bar y$ has been violated.

[axis lines=center,
axis on top,

[domain=-5:5,y domain=-5:5,red]
[domain=-5:5,y domain=-5:5,blue]

Plot of $f^L$ and $f^U$ in Example <ref>.

Next we present a theorem that formulates strong duality relations between the primal problem (<ref>) and the dual problem (<ref>).
Let $\bar x$ be a type-2 $\mathcal{E}$-quasi-weakly Pareto solution of (<ref>) and assume that the (<ref>) holds at this point. Then there exist $\bar \lambda^L, \bar\lambda^U\in \mathbb{R}^m_+$, and $\bar \mu\in A(\bar x)$ such that $(\bar x, \bar \lambda^L, \bar \lambda^U, \bar \mu)\in \mathcal{F}_{MW}$, $f(\bar x)=\mathcal{L}(\bar x, \bar \lambda^L, \bar\lambda^U, \bar \mu)$. Furthermore,

* If $(f, g_T)$ is $\mathcal{E}$-pseudo-quasi generalized convex on $\Omega$ at $\bar x$, then $(\bar x, \bar \lambda^L, \bar \lambda^U, \bar \mu)$ is a type-2 $\mathcal{E}$-quasi weakly Pareto solution of (<ref>).

* If $(f, g_T)$ is strictly $\mathcal{E}$-pseudo-quasi generalized convex on $\Omega$ at $\bar x$, then $(\bar x, \bar \lambda^L, \bar \lambda^U, \bar \mu)$ is a type-1 $\mathcal{E}$-quasi  Pareto solution of (<ref>).

By Theorem <ref>, there exist $\bar\lambda^L, \bar\lambda^U\in \mathbb{R}^m_+$, and $\bar\mu\in A(\bar x)$ satisfying $(\bar x, \bar\lambda^L, \bar\lambda^U, \bar\mu)\in \mathcal{F}_{MW}$. Clearly,

$$f(\bar x)=\mathcal{L}(\bar x, \bar\lambda^L, \bar\lambda^U, \bar\mu).$$

(i) If $(f, g_T)$ is $\mathcal{E}$-pseudo-quasi generalized convex on $\Omega$ at $\bar x$, then, by invoking now (i) of Theorem <ref>, we obtain

\begin{equation*}
f(\bar x)=\mathcal{L}(\bar x, \bar\lambda^L, \bar\lambda^U, \bar\mu)\nprec^s_{LU} L(y,\lambda^L,\lambda^U, \mu)-\mathcal{E}\|\bar x-y\|
\end{equation*}

for all $(y,\lambda^L,\lambda^U, \mu)\in\mathcal{F}_{MW}$. Therefore, $(\bar x, \bar\lambda^L, \bar\lambda^U, \bar\mu)$ is a type-2 $\mathcal{E}$-quasi weakly Pareto solution of (<ref>).

The proof of (ii) is similar to that of (i) by using the strictly $\mathcal{E}$-pseudo-quasi generalized convexity of $(f, g_T)$ on $\Omega$ at $\bar x$ instead of the $\mathcal{E}$-pseudo-quasi generalized convexity of $(f, g_T)$ on $\Omega$ at the corresponding point.

We close this section by presenting converse-like duality relations for approximate quasi Pareto solutions between the primal problem (<ref>)  and the dual problem  (<ref>).
Let $(\bar x, \bar\lambda^L, \bar\lambda^U, \bar\mu)\in \mathcal{F}_{MW}$.

* If $\bar x\in\mathcal{F}$ and $(f, g_T)$ is $\mathcal{E}$-pseudo-quasi generalized convex on $\Omega$ at $\bar x$, then $\bar x$ is a type-2 $\mathcal{E}$-quasi weakly Pareto solution of (<ref>).

* If $\bar x\in\mathcal{F}$ and $(f, g_T)$ is strictly $\mathcal{E}$-pseudo-quasi generalized convex on $\Omega$ at $\bar x$, then $\bar x$ is a type-1 $\mathcal{E}$-quasi  Pareto solution of (<ref>).

(i) Since $(\bar x, \bar\lambda^L, \bar\lambda^U, \bar\mu)\in \mathcal{F}_{MW}$, we have

\begin{equation*}
0\in \sum_{i\in I}[\bar\lambda_i^L\partial f_i^L(\bar x)+\bar\lambda_i^U\partial f_i^U(\bar x)]+\sum_{t\in T}\bar \mu_t\partial g_t(\bar x)+\sum_{i\in I}(\bar \lambda_i^L\epsilon_i^U+\bar \lambda_i^U\epsilon_i^L)\mathbb{B}_{\mathbb{R}^n}+N(\bar x; \Omega),
\end{equation*}

$\sum_{i\in I}(\bar\lambda_i^L+\bar\lambda_i^U)=1$, and

\begin{equation}\label{equ:9}
\bar \mu_t g_t(\bar x)\geq 0, \ \ \forall t\in T.
\end{equation}

It follows from  $\bar x\in\mathcal{F}$ and (<ref>) that $\bar \mu_t g_t(\bar x)= 0$ for all $t\in T$, i.e., $\bar \mu\in A(\bar x)$. By the  $\mathcal{E}$-pseudo-quasi generalized convexity of $(f, g_T)$ on $\Omega$ at $\bar x$ and  Theorem <ref>(i), $\bar x$ is a type-2 $\mathcal{E}$-quasi weakly Pareto solution of (<ref>).

(ii) The proof of (ii) is quite similar to that of (i) by using the strictly $\mathcal{E}$-pseudo-quasi generalized convexity of $(f, g_T)$  and Theorem <ref>(ii), so it is omitted.

§ CONCLUSIONS

In this paper, we focus for the first time on studying approximate solutions of semi-infinite interval-valued vector optimization problems. By employing    the Mordukhovich/limiting subdifferential and the Mordukhovich/limiting normal cone and introducing some types of approximate pseudo-quasi generalized convexity of a family of locally Lipschitz functions, we establish optimality conditions of KKT-type and duality relations for proposed approximate solutions of the considered problem. Observe that the class of approximate pseudo-quasi generalized convex functions is properly wider than the class of convex functions  in the sense of [37, 27, 41], our results generalize  and improve some  existing results. Furthermore, the model of problems of the form (<ref>) covers the one of cone-constrained convex vector optimization problems and semidefinite vector optimization problems,  and so our approach can be developed to study these problems. We aim to investigate this problem in future work.

§ ACKNOWLEDGMENTS

We would like to thank the anonymous referee for insightful comments and valuable suggestions.

§ DISCLOSURE STATEMENT

No potential conflict of interest was reported by the authors.

§ FUNDING

This research is funded by Hanoi Pedagogical University 2 under grant number HPU2.UT-2021.15.

[1]
Ben-Tal  A, Nemirovski A. Robust convex optimization. Math Oper Res. 1998;23:769–805.

[2]
Ben-Tal  A, Nemirovski A. A selected topics in robust convex optimization. Math Program. 2008;112:125–158.

[3]
Ben-Tal A, Ghaoui LE, Nemirovski A.
Robust optimization. Princeton(NJ): Princeton University Press; 2009.
[4]
Rahimi M, Soleimani-Damaneh M. Robustness in deterministic vector optimization. J Optim  Theory Appl. 2018;179:137–162.
[5]
SŁowiński R. Fuzzy sets in decision analysis, operations research and statistics. Boston (MA): Kluwer Academic Publishers;   1998.
[6]
Wu HC.  Duality theory for optimization problems with interval-valued objective functions. J Optim
Theory Appl. 2010;144:615–628.
[7]
Chalco-Cano Y,  Lodwick WA, Rufian-Lizana A.  Optimality conditions of type KKT for optimization problem with interval-valued objective function via generalized derivative. Fuzzy Optim Decis Mak. 2013;12:305–322.
[8]
Ishibuchi H,  Tanaka H. Multiobjective programming in optimization of the interval objective function. European J Oper Res. 1990;48:219–225.

[9]
Jennane M, Kalmoun EM, Lafh L. Optimality conditions for nonsmooth interval-valued and multiobjective semi-infinite programming. RAIRO-Oper Res. 2021;55:1–11.
[10]
Kumar P, Sharma B,  Dagar J. Interval-valued programming problem with infinite constraints.  J Oper Res Soc China. 2018;6:611–626.
[11]
Osuna-Gómez  R, Hernádez-Jiménez B, Chalco-Cano Y, et al. New efficiency conditions 	for multiobjective interval-valued programming problems. Inf Sci. 2017;420:235–248.
[12]
Qian X, Wang KR, Li XB. Solving vector interval-valued optimization problems with infinite interval constraints via integral-type penalty function. Optimization. 2021. https://doi.org/10.1080/02331934.2021.1906872.
[13]
Singh AD, Dar BA. Optimality conditions in multiobjective programming problems with interval valued objective functions. Control Cybern.  2015;44:19–45.

[14]
Singh D,  Dar  BA,  Kim  DS. KKT optimality conditions in interval valued multiobjective programming with generalized differentiable functions. European J Oper Res. 2016;254:29–39.

[15]
Singh D,  Dar  BA,  Kim  DS. Sufficiency and duality in non-smooth interval valued programming problems. J Ind Manag Optim.  2019;15:647–665.
[16]
Su TV, Dinh DH. Duality results for interval-valued pseudoconvex optimization problem with equilibrium constraints with applications. Comput Appl Math. 2020;39:1–24.

[17]
Tung LT. Karush–Kuhn–Tucker optimality conditions and duality for semi-infinite programming with multiple interval-valued objective functions. J Nonlinear Funct Anal.  2019;2019:1–21.

[18]
Tung LT. Karush–Kuhn–Tucker optimality conditions and duality for convex semi-infinite programming with multiple interval-valued objective functions. J Appl Math Comput. 2020;62:67–91.
[19]
Wu HC.  The Karush–Kuhn–Tuker optimality conditions in an optimization problem with interval valued objective functions. Eur J Oper Res. 2007;176:46–59.

[20]
Wu HC.  The Karush–Kuhn–Tucker optimality conditions in multiobjective programming problems with interval-valued objective functions. Eur J Oper Res.  2009;196:49–60.

[21]
Wu HC.  The Karush–Kuhn–Tucker optimality conditions for multi-objective programming problems with fuzzy-valued objective functions. Fuzzy Optim Decis Mak. 2009;8:1–28.
[22]
Tuyen NV. Approximate solutions of interval-valued optimization problems. Investigación Oper.  2021;42:223–237.
[23]  Jahn J. Vector optimization: theory, applications and extensions. Berlin: Springer; 2011.
[24] Luc DT. Theory of vector optimization. Berlin: Springer; 1989.
[25]
Bao TQ, Eichfelder G, Soleimani B, et al. Ekeland's variational principle for vector optimization with variable ordering structure. J Convex Anal. 2017;24:393–415.
[26]
Chen Z. Characterizations of solution sets for parametric multiobjective optimization problems. Appl. Anal. 2013;92:2457–2467.
[27]
Chuong TD,  Kim DS. Approximate solutions of multiobjective optimization problems. Positivity. 2016;20:187–207.
[28]
Kim DS, Son TQ. An approach to $\epsilon$-duality theorems for nonconvex semi-infinite multiobjective optimization problems.  Taiwanese J Math. 2018;22:1261–1287.
[29]
Loridan P.  $\epsilon$-solutions in vector minimization problems. J Optim Theory Appl.  1984;43:265–276.
[30]
Son TQ,   Tuyen  NV, Wen  CF. Optimality conditions for approximate Pareto solutions of a nonsmooth vector optimization problem with an infinite number of constraints. Acta Math Vietnam. 2020;45:435–448.
[31]
Tuyen NV,  Xiao YB, Son TQ.  On approximate KKT optimality conditions for cone-constrained vector optimization problems. J Nonlinear Convex Anal. 2020;21:105–117.
[32]
Mordukhovich BS. Variational analysis and generalized differentiation, I: basic theory, II: applications. Grundlehren series 	(fundamental principles of mathematical sciences). Vol. 330. Berlin: Springer; 2006.
[33]
Alefeld G,  Herzberger J. Introduction to interval computations. New-York (NY): Academic Press;  1983.
[34]
Moore RE. Interval analysis. Englewood Cliffs (NJ): Prentice-Hall; 1966.

[35]
Moore RE. Method and applications of interval analysis. Philadelphia (PA): SIAM; 1979.
[36]  Chuong TD, Huy NQ,   Yao JC.  Subdifferentials of marginal functions in semi-infinite programming. SIAM J Optim. 2009;20:1462–1477.

[37]  Chuong TD,  Kim DS.  Nonsmooth semi-infinite multiobjective optimization problems. J Optim
Theory Appl. 2014;160:748–762.

[38]
Jiao LG, Kim  DS, Zhou Y. Quasi $\epsilon $-solutions in a semi-infinite programming problem with locally Lipschitz data.  Optim Lett. 2021;15:1759–1772.
[39] Ioffe AD, Tikhomirov VM. Theory of extremal problems. Amsterdam: North-Holland; 1979.
[40]
Fakhar M, Mahyarinia  MR, Zafarani J. On approximate solutions for nonsmooth robust multiobjective optimization problems. Optimization.  2019. https://doi.org/10.1080/02331934.2019.1579212

[41] 	Rockafellar   RT. Convex analysis. Princeton (NJ): Princeton University Press; 1970.


# About exchanging expectation and supremum for conditional Wasserstein GANs

Jörg Martin joerg.martin(at)ptb.de, Physikalisch-Technische Bundesanstalt
(PTB), Abbestraße 2, 10587 Berlin, Germany.

###### Abstract

In cases where a Wasserstein GAN depends on a condition the latter is usually
handled via an expectation within the loss function. Depending on the way this
is motivated, the discriminator is either required to be Lipschitz-1 in both
or in only one of its arguments. For the weaker requirement to become usable
one needs to exchange a supremum and an expectation. This is a mathematically
perilous operation, which is, so far, only partially justified in the
literature. This short mathematical note intends to fill this gap and provides
the mathematical rationale for discriminators that are only partially
Lipschitz-1 for cases where this approach is more appropriate or successful.

Suppose our training data follows a distribution with the density $\pi(x,y)$
on $X\times Y$, where $X=\mathbb{R}^{n_{X}}$, $Y=\mathbb{R}^{n_{Y}}$. 111The
assumption that $X$ and $Y$ are finite-dimensional is mostly for the sake of
presentation, the argument can easily be extended to a Banach space setting as
in [1]. We assume that for each $y\in Y$ there is a probability distribution
$\mathcal{G}(y)$ on $X$ that can be sampled from via a measurable map
$G(z,y),z\sim\eta$, where $\eta$ denotes the distribution on an arbitrary
probability space. In terms of generative adversarial networks (GANs) the
mapping $G$ is typically known as the generator [6], is implemented as a
neural network and is used to approximate a target distribution. In contrast
to the original GAN framework, $\mathcal{G}$ and $G$ are here assumed to
depend on $y$, which is known as a conditional GAN (cGAN) [10].

We want $\mathcal{G}(y)$ to approximate the conditional distribution
$\pi(x|y)$:

$\displaystyle\mathcal{G}(y)\approx\pi(x|y)=\frac{\pi(x,y)}{\pi(y)}\,.$ (1)

If this approximation is done by minimizing the Wasserstein-1-distance , the
term (conditional) Wasserstein GAN ((c)WGAN is used. The concept of WGANs was
first proposed in the seminal works [4, 7]. A study of a conditional relation
(1) under the Wasserstein metric is done in various works in the literature
for different applications, compare e.g. [5, 9, 8, 12, 2, 3, 11]. In these
works there appear to be several philosophies on how to deal with the
$y$-dependency in (1). One natural approach is the following: Sample
$\tilde{y}\sim\pi(y)$ and then $\tilde{x}\sim\mathcal{G}(\tilde{y})$ and call
their distribution $\pi(\tilde{x},\tilde{y})$. Then, minimize the following
distance w.r.t $G$

$\displaystyle\mathcal{W}(\pi(\tilde{x},\tilde{y}),\pi(x,y))=\\!\\!\\!\\!\\!\sup_{D\in\mathrm{Lip}_{1}(X{\times}Y)}\\!\\!\left(\mathbb{E}_{x,y\sim\pi(x,y)}\left[D(x,y)\right]{-}\mathbb{E}_{z\sim\eta,y\sim\pi(y)}\left[D(G(z,y),y)\right]\right)\,,$
(2)

with $\mathrm{Lip}_{1}(X{\times}Y)$ denoting the set of real-valued
Lipschitz-1 functions on $X\times Y$ and where we used the Kantorovich-
Rubinstein duality. Note that this requires the discriminator $D$ to be
Lipschitz-1 in _both of its arguments_. While the conception above seems to
be, at least implicitly, behind many works on cWGANs in the literature
(compare e.g. [5, 9, 8, 12]), the Lipschitz constraint is often only enforced
w.r.t the first argument. One might attempt to motivate this with the fact
that the marginal $\pi(y)$ is identical for $\pi(\tilde{x},\tilde{y})$ and
$\pi(x,y)$, but to the best of the author’s knowledge there is no exhaustive
argument in the literature.

A different line of reasoning for a partial Lipschitz continuity of $D$, which
this note will follow, is given in [2, 3], where instead of (2) the objective

$\displaystyle\mathbb{E}_{y\sim\pi(y)}[\mathcal{W}(\mathcal{G}(y),\pi(x|y))]$
(3)

is considered, from which the authors obtain

$\displaystyle\sup_{D\in\mathcal{D}}\mathbb{E}_{(x,y)\sim\pi(x,y)}\left[D(x,y)-\mathbb{E}_{z\sim\eta}\left[D(G(z,y),y)\right]\right]\,,$
(4)

where $\mathcal{D}$ denotes the set of real-valued (and measurable) maps
defined on $X\times Y$ and Lipschitz-1 in $X$ \- and not necessarily in $Y$.
The objective (4) is in alignment with a discriminator that is Lipschitz-1 in
$x$ only. However, to get from (3) to (4) a supremum and an expectation have
to be exchanged which might cause problems concerning measurability. This
issue is not entirely resolved in [2, 3] and the authors need to assume that
there are no problems in this regard.

A workaround could be to assume that $\pi(y)$ is only supported on a discrete
subset, on which measurability is trivial. For cases where $y$ describes
continuous objects such as images or audio data this becomes a bit of a cheat,
as this support would not only have to include all of the training data, but
also any possible $y$ one might encounter during evalution of $\mathcal{G}(y)$
after the training. This would thus require some sort of further artificial
discretization (e.g. by machine epsilon). It seems however much more natural,
satisfactory and intuitive to allow for a continuous $\pi(y)$ in these cases.
The goal of this mathematical note is to demonstrate that this can be done and
that, under under relatively mild conditions, there is a consistent way to get
from (3) to (4). We thereby provide a mathematical rationale for using the
object (4) without any need of artifical discretization.

######  Assumption 1.

The marginal $\pi(y)$ has a compact support, say on $K_{Y}\subseteq Y$.

######  Assumption 2.

The mapping $y\mapsto\pi(x|y)$ is well-defined on $K_{Y}$ and continuous w.r.t
the Wasserstein metric.

######  Assumption 3.

The object $\mathbb{E}_{z\sim\eta}\left[|G(z,y)-G(z,y^{\prime})|\right]$
converges to $0$ for $y\rightarrow y^{\prime}$ uniformly for all
$y,y^{\prime}\in K_{Y}$.

The assumptions above could probably be weakened. However, they appear to be
not unreasonable. Assumption $3$, for instance, is true if $G$ is a neural
network with Lipschitz continuous activations and $\eta$ is a Gaussian
distribution or a distribution of bounded support. Assumption $1$ is often
true in practice, but can be dropped provided that the continuity in
Assumption 2 is uniform and the following lemma remains true.

######  Lemma 4.

The map $F:K_{Y}\ni y\mapsto\mathcal{W}(\mathcal{G}(y),\pi(x|y))$ is uniformly
continuous .

###### Proof.

As $K_{Y}$ is compact (Assumption 1), it is enough to show that $F$ is
continuous. Take $y,y^{\prime}\in K_{y}$ and note that

$\displaystyle F(y)$
$\displaystyle=\sup_{f\in\mathrm{Lip}_{1}(X)}\left(\mathbb{E}_{x\sim\pi(x|y)}\left[f(x)\right]-\mathbb{E}_{z\sim\eta}\left[f(G(z,y)\right]\right)$
$\displaystyle\leq\mathcal{W}(\pi(x|y),\pi(x|y^{\prime}))+\mathbb{E}_{z\sim\eta}\left[|G(z,y)-G(z,y^{\prime})|\right]+F(y^{\prime})\,.$

By symmetry we can exchange the roles of $y$ and $y^{\prime}$ and arrive at
$|F(y)-F(y^{\prime})|\leq\mathcal{W}(\pi(x|y),\pi(x|y^{\prime}))+\mathbb{E}_{z\sim\eta}\left[|G(z,y)-G(z,y^{\prime})|\right]$,
which converges to 0 as $y\rightarrow y^{\prime}$ due to Assumption 2 and 3. ∎

Assumptions 1-3 were basically chosen to make Lemma 4 true as it is the key to
proving the main statement of this note, which will be our next task. In
essence, Lemma 4 allows us to reduce the average over $y\sim\pi(y)$ in (3) to
a finite sum up to an arbitrarily small error. On finite sets measurability is
no issue, so that we can safely exchange the supremum and the expectation.

######  Theorem 5.

Under the Assumptions 1-3 we have

$\displaystyle\mathbb{E}_{y\sim\pi(y)}\left[\mathcal{W}(\mathcal{G}(y),\pi(x|y)\right]=\sup_{D\in\mathcal{D}}\text{$\mathbb{E}$}_{(x,y)\sim\pi(x,y)}\left[D(x,y)-\mathbb{E}_{z\sim\eta}\left[D(G(z,y),y)\right]\right]\,.$

###### Proof.

The inequality

$\displaystyle\mathbb{E}_{y\sim\pi(y)}[F(y)]$
$\displaystyle=\mathbb{E}_{y\sim\pi(y)}\left[\mathcal{W}(\mathcal{G}(y),\pi(x|y))\right]$
$\displaystyle\geq\sup_{D\in\mathcal{D}}\text{$\mathbb{E}$}_{(x,y)\sim\pi(x,y)}\left[D(x,y)-\mathbb{E}_{z\sim\eta}\left[D(G(z,y),y)\right]\right]$

is simply a consequence of the Kantorovich-Rubinstein duality, cf. [2]. We can
therefore prove the claim once we have shown that for any $\varepsilon>0$

$\displaystyle\mathbb{E}_{y\sim\pi(y)}[F(y)]\leq\sup_{D\in\mathcal{D}}\text{$\mathbb{E}$}_{(x,y)\sim\pi(x,y)}\left[D(x,y)-\mathbb{E}_{z\sim\eta}\left[D(G(z,y),y)\right]\right]+\varepsilon\,.$
(5)

Using the Lemma above and Assumption 2 and 3 we can pick
$\delta_{\varepsilon}>0$ such that
$|F(y)-F(y^{\prime})|\leq\frac{\varepsilon}{4}$,
$\mathcal{W}(\pi(x|y),\pi(x|y^{\prime}))\leq\frac{\varepsilon}{4}$ and
$\mathbb{E}_{z\sim\eta}\left[|G(z,y)-G(z,y^{\prime})|\right]\leq\frac{\varepsilon}{4}$
whenever $|y-y^{\prime}|<\delta_{\varepsilon}$ for any $y,y^{\prime}\in
K_{Y}$. The compact set $K_{Y}$ can be covered with a finite numbers of
disjoint boxes $C_{k}^{\varepsilon}$ of diameter smaller than
$\delta_{\varepsilon}$. For each $k$ fix some $y_{k}^{\varepsilon}\in
C_{k}^{\varepsilon}\cap K_{Y}$. Writing
$F^{\varepsilon}(y)=\sum_{k}F(y_{k}^{\varepsilon})1_{C_{k}^{\varepsilon}}(y)$
with $F$ as above and with $1_{C_{{}^{k}}^{\varepsilon}}(y)$ equal to 1 if
$y\in C_{k}^{\varepsilon}$ and else 0, we have

$\displaystyle\mathbb{E}_{y\sim\pi(y)}\left[F(y)\right]$
$\displaystyle=\mathbb{E}_{y\sim\pi(y)}\left[F(y)-F^{\varepsilon}(y)\right]+\mathbb{E}_{y\sim\pi(y)}\left[F^{\varepsilon}(y)\right]$
$\displaystyle\leq\frac{\varepsilon}{4}+\mathbb{E}_{y\sim\pi(y)}\left[F^{\varepsilon}(y)\right]\,.$
(6)

Using once more the Kantorovich-Rubinstein duality we can now pick for every
$k$ a $f_{k}^{\varepsilon}\in\mathrm{Lip}_{1}(X)$ such that

$\displaystyle
F(y_{k}^{\varepsilon})\leq\frac{\varepsilon}{4}+\mathbb{E}_{x\sim\pi(x|y_{k}^{\varepsilon})}\left[f_{k}^{\varepsilon}(x)\right]-\mathbb{E}_{z\sim\eta}\left[f_{k}^{\varepsilon}(G(z,y_{k}^{\varepsilon})\right]\,.$
(7)

Given $y\in C_{k}^{\varepsilon}\cap K_{Y}$ we can use
$\mathbb{E}_{z\sim\eta}\left[|G(z,y)-G(z,y_{k}^{\varepsilon})|\right]\leq\frac{\varepsilon}{4}$
by choice of $\delta_{\varepsilon}$ and bound (7) further by

$\displaystyle F(y_{k}^{\varepsilon})\leq
2\cdot\frac{\varepsilon}{4}+\mathbb{E}_{x\sim\pi(x|y_{k}^{\varepsilon})}\left[f_{k}^{\varepsilon}(x)\right]-\mathbb{E}_{z\sim\eta}\left[f_{k}^{\varepsilon}(G(z,y))\right]\,.$
(8)

Combining (6) and (8) we end up with

$\displaystyle\mathbb{E}_{y\sim\pi(y)}\left[F(y)\right]$
$\displaystyle\leq\frac{3\varepsilon}{4}+\mathbb{E}_{y\sim\pi(y)}\left[\sum_{k}1_{C_{k}^{\varepsilon}}(y)\left(\mathbb{E}_{x\sim\pi(x|y_{k}^{\varepsilon})}\left[f_{k}^{\varepsilon}(x)\right]-\mathbb{E}_{z\sim\eta}\left[f_{k}^{\varepsilon}(G(z,y))\right]\right)\right]$
$\displaystyle\leq\varepsilon+\mathbb{E}_{y\sim\pi(y)}\left[\sum_{k}1_{C_{k}^{\varepsilon}}(y)\left(\mathbb{E}_{x\sim\pi(x|y)}\left[f_{k}^{\varepsilon}(x)\right]-\mathbb{E}_{z\sim\eta}\left[f_{k}^{\varepsilon}(G(z,y))\right]\right)\right]\,,$
(9)

where we used in the last step that
$\mathcal{W}(\pi(x|y),\pi(x|y_{k}^{\varepsilon}))\leq\frac{\varepsilon}{4}$.
Writing now $D^{\varepsilon}(x,y)=\sum_{k}1_{C_{k}^{\varepsilon}}(y)\cdot
f_{k}^{\varepsilon}(x)$ we have apparently $D^{\varepsilon}\in\mathcal{D}$
and, moreover,

$\displaystyle\text{$\mathbb{E}$}_{(x,y)\sim\pi(x,y)}\left[D^{\varepsilon}(x,y)-\mathbb{E}_{z\sim\eta}\left[D^{\varepsilon}(G(z,y),y)\right]\right]$
$\displaystyle=\mathbb{E}_{y\sim\pi(y)}\left[\sum_{k}1_{C_{k}^{\varepsilon}}(y)\left(\mathbb{E}_{x\sim\pi(x|y)}\left[f_{k}^{\varepsilon}(x)\right]-\mathbb{E}_{z\sim\eta}\left[f_{k}^{\varepsilon}(G(z,y))\right]\right)\right]\,,$

which is just the second term on the right hand side of (9); this yields

$\displaystyle\mathbb{E}_{y\sim\pi(y)}\left[F(y)\right]$
$\displaystyle\leq\varepsilon+\text{$\mathbb{E}$}_{(x,y)\sim\pi(x,y)}\left[D^{\varepsilon}(x,y)-\mathbb{E}_{z\sim\eta}\left[D^{\varepsilon}(G(z,y),y)\right]\right]$
$\displaystyle\leq\varepsilon+\sup_{D\in\mathcal{D}}\text{$\mathbb{E}$}_{(x,y)\sim\pi(x,y)}\left[D(x,y)-\mathbb{E}_{z\sim\eta}\left[D(G(z,y),y)\right]\right]\,,$

which shows (5). ∎

## References

  * [1] Adler, J., and Lunz, S. Banach wasserstein GAN.  In NeurIPS (2018), pp. 6755–6764.
  * [2] Adler, J., and Öktem, O. Deep bayesian inversion.  arXiv preprint arXiv:1811.05910, 2018.
  * [3] Adler, J., and Öktem, O. Deep posterior sampling: Uncertainty quantification for large scale inverse problems.  In International Conference on Medical Imaging with Deep Learning (2019).
  * [4] Arjovsky, M., Chintala, S., and Bottou, L. Wasserstein GAN.  arXiv e-prints, Jan. 2017.
  * [5] Ebenezer, J. P., Das, B., and Mukhopadhyay, S. Single image haze removal using conditional wasserstein generative adversarial networks.  In 2019 27th European Signal Processing Conference (EUSIPCO) (2019), IEEE, pp. 1–5.
  * [6] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial networks.  Commun. ACM 63, 11 (Oct. 2020), 139144.
  * [7] Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A. C. Improved training of wasserstein GANs.  In NeurIPS (2017), I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett, Eds., pp. 5767–5777.
  * [8] Jin, Q., Luo, X., Shi, Y., and Kita, K. Image generation method based on improved condition GAN.  In 2019 6th International Conference on Systems and Informatics (ICSAI) (2019), IEEE, pp. 1290–1294.
  * [9] Luo, Y., and Lu, B.-L. Eeg data augmentation for emotion recognition using a conditional wasserstein GAN.  In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (2018), IEEE, pp. 2535–2538.
  * [10] Mirza, M., and Osindero, S. Conditional generative adversarial nets.  arXiv preprint arXiv:1411.1784, 2014.
  * [11] Ohayon, G., Adrai, T., Vaksman, G., Elad, M., and Milanfar, P. High perceptual quality image denoising with a posterior sampling cgan.  arXiv preprint arXiv:2103.04192 (2021).
  * [12] Patzelt, F., Haschke, R., and Ritter, H. J. Conditional WGAN for grasp generation.  In ESANN (2019).