INC\](b), $u_t=\|x_t-x_h^*\|$ converges to zero. Therefore, for $\hat u=0$ we can apply Lemma \[lemma thm for avr\] and thus, $\|\bar x_{k+1}-x_h^*\|$ converges to zero.

Rate analysis {#sec:rate result}
=============

In this section, we first find an error bound with respect to the optimal values of the lower level function $f$ which indeed shows the feasibility of [[the problem]{}]{} . Then, we apply it to derive a convergence rate for the algorithm.

\[lemma averaging\] Consider the sequence $\{\bar x_N\}$ generated by Algorithm \[algorithm:IRIG\]. Let Assumption \[assum:properties\] hold and $\{\gamma_k\}$ and $\{\lambda_k\}$ be positive and non-increasing sequences. Then, for all $N\geq 1$ and $z\in X$ we have $$\begin{aligned}
    &f(\bar x_N)-f^*\leq \left(\sum_{k=0}^{N-1} \gamma_k^r\right)^{-1} \left( m \sum_{k=0}^{N-1} \gamma_k^{r+1} (C_f^2 + \lambda_k^2 C_h^2) \right. \cr\\& \left. 
    +m^2 C_f \sum_{k=0}^{N-1} \gamma_k^{r+1} \left( C_f + \lambda_k C_h \right) + 2 M_h  \sum_{k=0}^{N-1} \gamma_k^r \lambda_k + 2M^2\gamma_{N-1}^{r-1} \right),
    \end{aligned}$$ where $M_h, M$ are scalars such that $\|h(x)\|\leq M_h$, $\|x\| \leq M$ for all $x \in X$.

[[Similar to]{}]{} relation , we [[can have]{}]{} $$\begin{aligned}
    &\left \|x_{k+1} - x^* \right \|^2  \leq \left \|x_k - x^* \right \|^2 + 2 m \gamma_k^2 (C_f^2 + \lambda_k^2 C_h^2) \\
    & - 2\gamma_k \sum_{i=0}^{m-1} \left( f_{i+1}(x_{k,i}) + \frac{\lambda_k}{m} h(x_{k,i}) \right) + 2\gamma_k 