where the objective function is $\|\cdot\|_2$ and the constraint is solutions of a stochastic variational inequality problem. Recently, a primal-dual algorithm in [@Rosasco18] was offered that uses the idea of iterative regularization. Despite [[a sublinear rate in terms of $h$]{}]{}, the algorithm can only be applied to continuously differentiable [[and small-scale (i.e., $m=1$)]{}]{} regimes. [**(iii) Minimal norm gradient:**]{} In [@Beck14], the *minimal norm gradient* (MNG) method was developed for solving [[problem with $m=1$]{}]{}. The rate of $\mathcal{\cal O} (1/\sqrt{k})$ was derived for the convergence with respect to lower level problem. The main disadvantage of the MNG method is that it is a two-loop scheme where at each iteration a minimization problem should be solved.

[**(iv) Sequential averaging:**]{} The *sequential averaging method* (SAM), developed in [@Xu04], was [[employed]{}]{} in [@Sabach17] for solving the problem in a more general setting. The proposed method is proved to have the rate of convergence of $\mathcal{\cal O} (1/k)$ in terms of the function $f$. The method is called *Big-SAM*. [[Despite that it is a single-loop scheme, sequential averaging schemes require smoothness properties of the problem and seem not to lend themselves to distributed implementations.]{}]{}

Main Contributions
------------------

[[For more details on the main distinctions between 