be swapped, depending on the application.

#### Optimality Conditions.

The first-order optimality conditions for our pair of vectors ${\bm{w}}\in {\mathbb{R}}^d, {\bm{x}}\in {\mathbb{R}}^n$ in problems  and  are given as

$$\begin{aligned}
 {\bm{w}}\in&\ \partial f(A{\bm{x}}) \label{eq:opt_f} \ ,\\ 
A{\bm{x}}\in&\ \partial f^*({\bm{w}}) \label{eq:opt_fstar} \ ,\end{aligned}$$

$$\begin{aligned}
 -A^\top {\bm{w}}\in&\ \partial g({\bm{x}}) \label{eq:opt_g} \ ,\\ 
 {\bm{x}}\in&\ \partial g^*(-A^\top{\bm{w}}) \!\!\!\!\!\!&\label{eq:opt_gstar}\end{aligned}$$

see e.g. [@Bauschke:2011ik Proposition 19.18]. The stated optimality conditions are equivalent to ${\bm{x}},{\bm{w}}$ being a saddle-point of the Lagrangian, which is given as $\mathcal{L}({\bm{x}},{\bm{w}}) =f^*({\bm{w}}) - \langle A{\bm{x}},{\bm{w}}\rangle - g({\bm{x}}) $ if ${\bm{x}}\in \operatorname{dom}(g)$ and ${\bm{w}}\in \operatorname{dom}(f^*)$, see Appendix \[app:primaldual\] for details.

#### The Constrained Case.

Any constrained convex optimization problem of the form $$\label{eq:constrainedOpt}
\min_{{\bm{x}}\in{\mathcal{C}}} f(A{\bm{x}})$$ for a constraint set ${\mathcal{C}}$ can be directly written in the form by using the indicator function of the constraint set as the penalization term $g$. (The indicator function ${\bm{\iota}}_{{\mathcal{C}}}$ of a set ${\mathcal{C}}\subset {\mathbb{R}}^n$ is defined as ${\bm{\iota}}_{{\mathcal{C}}}({\bm{x}}) := 0$ if ${\bm{x}}\in{\mathcal{C}}$ and ${\bm{\iota}}_{{\mathcal{C}}}({\bm{x}}) := +\infty$ otherwise.)

#### The Partially Separable Case.

A very important special case arises when one part of the objective becomes separable. Formally, this is expressed as $g({\bm{x}}) = \sum_{i=1}^{n} g_i(x_i)$ for univariate functions $g_i:{\mathbb{R}}\rightarrow{\mathbb{R}}$ for $i\in[n]$. Nicely in this case, the conjugate of $g$ 