designed the proposed solution to meet this condition (in contrast to an end-to-end approach). Let us define $\mathcal{B}_{1}$, $\mathcal{B}_{2}$, and $\mathcal{B}_{3}$ three datasets containing ground truth depth information, ground truth identity for rgb facial images, and ground truth identity for depth facial images, respectively. More precisely, $\mathcal{B}_{1}=\{(I_i(x,y), {I_0}_i(x,y), z_i(x,y)),\ i=1,\,...,\,n_1\}$, $\mathcal{B}_{2}=\{({I_0}_i(x,y), y_i),\ i=1,\,...,\,n_2\}$, and $\mathcal{B}_{3}=\{(z_i(x,y), y_i),\ i=1,\,...,\,n_3\}$, where $I_i(x,y)$ denotes a (facial or generic) RGB image acquired under the projection of the designed pattern, ${I_0}_i(x,y)$ represents (facial or generic) standard RGB images, $z_i(x,y)$ denotes a gray image representing the depth of the scene, and $y_i$ a scalar integer representing the subject id. We denote as $\{g_1(I), g_2(I)\} = g(I)$ the RGB and gradient depth components estimated by the decomposition operation $g$. We partitioned the parameters of $g$ into two sets of dedicated kernels $\theta_g = \{{\theta_g}_1, {\theta_g}_2\}$, the first group focuses on retrieving the texture component while the second group retrieves the depth gradient. These parameters can be optimized as $$\begin{aligned}
\label{eq:loss_g}
    {\theta_g}_1 = \operatorname*{argmin}\sum_{({I_0}_i,I_i)\in\mathcal{B}_1} \|g_1(I_i) - {I_0}_i \|_2^2 \\
    {\theta_g}_2 = \operatorname*{argmin}\sum_{({z}_i,I_i)\in\mathcal{B}_1} \|g_2(I_i) - \nabla z_i \|_2^2.\end{aligned}$$ (We also evaluated training a shared set of kernels trained with an unified loss, this alternative is harder to train in practice, 