to the first/second coordinate.)

![image](figures/framework.png){width=".9\textwidth"}

#### Combining depth and RGB information.

The proposed approach consists of three main modules as illustrated in Figure \[fig:proposedFramework\]: $g:{\mathcal{I}}_{rgb}\rightarrow {\mathcal{I}}_{rgb}\times{\mathcal{I}}_{\nabla z}$ performs a decomposition of the input image into texture and depth information, $f_{rgb}:{\mathcal{I}}_{rgb}\rightarrow {\mathcal{X}}_{n/2}$, and $f_{\nabla z}:{\mathcal{I}}_{\nabla z}\rightarrow {\mathcal{X}}_{n/2}$ extract facial features associated to the facial texture and depth respectively. These three components are illustrated in Figure \[fig:proposedFramework\] in blue, yellow, and green, respectively. (We decided to have three modules instead of a single *end-to-end* design for several reasons that will be discussed below.) We denote the facial feature extraction from the input image as $f_{\theta} : {\mathcal{I}}_{rgb} \rightarrow {\mathcal{X}}_n$, where $f_\theta(I) = (f_{rgb}(I_{rgb}), f_{\nabla z}(I_{\nabla z}))^T$ with $\{I_{rgb},I_{\nabla z}\} = g(I)$. The subscript $\theta$ represent the parameters of the mapping $f$, which can be decomposed in three groups $\theta = (\theta_g, \theta_{rgb}, \theta_{\nabla z})$, associated to the image decomposition, RGB feature extraction, and depth feature extraction respectively. In the following we discuss how these parameters are optimized for each specific task, which is one of the advantages of formulating the problem in a modular fashion. Once texture and depth facial information is extracted into a suitable vector representation $x = f_{\theta}(I)$ (as 