(categorical) and floating point values (continuous).

#### Deep Learning Recommendation Model

Facebook’s state-of-the-art deep learning recommendation model (DLRM) [@naumov2019deep] allows for $\kappa+1$ categorical features, which can represent arbitrary details about the context of an on-click or personalization event. The $i$-th categorical feature can be represented by an index $x_i \in \{0,...,n_i-1\}$ for $i=0,...,\kappa$. In addition to $\kappa+1$ categorical features, we also have $s$ scalar features, together producing a dense feature vector $\textbf{x}' \in \mathbb{R}^s$. Thus, given some $(x_0,...,x_{\kappa},\textbf{x}') \in ([n_0] \times ... \times [n_{\kappa}]) \times \mathbb{R}^s$, we would like to predict $y \in \{0,1\}$ which denotes an on-click event in response to a particular personalized context $(x_0,...,x_{\kappa},\textbf{x}')$.

In DLRM we define $\mathcal{A}$ as solving the following optimization problem

$$\min_{E^{(0)}, .., E^{(\kappa)}, \theta} \sum (G(\textbf{x}'; \textbf{e}^{(0)}_{x_0},..,\textbf{e}^{(\kappa)}_{x_\kappa},\theta) - y)^2$$

where $E^{(i)}=[\textbf{e}^{(i)}_{0}, ..., \textbf{e}^{(i)}_{n_i-1}]$ for $i=0,...,\kappa$ are embeddings, while $\theta$ denotes any additional parameters (mostly related to MLPs) in the model $G$. Note that we often use binary cross-entropy instead of MSE loss in practice.

Mixed Dimension Embedding Layer
===============================

Let us now define the mixed dimension embedding layer and describe the equipartition- as well as popularity-based schemes used to determine its dimensionality. We will also discuss how we can apply it to 