to the nearest power of two, after applying Alg. \[alg:alpha\].

Notice that we finally have all the tools to size the mixed dimension embedding layers as well as to perform forward and backward propagation through them, using Alg. \[alg:emb\_layer\] - \[alg:alpha\].

Experiments
===========

We proceed to present our experiments validating mixed dimension embedding layers. All implementations described herein are done in the Pytorch framework [@paszke2017automatic] using NVIDIA V100 32GB GPUs [@NvidiaVolta100]. In total, we run experiments on two datasets with corresponding models summarized in Table \[table:dataset\]. We note that we always use the MLP variation of the NCF model.

  --------------------- ------- ------------ --------------- -------------
         Dataset         Tasks   \# Samples   \# Categories   Models Used
   \[0.5ex\] MovieLens    CF        27M           300K          MF, NCF
      Criteo Kaggle       CTR       40M            32M           DLRM
  --------------------- ------- ------------ --------------- -------------

  : Datasets, tasks and models[]{data-label="table:dataset"}

[0.49]{} ![Embedding dimensions allocated with different choices of $\alpha$ for the same budget.[]{data-label="fig:fixed_budget_alpha_assignments"}](images/embedding_dim_sorted_pop_movielens.png "fig:"){width="\textwidth"}

[0.49]{} ![Embedding dimensions allocated with different choices of $\alpha$ for the same budget.[]{data-label="fig:fixed_budget_alpha_assignments"}](images/ctr_emb_pop_rounded.png "fig:"){width="\textwidth"}

We explore the trade-off between memory and statistical performance for mixed dimension embeddings, parameterized by varying $\alpha$. For a given $\alpha$, we vary the budget by varying the base dimension, $\bar{d}$. In our setting, we measure memory 