![Histogram of accesses for different embedding vectors[]{data-label="fig:hist_datasets"}](images/hist_criteo.png "fig:"){width="\textwidth"}

We propose mixed dimension embedding layers, a novel architecture for embedding representations. The central thesis behind mixed dimension embedding layers is that the dimension of a particular embedding vector should not remain uniform, but should scale with that feature’s popularity. In particular, we show that popular embedding features are often not allotted sufficient parameters, whereas infrequent embeddings waste them. By architecting mixed dimension embedding layers such that the dimension of an embedding vector scales with its popularity, we can make significant improvements to statistical performance at small parameter budgets.

In order to illustrate that the proposed mixed dimension embedding layers greatly improve parameter-efficiency we show that mixed dimension layers achieve slightly lower loss, while using 8$\times$ fewer parameters compared to uniform dimension embeddings on the Movielens dataset [@Harper2015]. Also, we show that mixed dimension embeddings improve accuracy by 0.1% using half as many parameters and maintain the baseline accuracy using 16$\times$ fewer parameters on the Criteo Kaggle dataset [@Criteo2014].

We point out that even though we focus on recommendation and and event-probability prediction problems, our methods are applicable to representation learning in general. We state our main contributions next.

#### Summary of 