Contributions

1.  We propose mixed dimension embedding layers, where the dimension of a particular embedding vector scales with said vector’s popularity.

2.  We provide a simple heuristic scheme for sizing of embedding vectors given a prior distribution or training sample. The dimensions prescribed by our scheme compare favorably to uniform dimensions.

3.  We identify two distinct mechanisms by which mixed dimension layers improve parameter-efficiency and achieve superior statistical performance at a given parameter budget.

Background & Related Works
==========================

In this work, we focus on explicit CF as well as CTR prediction. In explicit CF user ratings for particular items are directly observed and therefore it can be formally framed as a matrix completion problem [@candes2010matrix; @candes2009exact; @candes2010power; @hastie2015matrix]. Embedding-based approaches, such as matrix factorization (MF) [@hastie2015matrix; @Koren2009; @Rendle2019] or neural collaborative filtering (NCF) [@Dacrema2019; @he2017neural] are among the most popular and efficient solutions to matrix completion. The main alternative is to use a convex relaxation to find the minimum nuclear norm solution. This entails solving a semi-definite program, which takes time $O(n^4)$ [@zheng2012efficient], and thus will not scale to real-world applications. Instead, embedding/factorization approaches have the drawback of explicitly requiring an embedding dimension, but in practice this can be 