of particular importance and interest.

In many models, embedding layers are used to map input categorical values into a vector space. This feature mapping outputs a $d$-dimensional vector for each value, and is learned concurrently with the model during supervised training. However, the relationship between data distributions, embedding representations, model architectures, and optimization algorithms is highly complex. This makes it difficult to design efficient quantization and compression algorithms for embedding representations from information-theoretic principles.

Nevertheless, the distributions of accesses for users and items is often heavily skewed in many real-world applications. For instance, for the CF task on MovieLens dataset[^1], the top 10% of users receive as many queries as the remaining 90% and the top 1% of items receive as many queries as the remaining 99%. To an even greater extent, for the CTR prediction task on the Criteo AI Labs Ad Kaggle dataset[^2] the top $0.0003\%$ of indices receive as many queries as the remaining 30 million, as summarized on Fig. \[fig:hist\_datasets\]. In networks science, this phenomena is referred to as *popularity* [@cho2012wave; @papadopoulos2012popularity], a terminology we adopt here.

[0.32]{} ![Histogram of accesses for different embedding vectors[]{data-label="fig:hist_datasets"}](images/hist_ml_items.png "fig:"){width="\textwidth"}

[0.32]{} ![Histogram of accesses for different embedding vectors[]{data-label="fig:hist_datasets"}](images/hist_ml_user.png "fig:"){width="\textwidth"}

[0.32]{} 