state-of-the-art applications in diverse domains, including computer vision [@barz2019hierarchy; @vasileva2018learning], natural language processing [@chiu2016train; @Mikolov2013distributed; @liu2015topical; @shoeybi2019megatron; @akbik2018contextual; @peters2018dissecting; @liu2019roberta], computational biology [@Asgari2015], and recommendation systems [@cheng2016wide; @Park2018; @Wu2019].

There seems to be a fundamental trade-off between the dimension of an embedding representation and the statistical performance (i.e. accuracy or loss) of embedding-based models on a particular learning task. It is well-documented that statistical performance suffers when embedding dimension is too low [@yin2018dimensionality]. Thus, we are interested in the fundamental question: *Is it possible to re-architect embedding representations for a more favorable trade-off between number of parameters and statistical performance?*

This challenge is particularly prominent in recommendation systems tasks such as collaborative filtering (CF) and click-through rate (CTR) prediction problems. Recommendation models power some of the most commonplace data-driven services on the internet that benefit users across the globe with personalized experiences on a daily basis [@Aggarwal2016; @hazelwood2018; @he2014practical; @Pi2019].

At present, the out-sized memory consumption of standard embedding layers in these recommendation systems is a major burden on the memory hierarchy – an embedding layer for a single recommendation engine can consume tens of gigabytes of space [@Park2018; @Pi2019]. This makes the engineering challenges associated with large-scale recommendation embeddings 