scheme that operates on a block-level and that is based on a heuristic: *Each embedding should be assigned a dimension proportional to some fractional power of its popularity*. Note that here we distinguish *block-level probability* $\mathbf{p}$ from *row-wise frequency* $\mathbf{f}$. Given $\mathbf{f}$, we define $a_i = \sum_{j=t_{i}}^{t_{i+1}} f_j$ as the area under the frequency curve in the interval $[t_i,t_{i+1}]$ and total $\tau = \sum_{j=0}^n f_j$. Then, we let block-level probability vector $\mathbf{p} \in \mathbb{R}^{k+1}$ be defined by its elements $p_i = a_i / \tau $. We formalize the popularity-based scheme in Alg. \[alg:alpha\], with an extra hyperparameter temperature $\alpha > 0$.

**Input:** Desired baseline dimension $\bar{d}$ and fixed temperature $0 \leq \alpha \leq 1$ **Input:** Probability vector $\textbf{p}$ **Output:** Dimension assignment vector $\textbf{d}$ $\lambda \gets \bar{d} p_0^{-\alpha}$ $\textbf{d} \gets \lambda \mathbf{p}^\alpha$

\[alg:alpha\]

The proposed technique requires knowledge of probability vector $\textbf{p}$ that governs the feature popularity. When such distribution is unknown, we may easily replace it by the empirical distributions from the data sample.[^3] Alternatively, we can set $\lambda \gets B(\sum_i p_i^{\alpha-1})^{-1}$ to approximately[^4] constrain the embedding layer sizing to a total (number of parameters) budget $B$. Furthermore, to get crisp sizing, one might also elect to round $d$, perhaps 