the maximum value of correlation between $x_1$ and $x_2$ under rotational transformation. The entropy of the two dimensional binary code is then given as $$\begin{aligned}
S(\gamma, \theta) = 2 \times (- p_{(1,1)} \log p_{(1,1)} - p_{(-1,1)} \log p_{(-1,1)}).\end{aligned}$$

Fig. \[fig:qerr\_entropy\] is plots of quantization error and entropy with respect to angle $\theta$. When quantization error is minimized, entropy is also minimized and vice versa. This means that the quantization error and the entropy have a trade-off relationship. Compatibility of the two factors depends on the “sharpness” of the distribution. When the distribution is sharp ($\lambda_1 \gg \lambda_2$), entropy is heavily damaged with isotropic variances. There is no trade-off relationship if the distribution is circular ($\lambda_1 = \lambda_2$). But in general case, we should balance these criteria.

An analysis that is similar to ours is recently proposed in [@Ge2013]. In the paper, the quantization error for Product Quantization was discussed. The authors showed that it is bounded by determinant of variance-covariance matrix, and proposed an algorithm that is minimizing the bound under rotational transformation. The result indicates trade-off relationship between quantization error and entropy of the gaussian distribution. However, since the entropy of the gaussian distribution is invariant under rotational transformation, 