= \lim_{n \to \infty} \frac{H(Z_1^n)}{\log n}$$ when the limit exists. With conditioning on another random variable $\Theta$, define the log-blocklength normalized conditional entropy rate as $$\mathfrak{H}(\mathfrak{Z} \mid \Theta) = \lim_{n \to \infty} \frac{H(Z_1^n \mid \Theta)}{\log n}$$ when the limit exists. Similarly, define the log-blocklength normalized information rate as $$\mathfrak{I}(\mathfrak{Z};\Theta) = \lim_{n \to \infty} \frac{I(Z_1^n;\Theta)}{\log n}$$ when the limit exists. While these definitions parallel the standard definitions, none of the limits would generally exist for sequence coding because the numerators grow linearly with $n$.

For each $n \in \mathbb{Z}^+$, let $\phi_n$ be a source code for random variable $Z_1^n$. For this sequence of source codes, the average codeword lengths are $$C_{\phi,n} = \sum_{\mathcal{Z}_n} p_{Z_1^n}(z_1^n) \ell(z_1^n) \mbox{,}$$ where $\ell(\cdot)$ is the length of the codeword assigned to the source realization $z_1^n$. Shannon’s fixed-to-variable source coding theorem [@Shannon1948] establishes that there exists a sequence of source codes that satisfy the following inequalities for all $n$: $$H(Z_1^n) \le C_{\phi,n} \le H(Z_1^n) + 1 \mbox{.}$$ Dividing through by $\log n$ yields $$\frac{H(Z_1^n)}{\log n} \le \frac{C_{\phi,n}}{\log n} \le \frac{H(Z_1^n) + 1}{\log n} \mbox{.}$$ Taking the limit of large blocklength ($n \rightarrow \infty$), we see that when the limits exist there is a sequence of source codes 