text: user4962: I'm confused by the computation of sequence logo. Wikipedia gives a process about this without a concrete example.
  Let's just consider DNA, so there are 4 bases (nucleic acids).
  The following data comes from the book "Machine Learning - A Probabilistic Perspective (Figure 1)"
  
  The Shannon entropy of position $i$ is:
  ${\displaystyle H_{i}=-\sum _{b=a}^{t}f_{b,i}\times \log _{2}f_{b,i}}$
  Where ${\displaystyle f_{b,i}}$ is the relative frequency of base
  This post is computing position 3, where it seems that the relative frequency of a is 100%, and the relative frequency of 3 other bases are 0%.
  Now, how do you calculate the Shannon entropy of position 3? It seems to be 0, which is apparently incorrect, where am I wrong?
  
  user4645: Why do you think the entropy of 0 is incorrect? It intuitively makes sense, as there is no uncertainty about the base at position 3, and thus there is no entropy.
  However, what is plotted in a sequence logo isn't the entropy, but rather a measure of the "decrease in uncertainty" as the sequence is aligned. This is calculated by taking the entropy at this position if we randomly aligned sequences ($H_g(L)$), and subtracting from it the entropy of the alignment ($H_g(s)$): 
  $$
  R_{sequence}=H_g(L) - H_s(L)
  $$
  The entropy at position 3 based on a random alignment is calculated by assuming there are 4 equally likely events (one for each base), and thus:
  $$
  \begin{align*} 
  H_g(L) & = -((1/4 \times -2) + (1/4 \times -2) + (1/4 \times -2) + (1/4 \times -2)) \\
  H_g(L) & = 2
  \end{align*}
  $$
  Notably, $H_g(L)$ will always be 2 when dealing with nucleotide sequences.
  So in your example, we have:
  $$
  \begin{align*} 
  R_{sequence}&=H_g(L) - H_s(L) \\
  R_{sequence}&=2 - 0 \\
  R_{sequence}&=2 \text{ bits of entropy}
  \end{align*}
  $$
  Finally, to work out the height of each base at each position in the logo, we multiply the frequency of that base by the overall information gain $R_{sequence}$. In this case the frequency of the base A at position 3 is of course 1:
  $$
  \begin {align*}
  H(b,l) &=f(b, l) \times R_{sequence} \\
  H(A, 3) &= f(a, 3) \times 2 \\
  &= 1 \times 2 \\
  &= 2
  \end{align*}
  $$
  Source: Sequence logos: a new way to display consensus sequences.
   (Thomas D. Schneider, R. Michael Stephens, 1990)
  Note: I've excluded the correction factor $e(n)$ from the overall information-gain calculation for the sake of simplicity. Refer to the paper above for an explanation of this.
  
original_text: 4962: <p>I'm confused by the <a href="https://bioinformatics.stackexchange.com/q/9091/4962">computation</a> of sequence logo. <a href="https://en.wikipedia.org/wiki/Sequence_logo#Logo_creation" rel="nofollow noreferrer">Wikipedia</a> gives a process about this without a concrete example.</p>
  
  <p>Let's just consider DNA, so there are 4 bases (nucleic acids).</p>
  
  <p>The following data comes from the book "Machine Learning - A Probabilistic Perspective (Figure 1)"</p>
  
  <p><a href="https://i.stack.imgur.com/XP7q1.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/XP7q1.png" alt="enter image description here"></a></p>
  
  <p>The Shannon entropy of position <span class="math-container">$i$</span> is:</p>
  
  <p><span class="math-container">${\displaystyle H_{i}=-\sum _{b=a}^{t}f_{b,i}\times \log _{2}f_{b,i}}$</span></p>
  
  <p>Where <span class="math-container">${\displaystyle f_{b,i}}$</span> is the relative frequency of base</p>
  
  <p>This <a href="https://bioinformatics.stackexchange.com/a/9094/4962">post</a> is computing position 3, where it seems that the relative frequency of <code>a</code> is 100%, and the relative frequency of 3 other bases are 0%.</p>
  
  <p>Now, how do you calculate the Shannon entropy of position 3? It seems to be 0, which is apparently incorrect, where am I wrong?</p>
   4645: <p>Why do you think the entropy of 0 is incorrect? It intuitively makes sense, as there is no uncertainty about the base at position 3, and thus there is no entropy.</p>
  
  <p>However, what is plotted in a sequence logo isn't the entropy, but rather a measure of the "decrease in uncertainty" as the sequence is aligned. This is calculated by taking the entropy at this position if we randomly aligned sequences (<span class="math-container">$H_g(L)$</span>), and subtracting from it the entropy of the alignment (<span class="math-container">$H_g(s)$</span>): 
  <span class="math-container">$$
  R_{sequence}=H_g(L) - H_s(L)
  $$</span></p>
  
  <p>The entropy at position 3 based on a random alignment is calculated by assuming there are 4 equally likely events (one for each base), and thus:
  <span class="math-container">$$
  \begin{align*} 
  H_g(L) &amp; = -((1/4 \times -2) + (1/4 \times -2) + (1/4 \times -2) + (1/4 \times -2)) \\
  H_g(L) &amp; = 2
  \end{align*}
  $$</span>
  Notably, <span class="math-container">$H_g(L)$</span> will always be 2 when dealing with nucleotide sequences.</p>
  
  <p>So in your example, we have:
  <span class="math-container">$$
  \begin{align*} 
  R_{sequence}&amp;=H_g(L) - H_s(L) \\
  R_{sequence}&amp;=2 - 0 \\
  R_{sequence}&amp;=2 \text{ bits of entropy}
  \end{align*}
  $$</span></p>
  
  <p>Finally, to work out the height of each base at each position in the logo, we multiply the frequency of that base by the overall information gain <span class="math-container">$R_{sequence}$</span>. In this case the frequency of the base A at position 3 is of course 1:
  <span class="math-container">$$
  \begin {align*}
  H(b,l) &amp;=f(b, l) \times R_{sequence} \\
  H(A, 3) &amp;= f(a, 3) \times 2 \\
  &amp;= 1 \times 2 \\
  &amp;= 2
  \end{align*}
  $$</span></p>
  
  <p>Source: <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC332411/" rel="nofollow noreferrer">Sequence logos: a new way to display consensus sequences.
   (Thomas D. Schneider, R. Michael Stephens, 1990)</a></p>
  
  <p>Note: I've excluded the correction factor <span class="math-container">$e(n)$</span> from the overall information-gain calculation for the sake of simplicity. Refer to the paper above for an explanation of this.</p>
  