Answers.Body: <p>Why do you think the entropy of 0 is incorrect? It intuitively makes sense, as there is no uncertainty about the base at position 3, and thus there is no entropy.</p>
  
  <p>However, what is plotted in a sequence logo isn't the entropy, but rather a measure of the "decrease in uncertainty" as the sequence is aligned. This is calculated by taking the entropy at this position if we randomly aligned sequences (<span class="math-container">$H_g(L)$</span>), and subtracting from it the entropy of the alignment (<span class="math-container">$H_g(s)$</span>): 
  <span class="math-container">$$
  R_{sequence}=H_g(L) - H_s(L)
  $$</span></p>
  
  <p>The entropy at position 3 based on a random alignment is calculated by assuming there are 4 equally likely events (one for each base), and thus:
  <span class="math-container">$$
  \begin{align*} 
  H_g(L) &amp; = -((1/4 \times -2) + (1/4 \times -2) + (1/4 \times -2) + (1/4 \times -2)) \\
  H_g(L) &amp; = 2
  \end{align*}
  $$</span>
  Notably, <span class="math-container">$H_g(L)$</span> will always be 2 when dealing with nucleotide sequences.</p>
  
  <p>So in your example, we have:
  <span class="math-container">$$
  \begin{align*} 
  R_{sequence}&amp;=H_g(L) - H_s(L) \\
  R_{sequence}&amp;=2 - 0 \\
  R_{sequence}&amp;=2 \text{ bits of entropy}
  \end{align*}
  $$</span></p>
  
  <p>Finally, to work out the height of each base at each position in the logo, we multiply the frequency of that base by the overall information gain <span class="math-container">$R_{sequence}$</span>. In this case the frequency of the base A at position 3 is of course 1:
  <span class="math-container">$$
  \begin {align*}
  H(b,l) &amp;=f(b, l) \times R_{sequence} \\
  H(A, 3) &amp;= f(a, 3) \times 2 \\
  &amp;= 1 \times 2 \\
  &amp;= 2
  \end{align*}
  $$</span></p>
  
  <p>Source: <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC332411/" rel="nofollow noreferrer">Sequence logos: a new way to display consensus sequences.
   (Thomas D. Schneider, R. Michael Stephens, 1990)</a></p>
  
  <p>Note: I've excluded the correction factor <span class="math-container">$e(n)$</span> from the overall information-gain calculation for the sake of simplicity. Refer to the paper above for an explanation of this.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-30T22:33:26.880Z
Answers.Text: Thanks for your answer. What does "g" mean/represent/come from in $H_g(L)$
Answers.UserDisplayName: czlsws
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-31T03:00:43.367Z
Answers.Text: I'm not sure. My guess is g=genome and s=sequence(ing) but that's pure conjecture since the paper doesn't spell it out
Answers.UserDisplayName: Migwell
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-31T03:47:27.403Z
Answers.Text: Thanks for your reply. And does L mean/represent/come from `location` since the paper says "at various positions (L)".
Answers.UserDisplayName: czlsws
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-31T03:57:14.310Z
Answers.Text: Yes, that would be my assumption. This is important to include in the equation because it's only calculating the information at a single position. The authors propose another equation for an entire sequence (which is just summing these per-position scores)
Answers.UserDisplayName: Migwell
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-31T13:35:18.993Z
Answers.Text: Thanks a lot. And last question, does R come from/represent/mean acids?
Answers.UserDisplayName: czlsws
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-31T14:56:51.003Z
Answers.Text: I don't know. But also I think you're putting too much importance on the authors' choice of variables. You could write any of these equations with different letters and they would be just as correct.
Answers.UserDisplayName: Migwell
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-30T13:10:52.197Z
Answers.LastActivityDate: 2019-07-31T15:12:19.477Z
Answers.LastEditDate: 2019-07-31T15:12:19.477Z
Answers.UserDisplayName: Migwell
Body: <p>I'm confused by the <a href="https://bioinformatics.stackexchange.com/q/9091/4962">computation</a> of sequence logo. <a href="https://en.wikipedia.org/wiki/Sequence_logo#Logo_creation" rel="nofollow noreferrer">Wikipedia</a> gives a process about this without a concrete example.</p>
  
  <p>Let's just consider DNA, so there are 4 bases (nucleic acids).</p>
  
  <p>The following data comes from the book "Machine Learning - A Probabilistic Perspective (Figure 1)"</p>
  
  <p><a href="https://i.sstatic.net/XP7q1.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/XP7q1.png" alt="enter image description here"></a></p>
  
  <p>The Shannon entropy of position <span class="math-container">$i$</span> is:</p>
  
  <p><span class="math-container">${\displaystyle H_{i}=-\sum _{b=a}^{t}f_{b,i}\times \log _{2}f_{b,i}}$</span></p>
  
  <p>Where <span class="math-container">${\displaystyle f_{b,i}}$</span> is the relative frequency of base</p>
  
  <p>This <a href="https://bioinformatics.stackexchange.com/a/9094/4962">post</a> is computing position 3, where it seems that the relative frequency of <code>a</code> is 100%, and the relative frequency of 3 other bases are 0%.</p>
  
  <p>Now, how do you calculate the Shannon entropy of position 3? It seems to be 0, which is apparently incorrect, where am I wrong?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-30T06:48:19.637Z
Comments.Text: Note that the frequency is always in the interval [0,1] instead of [0,100]. H is 0 for columns with only one symbol.
Comments.UserDisplayName: Peter Menzel
Tags: <phylogenetics><ngs><sequence-analysis>
Title: How to compute the Shannon entropy for a strand of DNA?
ThreadText: # How to compute the Shannon entropy for a strand of DNA?
  
  (Asked by: username_0 on 2019-07-30)
  
  I'm confused by the [computation](https://bioinformatics.stackexchange.com/q/9091/4962) of sequence logo. [Wikipedia](https://en.wikipedia.org/wiki/Sequence_logo#Logo_creation) gives a process about this without a concrete example.
  
  Let's just consider DNA, so there are 4 bases (nucleic acids).
  
  The following data comes from the book "Machine Learning - A Probabilistic Perspective (Figure 1)"
  
  [![enter image description here](https://i.sstatic.net/XP7q1.png)](https://i.sstatic.net/XP7q1.png)
  
  The Shannon entropy of position $i$ is:
  
  ${\displaystyle H\_{i}=-\sum \_{b=a}^{t}f\_{b,i}\times \log \_{2}f\_{b,i}}$
  
  Where ${\displaystyle f\_{b,i}}$ is the relative frequency of base
  
  This [post](https://bioinformatics.stackexchange.com/a/9094/4962) is computing position 3, where it seems that the relative frequency of `a` is 100%, and the relative frequency of 3 other bases are 0%.
  
  Now, how do you calculate the Shannon entropy of position 3? It seems to be 0, which is apparently incorrect, where am I wrong?
  
  **Tags:** phylogenetics, ngs, sequence-analysis
  
  > **Comments:**
  > **username_1**: Note that the frequency is always in the interval [0,1] instead of [0,100]. H is 0 for columns with only one symbol.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-07-30. Score: 5 | Accepted answer
  
  Why do you think the entropy of 0 is incorrect? It intuitively makes sense, as there is no uncertainty about the base at position 3, and thus there is no entropy.
  
  However, what is plotted in a sequence logo isn't the entropy, but rather a measure of the "decrease in uncertainty" as the sequence is aligned. This is calculated by taking the entropy at this position if we randomly aligned sequences ($H\_g(L)$), and subtracting from it the entropy of the alignment ($H\_g(s)$):
  $$
  R\_{sequence}=H\_g(L) - H\_s(L)
  $$
  
  The entropy at position 3 based on a random alignment is calculated by assuming there are 4 equally likely events (one for each base), and thus:
  $$
  \begin{align\*}
  H\_g(L) & = -((1/4 \times -2) + (1/4 \times -2) + (1/4 \times -2) + (1/4 \times -2)) \\
  H\_g(L) & = 2
  \end{align\*}
  $$
  Notably, $H\_g(L)$ will always be 2 when dealing with nucleotide sequences.
  
  So in your example, we have:
  $$
  \begin{align\*}
  R\_{sequence}&=H\_g(L) - H\_s(L) \\
  R\_{sequence}&=2 - 0 \\
  R\_{sequence}&=2 \text{ bits of entropy}
  \end{align\*}
  $$
  
  Finally, to work out the height of each base at each position in the logo, we multiply the frequency of that base by the overall information gain $R\_{sequence}$. In this case the frequency of the base A at position 3 is of course 1:
  $$
  \begin {align\*}
  H(b,l) &=f(b, l) \times R\_{sequence} \\
  H(A, 3) &= f(a, 3) \times 2 \\
  &= 1 \times 2 \\
  &= 2
  \end{align\*}
  $$
  
  Source: [Sequence logos: a new way to display consensus sequences.
  (Thomas D. Schneider, R. Michael Stephens, 1990)](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC332411/)
  
  Note: I've excluded the correction factor $e(n)$ from the overall information-gain calculation for the sake of simplicity. Refer to the paper above for an explanation of this.
  
  > **Comments:**
  > **username_0**: Thanks for your answer. What does "g" mean/represent/come from in $H_g(L)$  
  > **username_2**: I'm not sure. My guess is g=genome and s=sequence(ing) but that's pure conjecture since the paper doesn't spell it out  
  > **username_0**: Thanks for your reply. And does L mean/represent/come from `location` since the paper says "at various positions (L)".  
  > **username_2**: Yes, that would be my assumption. This is important to include in the equation because it's only calculating the information at a single position. The authors propose another equation for an entire sequence (which is just summing these per-position scores)  
  > **username_0**: Thanks a lot. And last question, does R come from/represent/mean acids?  
  > **username_2**: I don't know. But also I think you're putting too much importance on the authors' choice of variables. You could write any of these equations with different letters and they would be just as correct.  
  
  ---
  
  