Answers.Body: <p>You are trying to obtain Hill numbers, sometimes called true diversity, and you have different results because you have uneven species frequencies.</p>
  <p><code>Vegan</code> calculates Shannon index like:</p>
  <p><span class="math-container">$$ H' = -\sum_{i=1}^R p_i \ln p_i $$</span></p>
  <p>And Gini-Simpson like:</p>
  <p><span class="math-container">$$ 1 - \lambda = 1 - \sum_{i=1}^R p_i^2 $$</span></p>
  <p>So the Hill numbers should be:</p>
  <p><span class="math-container">$$ H1 = exp(H') $$</span></p>
  <p><span class="math-container">$$ H2 = 1 / 1 - \lambda $$</span></p>
  <p>With <code>Phyloseq</code>:</p>
  <pre><code>H1 &lt;- 1 / estimate_richness(globp, measures=&quot;Simpson&quot;)
  H2 &lt;- exp(estimate_richness(globp, measures=&quot;Shannon&quot;)
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-28T14:59:07.853Z
Answers.Text: I do have a question, though...shouldn't H1 and H2 be equal to each other, since we are talking about true diversity? Why is that not the case?
Answers.UserDisplayName: gbt
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-28T21:33:52.670Z
Answers.Text: H1 is equal to H2 for an even species frequency. The text that you cite uses the term "true diversity" or the "effective number of species", which means [Hill numbers](https://en.wikipedia.org/wiki/Diversity_index#Effective_number_of_species_or_Hill_numbers).
Answers.UserDisplayName: zorbax
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-31T16:32:05.190Z
Answers.Text: alright I think I am understanding what you meant. thank you!
Answers.UserDisplayName: gbt
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-28T11:38:24.847Z
Answers.LastActivityDate: 2022-01-28T11:38:24.847Z
Answers.UserDisplayName: zorbax
Body: <p>So I am trying to compute diversity indices for a dataset I have but the math doesn't work out and I don't really understand how to translate Shannon and/or Simpson diversity indices into something that is understandable. Maybe I got everything wrong, this is certainly a possibility. However, this is what I have up to now:</p>
  <pre><code># load libraries
  library(&quot;phyloseq&quot;)
  data(&quot;GlobalPatterns&quot;)
  
  # load data
  globp = GlobalPatterns
  
  # compute transformation according to link below
  simpson_transformed &lt;- 1/(1-estimate_richness(globp, measures=&quot;Simpson&quot;))
  shannon_transformed &lt;- exp(estimate_richness(globp, measures=&quot;Shannon&quot;))
  
  # print side by side
  print(cbind.data.frame(simpson_transformed, shannon_transformed))
  </code></pre>
  <p>The <strong>comparing_indices</strong> columns are computed according to what I found <a href="http://www.loujost.com/Statistics%20and%20Physics/Diversity%20and%20Similarity/Diversity%20of%20a%20single%20community.htm" rel="nofollow noreferrer">at this link</a> (italic is mine):</p>
  <blockquote>
  <p>They <em>[i.e. the values]</em> should be converted to effective number of species,
  which are the true diversities. By referring to Table 1 we see that
  species richness is already a true diversity, so the true diversity
  according to this index is 5.000 species. Table 1 shows that the
  Shannon entropy or Shannon-Wiener index is converted to effective
  number of species or true diversity by taking the exponential:
  exp(1.609) is 5.000, so the true diversity according to the Shannon
  entropy is also 5.000 species. The Gini-Simpson index is converted to
  a  true diversity by subtracting it from unity and inverting:
  1/(1-0.8) = 5.000 species also. So in fact all these indices agree
  that the diversity of this community is 5.000 species.</p>
  </blockquote>
  <p>I also verified how <strong>phyloseq</strong> is computing the indices. I believe the transformation for Shannon should be the same mentioned above. I have doubts about the Simpson, since they refer to the Gini-Simpson index and not the Simpson itself. However, the <a href="https://www.rdocumentation.org/packages/vegan/versions/2.4-2/topics/diversity" rel="nofollow noreferrer">documentation</a> for <strong>vegan</strong>'s <code>diversity()</code> function, which is the function that (I believe) is called by <strong>phyloseq</strong>'s <code>estimate_richness()</code> (see <a href="https://www.rdocumentation.org/packages/phyloseq/versions/1.16.2/topics/estimate_richness" rel="nofollow noreferrer">docs</a>), mention that the Simpson index is returned as <code>1-D</code> <a href="https://en.wikipedia.org/wiki/Diversity_index#Gini%E2%80%93Simpson_index" rel="nofollow noreferrer">which seems to be the Gini-Simpson index</a> (see the <code>1-lambda</code>). In other words, it looks like the transformation I linked above should be exactly the one that are used in the code I provided. But, I was expecting the same values in the two columns. But they are not there and I wonder why this happens. What did I do wrong?</p>
  
Tags: <phyloseq>
Title: Translating Simpson and Shannon diversity indices?
ThreadText: # Translating Simpson and Shannon diversity indices?
  
  (Asked by: username_0 on 2022-01-27)
  
  So I am trying to compute diversity indices for a dataset I have but the math doesn't work out and I don't really understand how to translate Shannon and/or Simpson diversity indices into something that is understandable. Maybe I got everything wrong, this is certainly a possibility. However, this is what I have up to now:
  
  ```
  # load libraries
  library("phyloseq")
  data("GlobalPatterns")
  
  # load data
  globp = GlobalPatterns
  
  # compute transformation according to link below
  simpson_transformed <- 1/(1-estimate_richness(globp, measures="Simpson"))
  shannon_transformed <- exp(estimate_richness(globp, measures="Shannon"))
  
  # print side by side
  print(cbind.data.frame(simpson_transformed, shannon_transformed))
  
  ```
  
  The **comparing\_indices** columns are computed according to what I found [at this link](http://www.loujost.com/Statistics%20and%20Physics/Diversity%20and%20Similarity/Diversity%20of%20a%20single%20community.htm) (italic is mine):
  
  > They *[i.e. the values]* should be converted to effective number of species,
  > which are the true diversities. By referring to Table 1 we see that
  > species richness is already a true diversity, so the true diversity
  > according to this index is 5.000 species. Table 1 shows that the
  > Shannon entropy or Shannon-Wiener index is converted to effective
  > number of species or true diversity by taking the exponential:
  > exp(1.609) is 5.000, so the true diversity according to the Shannon
  > entropy is also 5.000 species. The Gini-Simpson index is converted to
  > a true diversity by subtracting it from unity and inverting:
  > 1/(1-0.8) = 5.000 species also. So in fact all these indices agree
  > that the diversity of this community is 5.000 species.
  
  I also verified how **phyloseq** is computing the indices. I believe the transformation for Shannon should be the same mentioned above. I have doubts about the Simpson, since they refer to the Gini-Simpson index and not the Simpson itself. However, the [documentation](https://www.rdocumentation.org/packages/vegan/versions/2.4-2/topics/diversity) for **vegan**'s `diversity()` function, which is the function that (I believe) is called by **phyloseq**'s `estimate_richness()` (see [docs](https://www.rdocumentation.org/packages/phyloseq/versions/1.16.2/topics/estimate_richness)), mention that the Simpson index is returned as `1-D` [which seems to be the Gini-Simpson index](https://en.wikipedia.org/wiki/Diversity_index#Gini%E2%80%93Simpson_index) (see the `1-lambda`). In other words, it looks like the transformation I linked above should be exactly the one that are used in the code I provided. But, I was expecting the same values in the two columns. But they are not there and I wonder why this happens. What did I do wrong?
  
  **Tags:** phyloseq
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-01-28. Score: 2 | Accepted answer
  
  You are trying to obtain Hill numbers, sometimes called true diversity, and you have different results because you have uneven species frequencies.
  
  `Vegan` calculates Shannon index like:
  
  $$ H' = -\sum\_{i=1}^R p\_i \ln p\_i $$
  
  And Gini-Simpson like:
  
  $$ 1 - \lambda = 1 - \sum\_{i=1}^R p\_i^2 $$
  
  So the Hill numbers should be:
  
  $$ H1 = exp(H') $$
  
  $$ H2 = 1 / 1 - \lambda $$
  
  With `Phyloseq`:
  
  ```
  H1 <- 1 / estimate_richness(globp, measures="Simpson")
  H2 <- exp(estimate_richness(globp, measures="Shannon")
  
  ```
  
  > **Comments:**
  > **username_0**: I do have a question, though...shouldn't H1 and H2 be equal to each other, since we are talking about true diversity? Why is that not the case?  
  > **username_1**: H1 is equal to H2 for an even species frequency. The text that you cite uses the term "true diversity" or the "effective number of species", which means [Hill numbers](https://en.wikipedia.org/wiki/Diversity_index#Effective_number_of_species_or_Hill_numbers).  
  > **username_0**: alright I think I am understanding what you meant. thank you!  
  
  ---
  
  