Answers.Body: <p>The problem is that the enrichment score is modified based on whether the sets are positively or negatively associated, which means that an unbalanced set via either associative statistic or gene count (as is almost always the case) will lead to an incorrect total score.</p>
  <p>Put in other words, the calculated score from the positive genes alters the calculated score from the negative genes.</p>
  <p>Given this, it's a better idea to either separately calculate scores for the positive set and the negative set (this is the easiest approach, and works with existing methods), or (equivalently) recalculate enrichment score modifiers for positive and negative genes separately (i.e. adjust the graph) so that the zero point for the enrichment score crosses the zero point for association.</p>
  <p>A common argument against this (when I have mentioned it to others) is that GSEA has <em>always</em> been done on the total gene set, so it makes sense to do it on the total gene set because it has always been done that way. I'm not a huge fan of the status-quo argument.</p>
  <p>I alluded to this issue in <a href="https://bioinformatics.stackexchange.com/a/15963/73">this previous answer</a>.</p>
  <p>With regards to your figures:</p>
  <p><strong>a</strong>. Zero-point crossover is at an enrichment score of ~0.42, so the negative set will be underemphasised. It's not contributing much anyway (except for the one most negative gene at the end), so wouldn't affect the outcome much.</p>
  <p><strong>b</strong>. Zero-point crossover is at an enrichment score of ~0.45. Similar situation as in <strong>a</strong>.</p>
  <p><strong>c</strong>. Zero-point crossover is at an enrichment score of ~0.32. There aren't really any positive genes with substantial association scores, so no issue.</p>
  <p><strong>d</strong>. Plot looks to be identical to <strong>c</strong>, so the same comments apply.</p>
  <p>The scores are most affected when they have a lot of area on both sides of the Y axis. The top right plot of this sub-figure is an example of that, showing a sharp-ish tail on the negative side, so the calculated enrichment score for the negative peak is less extreme than it would be if only the negative genes were considered [<a href="https://www.nature.com/articles/s41590-021-01067-0/figures/8" rel="nofollow noreferrer">source</a>]:</p>
  <p><a href="https://i.sstatic.net/gYveS.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/gYveS.jpg" alt="GSEA plots with zero points marked; most plots show a large steep curve for positive genes, with a small hook for negative genes" /></a></p>
  <blockquote>
  <p>The way it is phrased in the paper seems to suggest that it's the weighting that causes an issue.</p>
  </blockquote>
  <p>Yes. If the positive genes are weighted separately from the negative genes, then the bias can be removed. I did briefly attempt a non-parametric weighting for GSEA (based on rank), but got too confused by the maths involved. Someone answered my question about it, but I haven't had to do any GSEA since then, so haven't actually verified if their solution works for me:</p>
  <p><a href="https://math.stackexchange.com/questions/3567714/probability-that-the-nth-highest-of-a-set-of-k-numbers-selected-from-m-numbers-h">https://math.stackexchange.com/questions/3567714/probability-that-the-nth-highest-of-a-set-of-k-numbers-selected-from-m-numbers-h</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-22T01:37:42.903Z
Answers.Text: Ah I believe I see your point now about recalibrating the graph I think. In practice, I'm guessing this would be done by having N - N_H in P_miss be equal to the "missing genes" on the positive side ONLY, for positive genes, and negative side ONLY, for negative genes
Answers.UserDisplayName: An Ignorant Wanderer
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-22T01:40:05.220Z
Answers.Text: Analogously we'd need to modify P_hit so the denominator only has genes on positive side vs negative side
Answers.UserDisplayName: An Ignorant Wanderer
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-21T21:00:04.993Z
Answers.LastActivityDate: 2023-05-09T21:17:54.697Z
Answers.LastEditDate: 2023-05-09T21:17:54.697Z
Answers.UserDisplayName: gringer
Body: <p>I'm reading Subramanian et. al's (2005) original GSEA <a href="https://www.pnas.org/doi/10.1073/pnas.0506580102" rel="nofollow noreferrer">paper</a>. In the paper, the authors mention the following:</p>
  <blockquote>
  <p>We noticed that the use of
  weighted steps could cause the distribution of observed ES scores to
  be asymmetric in cases where many more genes are correlated with one
  of the two phenotypes. We therefore estimate the significance levels
  by considering separately the positively and negatively scoring gene
  sets</p>
  </blockquote>
  <p>I understand what they're saying, but I simply don't understand why it's a problem. Basically, they're saying that in some cases, most genes in the list we're testing could appear mostly at the top or bottom of the list. But I don't understand why having asymmetric distributions is a problem. What is the issue here?</p>
  <p>EDIT: Here are some example GSEA plots I found via a web search with such gene sets. Where is the problem?
  <a href="https://i.sstatic.net/qJdzT.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/qJdzT.jpg" alt="enter image description here" /></a></p>
  <p>One thing I don't get though is why this problem would be introduced through &quot;weighting&quot; of the steps. I feel like it would also be a problem without weighting. The way it is phrased in the paper seems to suggest that it's the weighting that causes an issue.</p>
  
Tags: <gsea>
Title: Why are asymmetric distributions an issue for GSEA?
ThreadText: # Why are asymmetric distributions an issue for GSEA?
  
  (Asked by: username_0 on 2023-03-21)
  
  I'm reading Subramanian et. al's (2005) original GSEA [paper](https://www.pnas.org/doi/10.1073/pnas.0506580102). In the paper, the authors mention the following:
  
  > We noticed that the use of
  > weighted steps could cause the distribution of observed ES scores to
  > be asymmetric in cases where many more genes are correlated with one
  > of the two phenotypes. We therefore estimate the significance levels
  > by considering separately the positively and negatively scoring gene
  > sets
  
  I understand what they're saying, but I simply don't understand why it's a problem. Basically, they're saying that in some cases, most genes in the list we're testing could appear mostly at the top or bottom of the list. But I don't understand why having asymmetric distributions is a problem. What is the issue here?
  
  EDIT: Here are some example GSEA plots I found via a web search with such gene sets. Where is the problem?
  [![enter image description here](https://i.sstatic.net/qJdzT.jpg)](https://i.sstatic.net/qJdzT.jpg)
  
  One thing I don't get though is why this problem would be introduced through "weighting" of the steps. I feel like it would also be a problem without weighting. The way it is phrased in the paper seems to suggest that it's the weighting that causes an issue.
  
  **Tags:** gsea
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-03-21. Score: 1 | Accepted answer
  
  The problem is that the enrichment score is modified based on whether the sets are positively or negatively associated, which means that an unbalanced set via either associative statistic or gene count (as is almost always the case) will lead to an incorrect total score.
  
  Put in other words, the calculated score from the positive genes alters the calculated score from the negative genes.
  
  Given this, it's a better idea to either separately calculate scores for the positive set and the negative set (this is the easiest approach, and works with existing methods), or (equivalently) recalculate enrichment score modifiers for positive and negative genes separately (i.e. adjust the graph) so that the zero point for the enrichment score crosses the zero point for association.
  
  A common argument against this (when I have mentioned it to others) is that GSEA has *always* been done on the total gene set, so it makes sense to do it on the total gene set because it has always been done that way. I'm not a huge fan of the status-quo argument.
  
  I alluded to this issue in [this previous answer](https://bioinformatics.stackexchange.com/a/15963/73).
  
  With regards to your figures:
  
  **a**. Zero-point crossover is at an enrichment score of ~0.42, so the negative set will be underemphasised. It's not contributing much anyway (except for the one most negative gene at the end), so wouldn't affect the outcome much.
  
  **b**. Zero-point crossover is at an enrichment score of ~0.45. Similar situation as in **a**.
  
  **c**. Zero-point crossover is at an enrichment score of ~0.32. There aren't really any positive genes with substantial association scores, so no issue.
  
  **d**. Plot looks to be identical to **c**, so the same comments apply.
  
  The scores are most affected when they have a lot of area on both sides of the Y axis. The top right plot of this sub-figure is an example of that, showing a sharp-ish tail on the negative side, so the calculated enrichment score for the negative peak is less extreme than it would be if only the negative genes were considered [[source](https://www.nature.com/articles/s41590-021-01067-0/figures/8)]:
  
  [![GSEA plots with zero points marked; most plots show a large steep curve for positive genes, with a small hook for negative genes](https://i.sstatic.net/gYveS.jpg)](https://i.sstatic.net/gYveS.jpg)
  
  > The way it is phrased in the paper seems to suggest that it's the weighting that causes an issue.
  
  Yes. If the positive genes are weighted separately from the negative genes, then the bias can be removed. I did briefly attempt a non-parametric weighting for GSEA (based on rank), but got too confused by the maths involved. Someone answered my question about it, but I haven't had to do any GSEA since then, so haven't actually verified if their solution works for me:
  
  <https://math.stackexchange.com/questions/3567714/probability-that-the-nth-highest-of-a-set-of-k-numbers-selected-from-m-numbers-h>
  
  > **Comments:**
  > **username_0**: Ah I believe I see your point now about recalibrating the graph I think. In practice, I'm guessing this would be done by having N - N_H in P_miss be equal to the "missing genes" on the positive side ONLY, for positive genes, and negative side ONLY, for negative genes  
  > **username_0**: Analogously we'd need to modify P_hit so the denominator only has genes on positive side vs negative side  
  
  ---
  
  