Answers.Body: <p>My understanding of the subject is that the bias of the gene length (and other bias) should be taken care when analyzing the expression and <strong>before</strong> enrichment analysis. The enrichment analysis should be done once the corrections has been performed. Because as the abstract of the <a href="https://genomebiology.biomedcentral.com/articles/10.1186/gb-2010-11-2-r14" rel="nofollow noreferrer">GOseq paper states</a>:</p>
  
  <blockquote>
    <p>GO analysis is widely used ... but standard methods give biased results on RNA-seq data due to over-detection of differential expression for long and highly expressed transcripts.</p>
  </blockquote>
  
  <p>So first, take care of the differential expression bias by length and then use the GO to reduce complexity. How you take care of the bias in the RNA-seq data is another question. But the <a href="http://bioconductor.org/packages/cqn" rel="nofollow noreferrer">cqn</a> package of Bioconductor can correct the expression by gene length and GC content. However, this correction might hurt the differential tool used (see <a href="https://support.bioconductor.org/p/95683/#95713" rel="nofollow noreferrer">this discussion</a> in Bioconductor), so it might be better in some cases to use GOSeq. </p>
  
  <p>Now, to the question itself:</p>
  
  <blockquote>
    <p>So I'm know unsure what to do, should I use the analysis corrected for length or read count. Or perhaps take only terms significant in both? Or only in one?</p>
  </blockquote>
  
  <p>Use whatever correction method that yields better differentially expressed genes (DEG. If you find that the correction for length improves the accuracy of the predictions of DEG better than correcting by length and GC, then use that one. </p>
  
  <p>Another option to obtain accurate GO terms, then you could use other testing procedures which don't rely uniquely in the Fisher test, such as those that take into account the structure of the GO graph. <a href="http://bioconductor.org/packages/topGO" rel="nofollow noreferrer">TopGO</a> use this approach (note that it is a bit difficult to work with this package),this will reduce the role of the gene length bias (and probably other bias) in the resulting significant GO.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-21T16:04:07.653Z
Answers.Text: These bias' work through power. You cannot magically generate power out of thin air, so there are two ways you could remove the bias pre or at the DE stage: 1) reducing variance between samples 2) down-weighting more highly powered genes. As Mike Love notes in that link, modeling in the row-wise read counts is not compatible with DESeq2s methods (it were possible to adjust for extra power of highly expressed genes at the DE stage, presumably DESeq would already do it). Also who does one measure the accuracy of DEG prediction with no ground truth?
Answers.UserDisplayName: Ian Sudbery
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-21T18:56:32.230Z
Answers.Text: Taking into account the length of the genes won't reduce the variance between samples, so your only option is to down-weight highly powered genes. Maybe there are other methods to take it into account (or you could develop them). To find the true DEG for a given comparison: You can explore the FDR values of the genes in the comparison. The FDR values are assumed to follow an uniform distribution, because each gene has the same probability to be DEG. By exploring how uniform is that distribution you can find if the model captures the biases or not. You could also use the SeqGSA method
Answers.UserDisplayName: llrs
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-22T08:45:28.807Z
Answers.Text: But I'm not interested in finding a better set of DEGs, rather, in better assessing whether their division in categories is what you would expect by chance. GOSeq allows you to do this while correcting for bias, but only one bias at once.
Answers.UserDisplayName: Ian Sudbery
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-22T08:56:15.007Z
Answers.Text: The problem in this approach is that the DEG you test, are affected by this length and GC bias. If you want to to assess their division in categories use topGO, because it is the only method which does an ORA analysis taking into account the structure of the GO graph.  GOSeq is biased by not taking into account the relationships between GO terms.
Answers.UserDisplayName: llrs
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-21T15:43:53.230Z
Answers.LastActivityDate: 2017-08-21T15:43:53.230Z
Answers.UserDisplayName: llrs
Answers.Body: <p>I am not sure if it makes sense to use read counts as bias instead of gene length (and I certainly wouldn't expect the same results). </p>
  
  <p>Do you use total read counts of all your samples (library size)? </p>
  
  <p>The correction for gene length is a pure technical one, the longer a gene the more reads will align (and higher read count genes are easier significant, since they are way above the noise threshold). If you use read counts, you also have a biological factor (expression) in there, which (I think) is the stuff you test with statistics (e.g., with edgeR) and thus not the bias you want to correct for. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-21T13:27:54.280Z
Answers.Text: Imagine you have two genes: Gene A has 10,000 counts in Condition A and 5,000 in condition B. Gene B has 1,000 counts in Condition A and 500 in condition B. The lfcs are the same, but Gene A is more likely to be call differential than gene B. Now consider two GO categories. Category X is made up of 10,000/5,000 genes and Category Y is made up of 1,000/500 genes. Cat X is more likely to be enriched than Cat Y, despite the real LFC in the genes being the same.
Answers.UserDisplayName: Ian Sudbery
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-21T14:09:35.273Z
Answers.Text: I don't know if the second example is true (that category X is more likely enriched). Maybe you should ask real statisticians on [bioconductor](https://support.bioconductor.org/t/Latest/)?
Answers.UserDisplayName: benn
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-21T13:21:22.433Z
Answers.LastActivityDate: 2017-08-21T13:21:22.433Z
Answers.UserDisplayName: benn
Body: <p>It is a well reported fact that GO analysis of RNAseq results is affected by a number of biases, including length bias and expression level bias. </p>
  
  <p>The <code>bioconductor</code> package <a href="https://bioconductor.org/packages/release/bioc/html/goseq.html" rel="nofollow noreferrer"><code>goseq</code></a> allows you to correct for these biases. </p>
  
  <p>By default it corrects for length bias, but you can also get it to do read count bias. Using read counts to do the correction is attractive because in theory it should account for both sources of bias ($read counts\approx expression \times length$).</p>
  
  <p>I'm doing an enrichment analysis were I have tried both options (length and read counts) and get very different answers. If I run a binomial regression on expression and length vs probability of being differential, I can see that both are <em>independently</em> important.</p>
  
  <p></p>
  
  <pre><code>&gt; model &lt;- glm(sig ~  expression + log(length), data=retained_genes,  family=binomial(link="logit"))
  &gt; print(anova(model, test="Chisq"))
  
  Analysis of Deviance Table
  
  Model: binomial, link: logit
  
  Response: sig
  
  Terms added sequentially (first to last)
  
                         Df Deviance Resid. Df Resid. Dev  Pr(&gt;Chi)    
  NULL                                    6676     4507.1              
  expression              1  114.998      6675     4392.1 &lt; 2.2e-16 ***
  log(length)             1  102.553      6674     4289.5 &lt; 2.2e-16 ***
  expression:log(length)  1   34.094      6673     4255.4 5.252e-09 ***
  ---
  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
  </code></pre>
  
  <p>So I'm know unsure what to do, should I use the analysis corrected for length or read count. Or perhaps take only terms significant in both? Or only in one?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-08-21T15:37:37.837Z
Comments.Text: Did you test for the interaction of them (`expression*log(length)`)?
Comments.UserDisplayName: llrs
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-08-21T15:51:02.547Z
Comments.Text: I had not. Now I have. See updated question.
Comments.UserDisplayName: Ian Sudbery
Tags: <rna-seq><differential-expression><go>
Title: Correct for gene length or read counts in GO enrichment analysis
ThreadText: # Correct for gene length or read counts in GO enrichment analysis
  
  (Asked by: username_0 on 2017-08-21)
  
  It is a well reported fact that GO analysis of RNAseq results is affected by a number of biases, including length bias and expression level bias.
  
  The `bioconductor` package [`goseq`](https://bioconductor.org/packages/release/bioc/html/goseq.html) allows you to correct for these biases.
  
  By default it corrects for length bias, but you can also get it to do read count bias. Using read counts to do the correction is attractive because in theory it should account for both sources of bias ($read counts\approx expression \times length$).
  
  I'm doing an enrichment analysis were I have tried both options (length and read counts) and get very different answers. If I run a binomial regression on expression and length vs probability of being differential, I can see that both are *independently* important.
  
  ```
  > model <- glm(sig ~  expression + log(length), data=retained_genes,  family=binomial(link="logit"))
  > print(anova(model, test="Chisq"))
  
  Analysis of Deviance Table
  
  Model: binomial, link: logit
  
  Response: sig
  
  Terms added sequentially (first to last)
  
                         Df Deviance Resid. Df Resid. Dev  Pr(>Chi)    
  NULL                                    6676     4507.1              
  expression              1  114.998      6675     4392.1 < 2.2e-16 ***
  log(length)             1  102.553      6674     4289.5 < 2.2e-16 ***
  expression:log(length)  1   34.094      6673     4255.4 5.252e-09 ***
  ---
  Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
  
  ```
  
  So I'm know unsure what to do, should I use the analysis corrected for length or read count. Or perhaps take only terms significant in both? Or only in one?
  
  **Tags:** rna-seq, differential-expression, go
  
  > **Comments:**
  > **username_1**: Did you test for the interaction of them (`expression*log(length)`)?  
  > **username_0**: I had not. Now I have. See updated question.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2017-08-21. Score: 2
  
  My understanding of the subject is that the bias of the gene length (and other bias) should be taken care when analyzing the expression and **before** enrichment analysis. The enrichment analysis should be done once the corrections has been performed. Because as the abstract of the [GOseq paper states](https://genomebiology.biomedcentral.com/articles/10.1186/gb-2010-11-2-r14):
  
  > GO analysis is widely used ... but standard methods give biased results on RNA-seq data due to over-detection of differential expression for long and highly expressed transcripts.
  
  So first, take care of the differential expression bias by length and then use the GO to reduce complexity. How you take care of the bias in the RNA-seq data is another question. But the [cqn](http://bioconductor.org/packages/cqn) package of Bioconductor can correct the expression by gene length and GC content. However, this correction might hurt the differential tool used (see [this discussion](https://support.bioconductor.org/p/95683/#95713) in Bioconductor), so it might be better in some cases to use GOSeq.
  
  Now, to the question itself:
  
  > So I'm know unsure what to do, should I use the analysis corrected for length or read count. Or perhaps take only terms significant in both? Or only in one?
  
  Use whatever correction method that yields better differentially expressed genes (DEG. If you find that the correction for length improves the accuracy of the predictions of DEG better than correcting by length and GC, then use that one.
  
  Another option to obtain accurate GO terms, then you could use other testing procedures which don't rely uniquely in the Fisher test, such as those that take into account the structure of the GO graph. [TopGO](http://bioconductor.org/packages/topGO) use this approach (note that it is a bit difficult to work with this package),this will reduce the role of the gene length bias (and probably other bias) in the resulting significant GO.
  
  > **Comments:**
  > **username_0**: These bias' work through power. You cannot magically generate power out of thin air, so there are two ways you could remove the bias pre or at the DE stage: 1) reducing variance between samples 2) down-weighting more highly powered genes. As Mike Love notes in that link, modeling in the row-wise read counts is not compatible with DESeq2s methods (it were possible to adjust for extra power of highly expressed genes at the DE stage, presumably DESeq would already do it). Also who does one measure the accuracy of DEG prediction with no ground truth?  
  > **username_1**: Taking into account the length of the genes won't reduce the variance between samples, so your only option is to down-weight highly powered genes. Maybe there are other methods to take it into account (or you could develop them). To find the true DEG for a given comparison: You can explore the FDR values of the genes in the comparison. The FDR values are assumed to follow an uniform distribution, because each gene has the same probability to be DEG. By exploring how uniform is that distribution you can find if the model captures the biases or not. You could also use the SeqGSA method  
  > **username_0**: But I'm not interested in finding a better set of DEGs, rather, in better assessing whether their division in categories is what you would expect by chance. GOSeq allows you to do this while correcting for bias, but only one bias at once.  
  > **username_1**: The problem in this approach is that the DEG you test, are affected by this length and GC bias. If you want to to assess their division in categories use topGO, because it is the only method which does an ORA analysis taking into account the structure of the GO graph.  GOSeq is biased by not taking into account the relationships between GO terms.  
  
  ---
  
  ### Answer by username_2 on 2017-08-21. Score: 1
  
  I am not sure if it makes sense to use read counts as bias instead of gene length (and I certainly wouldn't expect the same results).
  
  Do you use total read counts of all your samples (library size)?
  
  The correction for gene length is a pure technical one, the longer a gene the more reads will align (and higher read count genes are easier significant, since they are way above the noise threshold). If you use read counts, you also have a biological factor (expression) in there, which (I think) is the stuff you test with statistics (e.g., with edgeR) and thus not the bias you want to correct for.
  
  > **Comments:**
  > **username_0**: Imagine you have two genes: Gene A has 10,000 counts in Condition A and 5,000 in condition B. Gene B has 1,000 counts in Condition A and 500 in condition B. The lfcs are the same, but Gene A is more likely to be call differential than gene B. Now consider two GO categories. Category X is made up of 10,000/5,000 genes and Category Y is made up of 1,000/500 genes. Cat X is more likely to be enriched than Cat Y, despite the real LFC in the genes being the same.  
  > **username_2**: I don't know if the second example is true (that category X is more likely enriched). Maybe you should ask real statisticians on [bioconductor](https://support.bioconductor.org/t/Latest/)?  
  
  ---
  
  