Answers.Body: <p>I think since your PCA shows that your normal samples cluster right alongside the tumor ones, you have to conclude that there is much more variation between tumors than between tumors and controls.  DESeq's differentially expressed gene list was generated properly.</p>
  
  <p>Your results are what they are, sorry they are so useless.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T17:15:37.113Z
Answers.LastActivityDate: 2018-07-10T17:15:37.113Z
Answers.UserDisplayName: swbarnes2
Answers.Body: <p>In general, messing with FDR or log fold change cutoffs beyond the ones you used gets pretty sketchy. I usually only adjust these values in the more conservative direction, although you may be able to get away with FDR &lt; 0.1 if you have a large effect size (which you don't).</p>
  
  <p>You might consider investigating why you don't seem to get any differentially expressed genes before adjusting your cutoffs: PCA on the samples based on their top 1,000 most variably expressed genes would be a good start. Do the tumor and normal transcriptomes cluster apart from each other?</p>
  
  <p>With so few normal samples, it is possible that you are not fully characterizing the variability of that population and it is affecting your ability to identify meaningful differences between normal and tumor.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-09T22:57:14.243Z
Answers.Text: Hi, I made the PCA and it is given in the question. please have a look.
Answers.UserDisplayName: beginner
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-19T13:25:27.553Z
Answers.Text: If you have a large effect size you wouldn’t need to increase the FDR to 0.1: a lower FDR would still capture the effect.
Answers.UserDisplayName: Konrad Rudolph
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-09T22:45:55.040Z
Answers.LastActivityDate: 2018-07-09T22:45:55.040Z
Answers.UserDisplayName: Ben D.
Answers.Body: <ol>
  <li><p>You can usually get away with FDR &lt; 0.1, but that's as high as you can go. This all presumes you're doing follow-up experiments of some sort, of course. I guess you could increase the FDR more, but you're then really increasing the odds that your follow-up experiments will fail. Obviously increasing the FDR will decrease the confidence in the results, it's called "false discovery rate" for a reason.</p></li>
  <li><p>Yes, fold-change cut-offs are mostly useful when you need to prioritize hits further. Small changes are almost impossible to validate with less-sensitive methods (qPCR) and less likely to be biologically relevant.</p></li>
  <li><p>You're just going to waste your time if you ignore FDR.</p></li>
  </ol>
  
  <p>Remember that all thresholds are intended to increase the odds of success in follow-up experiments. If you have infinite time and resources then you can set these to whatever you want, but you have neither so you better prioritize reasonably.</p>
  
  <p>It's likely that you have a large batch effect in your data, effect sizes in cancer tend to be much larger than this.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T13:13:06.150Z
Answers.Text: Thanks Devon. Yes there will be follow up experiments after selection. So, what would recommend me now? Can I go with FC and FDR < 0.1? Yes, there is batch effect what can I do now? Can I apply subsampling procedure to get more DEGs?
Answers.UserDisplayName: beginner
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T13:14:40.237Z
Answers.Text: Yes, FDR<0.1 and ideally some reasonable fold-change (say a 30% change). Have a look at packages like `sva` for dealing with your batch effect.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T13:35:04.717Z
Answers.Text: Is it a good idea to apply subsampling procedure for DEA?
Answers.UserDisplayName: beginner
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T14:21:22.020Z
Answers.Text: @beginner No, that's not useful, presuming you mean subsampling from the counts.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T14:28:32.870Z
Answers.Text: selecting random samples from cancer cohort and repeating differential analysis n times will give different set of DEGs and then select which DEGs are most repeated in all the analyses. This is the one I'm asking.
Answers.UserDisplayName: beginner
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T14:30:14.933Z
Answers.Text: No, that's not helpful, you're just approximating the results you would have gotten otherwise.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T14:42:09.317Z
Answers.Text: Sure thank you. And one last question. After selecting candidates..for experiments I have selected two cell-lines.To look which gene is more expressed in which cell line, doing a simple t-test between the cell-lines for those genes is fine or should I go with Deseq2 to check how those genes were expressed?
Answers.UserDisplayName: beginner
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T14:47:11.397Z
Answers.Text: Use DESeq2, it's more powerful than a T-test.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T15:05:57.167Z
Answers.Text: Yes, it is more powerful but Deseq2 needs replicates for performing differential analysis. It will give a warning/error if tried to analyze data without replicates.
Answers.UserDisplayName: beginner
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T15:08:38.413Z
Answers.Text: I assumed you were talking about cell lines with replicates. No method will reliably work without replicates.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T15:15:32.023Z
Answers.Text: No, I just have two breast cell-lines. Interested in doing experiments using one cell-line. So I thought of looking at candidates expression in those cell-lines and then select based on that.
Answers.UserDisplayName: beginner
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T15:41:53.520Z
Answers.Text: @beginner It’s the other way round: DESeq needs *fewer* replicates than a standard t-test, because it pools variance estimates. This is, in fact, the whole reason why dedicated DE analysis packages exist. With 45 samples a simple t-test should actually work just as well as DESeq though.
Answers.UserDisplayName: Konrad Rudolph
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T15:44:31.460Z
Answers.Text: @KonradRudolph you have mistaken. please check my previous comments.
Answers.UserDisplayName: beginner
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T15:50:38.593Z
Answers.Text: @beginner I’m puzzled. First off, your question mentions 45 replicates, not two (or one). Secondly, how do you suggest performing a t-test on two replicates?! (One replicate doesn’t even make sense … what would you compare against?)
Answers.UserDisplayName: Konrad Rudolph
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T15:55:10.473Z
Answers.Text: No. After doing differential analysis between 40 tumor and 5 normal I get some differential expressed genes. From that I will select strong candidates and do some experiments. For the experiments I have two cell-lines. To select the good cell-line where I can see high expression of those candidates I have to do differential analysis between those two cell-lines. Note: Please have a look at my comments and Devon's comments.
Answers.UserDisplayName: beginner
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-10T13:07:58.930Z
Answers.LastActivityDate: 2018-07-10T13:07:58.930Z
Answers.UserDisplayName: Devon Ryan
Body: <p>I'm working with RNA-seq data. I have 40 tumor samples and 5 Normal samples. Differential analysis with <code>Deseq2</code> based on Fold change > 1.2 and alpha &lt; 0.05 gave very low number of differentially expressed genes. Only 2 upregulated genes. </p>
  
  <pre><code>res &lt;- results(dds, lfcThreshold = log2(1.2), alpha = 0.05)
  </code></pre>
  
  <p>To get more number of differential expressed genes I have few questions now </p>
  
  <ol>
  <li><p>Instead of FDR &lt; 0.05 can I use FDR &lt; 0.1 (or) FDR &lt; 0.5. Will there be any low confidence with this?</p></li>
  <li><p>Can I select differential expressed genes only based on FDR &lt; 0.05 without any fold change cutoff?</p></li>
  <li><p>As I get very low number of DEGs with FDR &lt; 0.05 &amp; Foldchange > 1.2, Can I select DEGs based on Foldchange > 1.2 and p.value &lt; 0.01 or 0.05 ?</p></li>
  </ol>
  
  <p>PCA of the samples looks like this:</p>
  
  <p><a href="https://i.sstatic.net/Gfxbq.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Gfxbq.png" alt="PCA plot Tumor vs Normal"></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-07-10T15:38:27.450Z
Comments.Text: (FDR) ɑ < 0.5 would be completely meaningless. With 40 tumour and 5 normal samples you should get extremely robust results, and I’d suggest that you should use something like ɑ < 0.0001. Given your PCA and your lack of DE, the only valid conclusion is that you really have *no evidence whatsoever* for differential expression. The samples are indistinguishable by the state of the art. This in itself is a striking finding.
Comments.UserDisplayName: Konrad Rudolph
Tags: <r><rna-seq><differential-expression><deseq2><rna>
Title: Selection of differential expressed genes
ThreadText: # Selection of differential expressed genes
  
  (Asked by: username_0 on 2018-07-09)
  
  I'm working with RNA-seq data. I have 40 tumor samples and 5 Normal samples. Differential analysis with `Deseq2` based on Fold change > 1.2 and alpha < 0.05 gave very low number of differentially expressed genes. Only 2 upregulated genes.
  
  ```
  res <- results(dds, lfcThreshold = log2(1.2), alpha = 0.05)
  
  ```
  
  To get more number of differential expressed genes I have few questions now
  
  1. Instead of FDR < 0.05 can I use FDR < 0.1 (or) FDR < 0.5. Will there be any low confidence with this?
  2. Can I select differential expressed genes only based on FDR < 0.05 without any fold change cutoff?
  3. As I get very low number of DEGs with FDR < 0.05 & Foldchange > 1.2, Can I select DEGs based on Foldchange > 1.2 and p.value < 0.01 or 0.05 ?
  
  PCA of the samples looks like this:
  
  [![PCA plot Tumor vs Normal](https://i.sstatic.net/Gfxbq.png)](https://i.sstatic.net/Gfxbq.png)
  
  **Tags:** r, rna-seq, differential-expression, deseq2, rna
  
  > **Comments:**
  > **username_1**: (FDR) ɑ < 0.5 would be completely meaningless. With 40 tumour and 5 normal samples you should get extremely robust results, and I’d suggest that you should use something like ɑ < 0.0001. Given your PCA and your lack of DE, the only valid conclusion is that you really have *no evidence whatsoever* for differential expression. The samples are indistinguishable by the state of the art. This in itself is a striking finding.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_4 on 2018-07-10. Score: 3 | Accepted answer
  
  1. You can usually get away with FDR < 0.1, but that's as high as you can go. This all presumes you're doing follow-up experiments of some sort, of course. I guess you could increase the FDR more, but you're then really increasing the odds that your follow-up experiments will fail. Obviously increasing the FDR will decrease the confidence in the results, it's called "false discovery rate" for a reason.
  2. Yes, fold-change cut-offs are mostly useful when you need to prioritize hits further. Small changes are almost impossible to validate with less-sensitive methods (qPCR) and less likely to be biologically relevant.
  3. You're just going to waste your time if you ignore FDR.
  
  Remember that all thresholds are intended to increase the odds of success in follow-up experiments. If you have infinite time and resources then you can set these to whatever you want, but you have neither so you better prioritize reasonably.
  
  It's likely that you have a large batch effect in your data, effect sizes in cancer tend to be much larger than this.
  
  > **Comments:**
  > **username_0**: Thanks username_4. Yes there will be follow up experiments after selection. So, what would recommend me now? Can I go with FC and FDR < 0.1? Yes, there is batch effect what can I do now? Can I apply subsampling procedure to get more DEGs?  
  > **username_4**: Yes, FDR<0.1 and ideally some reasonable fold-change (say a 30% change). Have a look at packages like `sva` for dealing with your batch effect.  
  > **username_0**: Is it a good idea to apply subsampling procedure for DEA?  
  > **username_4**: @username_0 No, that's not useful, presuming you mean subsampling from the counts.  
  > **username_0**: selecting random samples from cancer cohort and repeating differential analysis n times will give different set of DEGs and then select which DEGs are most repeated in all the analyses. This is the one I'm asking.  
  > **username_4**: No, that's not helpful, you're just approximating the results you would have gotten otherwise.  
  > **username_0**: Sure thank you. And one last question. After selecting candidates..for experiments I have selected two cell-lines.To look which gene is more expressed in which cell line, doing a simple t-test between the cell-lines for those genes is fine or should I go with Deseq2 to check how those genes were expressed?  
  > **username_4**: Use DESeq2, it's more powerful than a T-test.  
  > **username_0**: Yes, it is more powerful but Deseq2 needs replicates for performing differential analysis. It will give a warning/error if tried to analyze data without replicates.  
  > **username_4**: I assumed you were talking about cell lines with replicates. No method will reliably work without replicates.  
  > **username_0**: No, I just have two breast cell-lines. Interested in doing experiments using one cell-line. So I thought of looking at candidates expression in those cell-lines and then select based on that.  
  > **username_1**: @username_0 It’s the other way round: DESeq needs *fewer* replicates than a standard t-test, because it pools variance estimates. This is, in fact, the whole reason why dedicated DE analysis packages exist. With 45 samples a simple t-test should actually work just as well as DESeq though.  
  > **username_0**: @username_1 you have mistaken. please check my previous comments.  
  > **username_1**: @username_0 I’m puzzled. First off, your question mentions 45 replicates, not two (or one). Secondly, how do you suggest performing a t-test on two replicates?! (One replicate doesn’t even make sense … what would you compare against?)  
  > **username_0**: No. After doing differential analysis between 40 tumor and 5 normal I get some differential expressed genes. From that I will select strong candidates and do some experiments. For the experiments I have two cell-lines. To select the good cell-line where I can see high expression of those candidates I have to do differential analysis between those two cell-lines. Note: Please have a look at my comments and username_4's comments.  
  
  ---
  
  ### Answer by username_2 on 2018-07-10. Score: 1
  
  I think since your PCA shows that your normal samples cluster right alongside the tumor ones, you have to conclude that there is much more variation between tumors than between tumors and controls. DESeq's differentially expressed gene list was generated properly.
  
  Your results are what they are, sorry they are so useless.
  
  
  ---
  
  ### Answer by username_3 on 2018-07-09. Score: 0
  
  In general, messing with FDR or log fold change cutoffs beyond the ones you used gets pretty sketchy. I usually only adjust these values in the more conservative direction, although you may be able to get away with FDR < 0.1 if you have a large effect size (which you don't).
  
  You might consider investigating why you don't seem to get any differentially expressed genes before adjusting your cutoffs: PCA on the samples based on their top 1,000 most variably expressed genes would be a good start. Do the tumor and normal transcriptomes cluster apart from each other?
  
  With so few normal samples, it is possible that you are not fully characterizing the variability of that population and it is affecting your ability to identify meaningful differences between normal and tumor.
  
  > **Comments:**
  > **username_0**: Hi, I made the PCA and it is given in the question. please have a look.  
  > **username_1**: If you have a large effect size you wouldn’t need to increase the FDR to 0.1: a lower FDR would still capture the effect.  
  
  ---
  
  