Answers.Body: <p>If you have your counts in a data.frame called <code>counts</code>, something like this might work:</p>
  
  <pre><code>filtered.counts &lt;- counts[rowSums(counts==0)&lt;10, ]
  </code></pre>
  
  <p>For example lets assume the following data frame.</p>
  
  <pre><code>&gt; A &lt;- c(0,0,0,0,0)
  &gt; B &lt;- c(0,1,0,0,0)
  &gt; C &lt;- c(0,2,0,2,0)
  &gt; D &lt;- c(0,5,1,1,2)
  &gt; 
  &gt; counts &lt;- data.frame(A=A, B=B, C=C, D=D)
  &gt; counts
    A B C D
  1 0 0 0 0
  2 0 1 2 5
  3 0 0 0 1
  4 0 0 2 1
  5 0 0 0 2
  </code></pre>
  
  <p>And now you would want to remove the genes with at least 3 zeroes or more:</p>
  
  <pre><code>&gt; filtered.counts &lt;- counts[rowSums(counts==0)&lt;3, ]
  &gt; 
  &gt; filtered.counts
    A B C D
  2 0 1 2 5
  4 0 0 2 1
  </code></pre>
  
  <p>Every gene with 3 or more zeroes is now removed.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-01T12:29:36.443Z
Answers.Text: In at least 10 samples. So I guess it should be this filtered.counts <- counts[rowSums(counts==0)>=10, ]
Answers.UserDisplayName: stack_learner
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-01T12:32:13.353Z
Answers.Text: No, you keep all rows with less than 10 zeroes with my example.
Answers.UserDisplayName: benn
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-01T12:38:08.597Z
Answers.Text: In your way I will be having only 1/4 th of genes remaining. I want to remove genes whose expression is == 0 in at least 10 samples. With my command now I have more than half genes remaining after filtering.
Answers.UserDisplayName: stack_learner
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-01T12:50:07.220Z
Answers.Text: If you use it your way, you keep all the genes with more than 10 zeroes, check it for your self if you don't believe me.
Answers.UserDisplayName: benn
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-01T13:02:49.403Z
Answers.Text: So, the filtered.counts output that u gave above should be used for normalization?
Answers.UserDisplayName: stack_learner
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-01T13:06:01.577Z
Answers.Text: If this filtering approach is what you want yes. However, in edgeR or limma manual they also give other options, such as at least 1 cpm in more than 3 samples. Usually 3 samples is then based on groups of 3 samples. If you have only 1/4 of your genes left with this filtering it might be too stringent. For limma voom, you need to have cpm values btw.
Answers.UserDisplayName: benn
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-01T13:21:25.790Z
Answers.Text: Ok. As you said I did it in edgeR way. 
  keep <- rowSums(cpm(counts) > 0.5) >= 10; y <- counts[keep,] So, now genes in "y" can be used for "voom" function to normalize the data. Do you think this is right?
Answers.UserDisplayName: stack_learner
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-01T13:25:54.210Z
Answers.Text: Yes I think it is right, good luck!
Answers.UserDisplayName: benn
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-01T13:34:57.567Z
Answers.Text: 10 out of 1200 samples is a very strict cut off. I'm not surprised you don't have many genes remaining.
Answers.UserDisplayName: Ian Sudbery
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-01T16:45:27.050Z
Answers.Text: Yes, but I'm interested in looking for a specific gene. So, to get that in my results I took that cutoff.
Answers.UserDisplayName: stack_learner
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-01T12:25:30.070Z
Answers.LastActivityDate: 2017-09-01T12:46:37.233Z
Answers.LastEditDate: 2017-09-01T12:46:37.233Z
Answers.UserDisplayName: benn
Answers.Body: <p>While <a href="https://bioinformatics.stackexchange.com/a/2441/48">this answers</a> explains how to do it I want to address when and why and which thresholds to do it.</p>
  
  <p>Filtering the genes with low counts is usually done because the counts are not reliable it would be noise, specially when there are low number of samples these genes disturb power of the analysis. However with 1200 samples having 10 or more samples without any count is valuable information because you have enough samples to estimate if this is due to a latent or known variable or if it is due to noise. </p>
  
  <p>The CPM are used because the size of the library per sample affects the power of the experiment. The higher the library size the easier is to find if a gene is expressed or not. CPM are calculated to take into account that effect. Filtering by counts or CPM will result in different genes filtered if you use other thresholds than 0 (for instance I have filtered those genes that the average CPM where below 1, which would be different to filtering by those that the average counts is below 1.) </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-02T10:32:29.980Z
Answers.LastActivityDate: 2017-09-02T10:32:29.980Z
Answers.UserDisplayName: llrs
Body: <p>I have around 1200 samples as columns and 60,000 genes with Htseq-Counts data. Before normalization with voom function I want to do filtering step. </p>
  
  <p>I want to remove genes whose expression is == 0 in at least 10 samples.</p>
  
  <p>Can I do this with read counts itself or should I transform counts to cpm?</p>
  
  <p>If filtering with counts data is fine May I know how to do this.</p>
  
Tags: <r><rna-seq><reads><filtering>
Title: Filtering step for read counts data
ThreadText: # Filtering step for read counts data
  
  (Asked by: username_0 on 2017-09-01)
  
  I have around 1200 samples as columns and 60,000 genes with Htseq-Counts data. Before normalization with voom function I want to do filtering step.
  
  I want to remove genes whose expression is == 0 in at least 10 samples.
  
  Can I do this with read counts itself or should I transform counts to cpm?
  
  If filtering with counts data is fine May I know how to do this.
  
  **Tags:** r, rna-seq, reads, filtering
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2017-09-01. Score: 8
  
  If you have your counts in a data.frame called `counts`, something like this might work:
  
  ```
  filtered.counts <- counts[rowSums(counts==0)<10, ]
  
  ```
  
  For example lets assume the following data frame.
  
  ```
  > A <- c(0,0,0,0,0)
  > B <- c(0,1,0,0,0)
  > C <- c(0,2,0,2,0)
  > D <- c(0,5,1,1,2)
  > 
  > counts <- data.frame(A=A, B=B, C=C, D=D)
  > counts
    A B C D
  1 0 0 0 0
  2 0 1 2 5
  3 0 0 0 1
  4 0 0 2 1
  5 0 0 0 2
  
  ```
  
  And now you would want to remove the genes with at least 3 zeroes or more:
  
  ```
  > filtered.counts <- counts[rowSums(counts==0)<3, ]
  > 
  > filtered.counts
    A B C D
  2 0 1 2 5
  4 0 0 2 1
  
  ```
  
  Every gene with 3 or more zeroes is now removed.
  
  > **Comments:**
  > **username_0**: In at least 10 samples. So I guess it should be this filtered.counts <- counts[rowSums(counts==0)>=10, ]  
  > **username_1**: No, you keep all rows with less than 10 zeroes with my example.  
  > **username_0**: In your way I will be having only 1/4 th of genes remaining. I want to remove genes whose expression is == 0 in at least 10 samples. With my command now I have more than half genes remaining after filtering.  
  > **username_1**: If you use it your way, you keep all the genes with more than 10 zeroes, check it for your self if you don't believe me.  
  > **username_0**: So, the filtered.counts output that u gave above should be used for normalization?  
  > **username_1**: If this filtering approach is what you want yes. However, in edgeR or limma manual they also give other options, such as at least 1 cpm in more than 3 samples. Usually 3 samples is then based on groups of 3 samples. If you have only 1/4 of your genes left with this filtering it might be too stringent. For limma voom, you need to have cpm values btw.  
  > **username_0**: Ok. As you said I did it in edgeR way.  keep <- rowSums(cpm(counts) > 0.5) >= 10; y <- counts[keep,] So, now genes in "y" can be used for "voom" function to normalize the data. Do you think this is right?  
  > **username_1**: Yes I think it is right, good luck!  
  > **username_2**: 10 out of 1200 samples is a very strict cut off. I'm not surprised you don't have many genes remaining.  
  > **username_0**: Yes, but I'm interested in looking for a specific gene. So, to get that in my results I took that cutoff.  
  
  ---
  
  ### Answer by username_3 on 2017-09-02. Score: 3
  
  While [this answers](https://bioinformatics.stackexchange.com/a/2441/48) explains how to do it I want to address when and why and which thresholds to do it.
  
  Filtering the genes with low counts is usually done because the counts are not reliable it would be noise, specially when there are low number of samples these genes disturb power of the analysis. However with 1200 samples having 10 or more samples without any count is valuable information because you have enough samples to estimate if this is due to a latent or known variable or if it is due to noise.
  
  The CPM are used because the size of the library per sample affects the power of the experiment. The higher the library size the easier is to find if a gene is expressed or not. CPM are calculated to take into account that effect. Filtering by counts or CPM will result in different genes filtered if you use other thresholds than 0 (for instance I have filtered those genes that the average CPM where below 1, which would be different to filtering by those that the average counts is below 1.)
  
  
  ---
  
  