Answers.Body: <p>One way would be to use <a href="https://bioinf.shenwei.me/seqkit/usage/#stats" rel="noreferrer"><code>seqkit stats</code></a>, which can be installed using <code>conda</code>:</p>
  <pre><code>conda install -c bioconda seqkit
  </code></pre>
  <p>It has a number of useful options, including ways to parallelize the counting, using <code>-j</code>, <code>--threads</code> (default 4), and to skip errors using <code>-e</code>, <code>--skip-err</code>. The latter can be especially useful if you have a larger number of FASTQ files with millions of reads and you'd like to run the command overnight for example. There's also an option to produce a machine-friendly tabular output, using <code>-T</code>, <code>--tabular</code> which would allow you to parse the output without much effort. I'd be tempted to just write the output to a file using the <code>-o</code>, <code>--out-file</code> option:</p>
  <pre><code>seqkit stats -To stats.tsv *.fastq.gz
  </code></pre>
  <p>Example results:</p>
  <pre><code>$ cat stats.tsv 
  file    format  type    num_seqs    sum_len min_len avg_len max_len
  PHB_Rep1_R1.fastq.gz    FASTQ   RNA 118571  11857100    100 100.0   100
  PHB_Rep1_R2.fastq.gz    FASTQ   RNA 118571  11857100    100 100.0   100
  PHB_Rep2_R1.fastq.gz    FASTQ   RNA 144826  14482600    100 100.0   100
  PHB_Rep2_R2.fastq.gz    FASTQ   RNA 144826  14482600    100 100.0   100
  PHB_Rep3_R1.fastq.gz    FASTQ   RNA 129786  12978600    100 100.0   100
  PHB_Rep3_R2.fastq.gz    FASTQ   RNA 129786  12978600    100 100.0   100
  PUH_Rep1_R1.fastq.gz    FASTQ   RNA 227392  22739200    100 100.0   100
  PUH_Rep1_R2.fastq.gz    FASTQ   RNA 227392  22739200    100 100.0   100
  PUH_Rep2_R1.fastq.gz    FASTQ   RNA 162373  16237300    100 100.0   100
  PUH_Rep2_R2.fastq.gz    FASTQ   RNA 162373  16237300    100 100.0   100
  PUH_Rep3_R1.fastq.gz    FASTQ   RNA 185442  18544200    100 100.0   100
  PUH_Rep3_R2.fastq.gz    FASTQ   RNA 185442  18544200    100 100.0   100
  </code></pre>
  <p>And then just parse the output to get what you need, if or when you need it. For your specific requirements, you could just use AWK to skip the header line and print the first and fourth columns:</p>
  <pre><code>awk 'BEGIN {FS=OFS=&quot;\t&quot; } NR&gt;1 { print $1, $4 }' stats.tsv
  </code></pre>
  <p>Results:</p>
  <pre><code>PHB_Rep1_R1.fastq.gz    118571
  PHB_Rep1_R2.fastq.gz    118571
  PHB_Rep2_R1.fastq.gz    144826
  PHB_Rep2_R2.fastq.gz    144826
  PHB_Rep3_R1.fastq.gz    129786
  PHB_Rep3_R2.fastq.gz    129786
  PUH_Rep1_R1.fastq.gz    227392
  PUH_Rep1_R2.fastq.gz    227392
  PUH_Rep2_R1.fastq.gz    162373
  PUH_Rep2_R2.fastq.gz    162373
  PUH_Rep3_R1.fastq.gz    185442
  PUH_Rep3_R2.fastq.gz    185442
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-04T12:50:39.777Z
Answers.LastActivityDate: 2023-02-04T12:50:39.777Z
Answers.UserDisplayName: Steve
Answers.Body: <p>You can replace <code>gunzip -c</code> with <code>zcat</code>.</p>
     
  <pre class="lang-bash prettyprint-override"><code>#USAGE: script.sh /runFolder
  for i in <span class="math-container">$1/*.1.fastq.gz
  do
     echo "$</span>i&quot; <span class="math-container">$(gunzip -c $</span>i | echo `wc -l`/4 | bc -l)
  done
  
  #OR in the current folder, do this:
  for i in ./*.1.fastq.gz
  do
     echo &quot;<span class="math-container">$i" $</span>(gunzip -c $i | echo `wc -l`/4 | bc -l)
  done
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-04T02:53:35.437Z
Answers.Text: Backticks are legacy syntax. Subshells can be nested for example using: `echo "${i}: $(echo "scale=2; $(zcat "${i}" | wc -l)/4" | bc)"`
Answers.UserDisplayName: Steve
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-04T02:59:15.417Z
Answers.Text: Also, I think the OP is looking for a summary text file, so you could just have your loop append to an output file (i.e. `... >> counts.txt`)
Answers.UserDisplayName: Steve
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-04T00:03:49.843Z
Answers.LastActivityDate: 2023-02-04T00:08:56.743Z
Answers.LastEditDate: 2023-02-04T00:08:56.743Z
Answers.UserDisplayName: Supertech
Answers.Body: <p><code>zcat</code>, like <code>cat</code>, works with multiple file names as arguments, up to the argument limit (on Linux, that's usually at least a thousand), so counting multiple lines in multiple files still works, assuming the fastq files are formatted in the expected way with only four lines per record:</p>
  <pre><code>zcat filename_R1.fastq.gz filename_R2.fastq.gz ... | wc -l
  </code></pre>
  <p>If there's a consistent file pattern, that can be used with file name glob wildcards:</p>
  <pre><code>zcat *.fastq.gz | wc -l
  </code></pre>
  <p>If you have a file containing a list of files, this can be processed as shell arguments as well:</p>
  <pre><code>zcat $(cat list_of_files.txt) | wc -l
  </code></pre>
  <p>For an unlimited number of files, something like GNU parallel can be used to feed file names from piped input. It's also a bit safer for file names containing weird characters in them:</p>
  <pre><code>cat list_of_files.txt | parallel zcat {} | wc -l
  </code></pre>
  <p>... but [after checking the question again] given that you want counts for individual files, the file name needs to be added as well:</p>
  <pre><code>cat list_of_files.txt | parallel echo {} $(zcat {} | wc -l)
  </code></pre>
  <p>The <code>read</code> function provides an alternate way to do the same thing:</p>
  <pre><code>cat list_of_files.txt | while read fileName; do echo <span class="math-container">${fileName} $</span>(zcat ${fileName} | wc -l); done
  </code></pre>
  <p>Assuming you are using bash, dividing by 4 can be done via the arithmetic substitution operator, <code>$((X))</code>:</p>
  <pre><code>cat list_of_files.txt | parallel echo {} <span class="math-container">$(( $</span>(zcat {} | wc -l) / 4 ))
  </code></pre>
  <p>Or with <code>read</code>:</p>
  <pre><code>cat list_of_files.txt | while read fileName; do echo <span class="math-container">${fileName} $</span>(( <span class="math-container">$(zcat $</span>{fileName} | wc -l) / 4 )); done
  </code></pre>
  <p>And, finally, this can be output to a file using <code>&gt;</code>:</p>
  <pre><code>(cat list_of_files.txt | while read fileName; do echo <span class="math-container">${fileName} $</span>(( <span class="math-container">$(zcat $</span>{fileName} | wc -l) / 4 )); done) &gt; fastq_record_counts.txt
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-06T03:17:11.177Z
Answers.Text: You'll need to wrap your commands in single quotes to prevent the dollar variables from expanding. You can also avoid the Useless Use of Cat (UUoC) by redirecting stdin or using the `:::: argfiles` syntax with GNU parallel. To be safe, remember to always quote your variables.
Answers.UserDisplayName: Steve
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-06T11:48:24.957Z
Answers.Text: For `echo`? I don't think it'll make any difference
Answers.UserDisplayName: gringer
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-06T12:31:55.850Z
Answers.Text: Sorry, I mean the `parallel` commands need to be singled quoted: `parallel 'echo {} $(( $(zcat {} | wc -l) / 4 ))' :::: list_of_files.txt`
Answers.UserDisplayName: Steve
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-05T19:56:27.903Z
Answers.LastActivityDate: 2023-02-05T20:24:51.693Z
Answers.LastEditDate: 2023-02-05T20:24:51.693Z
Answers.UserDisplayName: gringer
Answers.Body: <p>If you're looking for a one-liner,</p>
  <pre><code>for i in $(ls *.fastq.gz); do count=$(zcat $i | grep &quot;^@&quot; | wc -l); printf &quot;%b\t%b&quot; $i $count; done &gt; counts.tsv
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-05T19:58:00.037Z
Answers.Text: Note that fastq files can include '@' in the first character of the quality string, so this may produce incorrect results.
Answers.UserDisplayName: gringer
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-06T10:30:14.353Z
Answers.Text: Also, NEVER do `for i in $(ls *)`, that is [very bad practice](https://mywiki.wooledge.org/ParsingLs), will break on even slightly odd file names and is really not needed since a simple `for i in  *.fastq.gz;` is safer, more robust and shorter to write.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-04T17:22:06.437Z
Answers.LastActivityDate: 2023-02-04T17:22:06.437Z
Answers.UserDisplayName: 1ndy
Body: <p>I have generated several FASTQ files and I would like to know the amount of reads for each of them.
  I am planning to run <a href="https://www.bioinformatics.babraham.ac.uk/projects/fastqc/" rel="nofollow noreferrer">FastQC</a> on the files which I know would give me the number of reads per sample but it would be a lot easier for me if I could get one text file with the file name and number of reads for all samples together.</p>
  <p>When I check for one file I normally do <code>zcat filename_R1.fastq.gz | wc -l</code> and then divide by 4.
  Is there a way to do it for all my files at once and get a text file with the file name and reads at the end?</p>
  <p>For example I will have 71 R1.fastq.gz files and I would like to output the amount of reads to a text file. I just meant that I would prefer to use one command instead of having to use the command I mentioned above for each file.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-03T22:26:50.467Z
Comments.Text: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.
Comments.UserDisplayName: Community
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-06T10:26:49.750Z
Comments.Text: Note that fastq files can have more than 1 line of sequence per entry so dividing by 4 is not robust. Make sure to check your file and see if you have cases of multi-line sequences.
Comments.UserDisplayName: terdon
Tags: <fastq><linux><command-line>
Title: How to get a file with the number of reads for several fastq.gz files?
ThreadText: # How to get a file with the number of reads for several fastq.gz files?
  
  (Asked by: username_0 on 2023-02-03)
  
  I have generated several FASTQ files and I would like to know the amount of reads for each of them.
  I am planning to run [FastQC](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/) on the files which I know would give me the number of reads per sample but it would be a lot easier for me if I could get one text file with the file name and number of reads for all samples together.
  
  When I check for one file I normally do `zcat filename_R1.fastq.gz | wc -l` and then divide by 4.
  Is there a way to do it for all my files at once and get a text file with the file name and reads at the end?
  
  For example I will have 71 R1.fastq.gz files and I would like to output the amount of reads to a text file. I just meant that I would prefer to use one command instead of having to use the command I mentioned above for each file.
  
  **Tags:** fastq, linux, command-line
  
  > **Comments:**
  > **username_1**: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.  
  > **username_2**: Note that fastq files can have more than 1 line of sequence per entry so dividing by 4 is not robust. Make sure to check your file and see if you have cases of multi-line sequences.  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_3 on 2023-02-04. Score: 6
  
  One way would be to use [`seqkit stats`](https://bioinf.shenwei.me/seqkit/usage/#stats), which can be installed using `conda`:
  
  ```
  conda install -c bioconda seqkit
  
  ```
  
  It has a number of useful options, including ways to parallelize the counting, using `-j`, `--threads` (default 4), and to skip errors using `-e`, `--skip-err`. The latter can be especially useful if you have a larger number of FASTQ files with millions of reads and you'd like to run the command overnight for example. There's also an option to produce a machine-friendly tabular output, using `-T`, `--tabular` which would allow you to parse the output without much effort. I'd be tempted to just write the output to a file using the `-o`, `--out-file` option:
  
  ```
  seqkit stats -To stats.tsv *.fastq.gz
  
  ```
  
  Example results:
  
  ```
  $ cat stats.tsv 
  file    format  type    num_seqs    sum_len min_len avg_len max_len
  PHB_Rep1_R1.fastq.gz    FASTQ   RNA 118571  11857100    100 100.0   100
  PHB_Rep1_R2.fastq.gz    FASTQ   RNA 118571  11857100    100 100.0   100
  PHB_Rep2_R1.fastq.gz    FASTQ   RNA 144826  14482600    100 100.0   100
  PHB_Rep2_R2.fastq.gz    FASTQ   RNA 144826  14482600    100 100.0   100
  PHB_Rep3_R1.fastq.gz    FASTQ   RNA 129786  12978600    100 100.0   100
  PHB_Rep3_R2.fastq.gz    FASTQ   RNA 129786  12978600    100 100.0   100
  PUH_Rep1_R1.fastq.gz    FASTQ   RNA 227392  22739200    100 100.0   100
  PUH_Rep1_R2.fastq.gz    FASTQ   RNA 227392  22739200    100 100.0   100
  PUH_Rep2_R1.fastq.gz    FASTQ   RNA 162373  16237300    100 100.0   100
  PUH_Rep2_R2.fastq.gz    FASTQ   RNA 162373  16237300    100 100.0   100
  PUH_Rep3_R1.fastq.gz    FASTQ   RNA 185442  18544200    100 100.0   100
  PUH_Rep3_R2.fastq.gz    FASTQ   RNA 185442  18544200    100 100.0   100
  
  ```
  
  And then just parse the output to get what you need, if or when you need it. For your specific requirements, you could just use AWK to skip the header line and print the first and fourth columns:
  
  ```
  awk 'BEGIN {FS=OFS="\t" } NR>1 { print $1, $4 }' stats.tsv
  
  ```
  
  Results:
  
  ```
  PHB_Rep1_R1.fastq.gz    118571
  PHB_Rep1_R2.fastq.gz    118571
  PHB_Rep2_R1.fastq.gz    144826
  PHB_Rep2_R2.fastq.gz    144826
  PHB_Rep3_R1.fastq.gz    129786
  PHB_Rep3_R2.fastq.gz    129786
  PUH_Rep1_R1.fastq.gz    227392
  PUH_Rep1_R2.fastq.gz    227392
  PUH_Rep2_R1.fastq.gz    162373
  PUH_Rep2_R2.fastq.gz    162373
  PUH_Rep3_R1.fastq.gz    185442
  PUH_Rep3_R2.fastq.gz    185442
  
  ```
  
  
  ---
  
  ### Answer by username_4 on 2023-02-04. Score: 1
  
  You can replace `gunzip -c` with `zcat`.
  
  ```bash
  #USAGE: script.sh /runFolder
  for i in $1/*.1.fastq.gz
  do
     echo "$i" $(gunzip -c $i | echo `wc -l`/4 | bc -l)
  done
  
  #OR in the current folder, do this:
  for i in ./*.1.fastq.gz
  do
     echo "$i" $(gunzip -c $i | echo `wc -l`/4 | bc -l)
  done
  
  ```
  
  > **Comments:**
  > **username_3**: Backticks are legacy syntax. Subshells can be nested for example using: `echo "${i}: $(echo "scale=2; $(zcat "${i}" | wc -l)/4" | bc)"`  
  > **username_3**: Also, I think the OP is looking for a summary text file, so you could just have your loop append to an output file (i.e. `... >> counts.txt`)  
  
  ---
  
  ### Answer by username_5 on 2023-02-05. Score: 0
  
  `zcat`, like `cat`, works with multiple file names as arguments, up to the argument limit (on Linux, that's usually at least a thousand), so counting multiple lines in multiple files still works, assuming the fastq files are formatted in the expected way with only four lines per record:
  
  ```
  zcat filename_R1.fastq.gz filename_R2.fastq.gz ... | wc -l
  
  ```
  
  If there's a consistent file pattern, that can be used with file name glob wildcards:
  
  ```
  zcat *.fastq.gz | wc -l
  
  ```
  
  If you have a file containing a list of files, this can be processed as shell arguments as well:
  
  ```
  zcat $(cat list_of_files.txt) | wc -l
  
  ```
  
  For an unlimited number of files, something like GNU parallel can be used to feed file names from piped input. It's also a bit safer for file names containing weird characters in them:
  
  ```
  cat list_of_files.txt | parallel zcat {} | wc -l
  
  ```
  
  ... but [after checking the question again] given that you want counts for individual files, the file name needs to be added as well:
  
  ```
  cat list_of_files.txt | parallel echo {} $(zcat {} | wc -l)
  
  ```
  
  The `read` function provides an alternate way to do the same thing:
  
  ```
  cat list_of_files.txt | while read fileName; do echo ${fileName} $(zcat ${fileName} | wc -l); done
  
  ```
  
  Assuming you are using bash, dividing by 4 can be done via the arithmetic substitution operator, `$((X))`:
  
  ```
  cat list_of_files.txt | parallel echo {} $(( $(zcat {} | wc -l) / 4 ))
  
  ```
  
  Or with `read`:
  
  ```
  cat list_of_files.txt | while read fileName; do echo ${fileName} $(( $(zcat ${fileName} | wc -l) / 4 )); done
  
  ```
  
  And, finally, this can be output to a file using `>`:
  
  ```
  (cat list_of_files.txt | while read fileName; do echo ${fileName} $(( $(zcat ${fileName} | wc -l) / 4 )); done) > fastq_record_counts.txt
  
  ```
  
  > **Comments:**
  > **username_3**: You'll need to wrap your commands in single quotes to prevent the dollar variables from expanding. You can also avoid the Useless Use of Cat (UUoC) by redirecting stdin or using the `:::: argfiles` syntax with GNU parallel. To be safe, remember to always quote your variables.  
  > **username_5**: For `echo`? I don't think it'll make any difference  
  > **username_3**: Sorry, I mean the `parallel` commands need to be singled quoted: `parallel 'echo {} $(( $(zcat {} | wc -l) / 4 ))' :::: list_of_files.txt`  
  
  ---
  
  ### Answer by username_6 on 2023-02-04. Score: -4
  
  If you're looking for a one-liner,
  
  ```
  for i in $(ls *.fastq.gz); do count=$(zcat $i | grep "^@" | wc -l); printf "%b\t%b" $i $count; done > counts.tsv
  
  ```
  
  > **Comments:**
  > **username_5**: Note that fastq files can include '@' in the first character of the quality string, so this may produce incorrect results.  
  > **username_2**: Also, NEVER do `for i in $(ls *)`, that is [very bad practice](https://mywiki.wooledge.org/ParsingLs), will break on even slightly odd file names and is really not needed since a simple `for i in  *.fastq.gz;` is safer, more robust and shorter to write.  
  
  ---
  
  