Answers.Body: <p>Snakemake is using bash strict mode to ensure best practice error reporting in shell commands. Unfortunately some tools violate bash strict mode.</p>
  
  <p>More details of the error, <a href="https://snakemake.readthedocs.io/en/stable/project_info/faq.html#my-shell-command-fails-with-exit-code-0-from-within-a-pipe-what-s-wrong" rel="nofollow noreferrer">here</a>. Alternatively, you can use if you don't want disable the strict mode.</p>
  
  <pre><code>zcat file.fastq.gz  | awk '(NR&lt;=400000)' &gt; subset.fastq
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-02T07:42:18.847Z
Answers.Text: This solves my problem exactly (the awk solution), it is fast and does what my original command does but without pipefail error. Thank you.
Answers.UserDisplayName: Freek
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-02T15:40:38.517Z
Answers.Text: @Freek another option would be to add `|| true` to give a 0 exit status
Answers.UserDisplayName: Chris_Rands
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-31T22:10:12.203Z
Answers.LastActivityDate: 2020-03-31T22:10:12.203Z
Answers.UserDisplayName: zorbax
Body: <p>All,</p>
  
  <p>As the first step in my Snakemake pipeline, I generate small fastq files from all input fastq.gz files. This is to run a quick round of "infer_experiment.py" (from RSeQC, to get the kit strandedness). However, Snakemake says this step fails. I think I know why, it is because a command like this</p>
  
  <pre><code>zcat fastq_raw/20190011-043_S43_R1_001.fastq.gz | head -n 400000 &gt; infer_strandedness/20190011-043_S43_subset_R1.fastq
  </code></pre>
  
  <p>Ends with an error: <code>gzip: stdout: Broken pipe</code> and Snakemake uses Bash strict mode. (Snakemake reports: <code>(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)</code>). So my question is:</p>
  
  <p>How to avoid this, or, are there other truncating methods that do finish cleanly? Preferebly they are also efficient (seqtk offers subsampling but this parses the entire fastq file!).</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-31T17:23:12.997Z
Comments.Text: You're better off using seqtk even if it's slower since you're then not performing a highly biased sampling from the beginning of the file, where the quality tends to be worse.
Comments.UserDisplayName: Devon Ryan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-02T07:19:35.240Z
Comments.Text: But the library is randomized during prep and flowing into the chip, why would the first part of the fastq file (first scanned tile of the chip) be biased? A bit worse quality would not be a problem if I only want to detect the strandedness...
Comments.UserDisplayName: Freek
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-02T07:20:57.900Z
Comments.Text: The beginning of the files represent the edge of the flow cell where the quality is lower.
Comments.UserDisplayName: Devon Ryan
Tags: <snakemake>
Title: truncate fastq.gz file without error in Snakemake
ThreadText: # truncate fastq.gz file without error in Snakemake
  
  (Asked by: username_0 on 2020-03-31)
  
  All,
  
  As the first step in my Snakemake pipeline, I generate small fastq files from all input fastq.gz files. This is to run a quick round of "infer\_experiment.py" (from RSeQC, to get the kit strandedness). However, Snakemake says this step fails. I think I know why, it is because a command like this
  
  ```
  zcat fastq_raw/20190011-043_S43_R1_001.fastq.gz | head -n 400000 > infer_strandedness/20190011-043_S43_subset_R1.fastq
  
  ```
  
  Ends with an error: `gzip: stdout: Broken pipe` and Snakemake uses Bash strict mode. (Snakemake reports: `(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)`). So my question is:
  
  How to avoid this, or, are there other truncating methods that do finish cleanly? Preferebly they are also efficient (seqtk offers subsampling but this parses the entire fastq file!).
  
  **Tags:** snakemake
  
  > **Comments:**
  > **username_1**: You're better off using seqtk even if it's slower since you're then not performing a highly biased sampling from the beginning of the file, where the quality tends to be worse.  
  > **username_0**: But the library is randomized during prep and flowing into the chip, why would the first part of the fastq file (first scanned tile of the chip) be biased? A bit worse quality would not be a problem if I only want to detect the strandedness...  
  > **username_1**: The beginning of the files represent the edge of the flow cell where the quality is lower.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-03-31. Score: 1 | Accepted answer
  
  Snakemake is using bash strict mode to ensure best practice error reporting in shell commands. Unfortunately some tools violate bash strict mode.
  
  More details of the error, [here](https://snakemake.readthedocs.io/en/stable/project_info/faq.html#my-shell-command-fails-with-exit-code-0-from-within-a-pipe-what-s-wrong). Alternatively, you can use if you don't want disable the strict mode.
  
  ```
  zcat file.fastq.gz  | awk '(NR<=400000)' > subset.fastq
  
  ```
  
  > **Comments:**
  > **username_0**: This solves my problem exactly (the awk solution), it is fast and does what my original command does but without pipefail error. Thank you.  
  > **username_3**: @username_0 another option would be to add `|| true` to give a 0 exit status  
  
  ---
  
  