Answers.Body: <p>This is not a Nextflow error. The problem is that without an input file, samtools <a href="https://www.htslib.org/doc/samtools-sort.html" rel="nofollow noreferrer"><code>sort</code></a> tries to read from stdin. Using a recent <code>samtools</code>, you can however coordinate sort the SAM and write a sorted BAM using:</p>
  <pre><code>samtools sort -o &quot;${baseName}.bam&quot; &quot;mapped_${baseName}.sam&quot;
  </code></pre>
  <p>You may have been intending to pipe the output to <code>samtools sort</code>, which would avoid writing large SAM files and is usually preferable. A minimal example might look like:</p>
  <pre><code>params.reads = './*.fastq.gz'
  params.reference = './gencode.v43.transcripts.fa'
  </code></pre>
  <pre><code>process minimap2 {
  
      tag { sample }
  
      cpus 8
      memory 20.GB
  
      input:
      tuple val(sample), path(fastq)
      path ref_fasta
  
      output:
      tuple val(sample), path(&quot;${sample}.bam&quot;)
  
      script:
      def cpus = task.cpus &gt; 1 ? task.cpus - 1 : task.cpus
  
      &quot;&quot;&quot;
      minimap2 \\
          -ax splice \\
          -uf \\
          -k14 \\
          -t <span class="math-container">${cpus} \\
          "$</span>{ref_fasta}&quot; \\
          &quot;<span class="math-container">${fastq}" |
      samtools sort \\
          -o "$</span>{sample}.bam&quot; 
      &quot;&quot;&quot;
  }
  </code></pre>
  <pre><code>workflow {
  
      reads = Channel.fromFilePairs( params.reads, size: 1 )
      reference = file( params.reference )
  
      minimap2( reads, reference )
  }
  </code></pre>
  <p>Also, be sure to avoid using absolute paths in your workflow scripts. For your workflow to be portable, processes should only access files in the working directory. The example above avoids this, by using a <a href="https://www.nextflow.io/docs/latest/channel.html#value-channel" rel="nofollow noreferrer">value channel</a>, to stage the reference FASTA file into the working directory.</p>
  <p>In case anyone is wondering, the ternary is to ensure that we do not oversubscribe the node(s) that these jobs land on. Otherwise, each job could use (up to) 900% (800% + 100% = 900%) CPU.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-23T03:28:44.053Z
Answers.LastActivityDate: 2023-03-23T03:58:42.217Z
Answers.LastEditDate: 2023-03-23T03:58:42.217Z
Answers.UserDisplayName: Steve
Body: <p>I am trying to run my nextflow pipeline, and have gotten this error:<br />
  <code>samtools sort: failed to read header from &quot;-&quot;</code></p>
  <p>I'm not sure why this error is occurring.</p>
  <pre><code>#!/usr/bin/env nextflow
  nextflow.enable.dsl=2
  import groovy.io.*
  
  params.t = 8
  
  process joinFastq {     
  input:
  path(ds_directory)
  
  output:
  tuple path(&quot;joined_<span class="math-container">${ds_directory.baseName}.fastq"), val("$</span>ds_directory.baseName&quot;)
  
  script:
  &quot;&quot;&quot;
  gunzip -c <span class="math-container">${ds_directory}/*/fastq_pass/*.fastq.gz &gt; 
   joined_$</span>{ds_directory.baseName}.fastq
  &quot;&quot;&quot;
  
  }
   process minimap2 {
  cpus 8
  memory &quot;20 GB&quot;
  input:
  tuple path(fastq), val(baseName)
  
  output:
  tuple path(&quot;*.bam&quot;), val(baseName)
  
    script:
     &quot;&quot;&quot;
    minimap2 -ax splice -uf -k14 -t 8 <span class="math-container">${params.genome} $</span>{fastq} &gt; mapped_<span class="math-container">${baseName}.sam
      samtools view -h -b mapped_$</span>{baseName}.sam &gt; <span class="math-container">${baseName}.bam
     samtools sort -o $</span>{baseName}.bam &gt; ${baseName}_sorted.bam
     &quot;&quot;&quot;
     }
  </code></pre>
  <p>I have tried applying the <code>-h</code> flag to <code>samtools sort</code> in the script</p>
  <pre><code>   samtools sort -h -o <span class="math-container">${baseName}.bam &gt; $</span>{baseName}_sorted.bam
  </code></pre>
  <p>and the same error is produced. I'm not sure why the header flag didn't solve this issue.</p>
  
Tags: <rna-seq><samtools><nextflow>
Title: Nextflow Error: failed to read header from "-"
ThreadText: # Nextflow Error: failed to read header from "-"
  
  (Asked by: username_0 on 2023-03-23)
  
  I am trying to run my nextflow pipeline, and have gotten this error:  
  `samtools sort: failed to read header from "-"`
  
  I'm not sure why this error is occurring.
  
  ```
  #!/usr/bin/env nextflow
  nextflow.enable.dsl=2
  import groovy.io.*
  
  params.t = 8
  
  process joinFastq {     
  input:
  path(ds_directory)
  
  output:
  tuple path("joined_${ds_directory.baseName}.fastq"), val("$ds_directory.baseName")
  
  script:
  """
  gunzip -c ${ds_directory}/*/fastq_pass/*.fastq.gz > 
   joined_${ds_directory.baseName}.fastq
  """
  
  }
   process minimap2 {
  cpus 8
  memory "20 GB"
  input:
  tuple path(fastq), val(baseName)
  
  output:
  tuple path("*.bam"), val(baseName)
  
    script:
     """
    minimap2 -ax splice -uf -k14 -t 8 ${params.genome} ${fastq} > mapped_${baseName}.sam
      samtools view -h -b mapped_${baseName}.sam > ${baseName}.bam
     samtools sort -o ${baseName}.bam > ${baseName}_sorted.bam
     """
     }
  
  ```
  
  I have tried applying the `-h` flag to `samtools sort` in the script
  
  ```
     samtools sort -h -o ${baseName}.bam > ${baseName}_sorted.bam
  
  ```
  
  and the same error is produced. I'm not sure why the header flag didn't solve this issue.
  
  **Tags:** rna-seq, samtools, nextflow
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-03-23. Score: 3 | Accepted answer
  
  This is not a Nextflow error. The problem is that without an input file, samtools [`sort`](https://www.htslib.org/doc/samtools-sort.html) tries to read from stdin. Using a recent `samtools`, you can however coordinate sort the SAM and write a sorted BAM using:
  
  ```
  samtools sort -o "${baseName}.bam" "mapped_${baseName}.sam"
  
  ```
  
  You may have been intending to pipe the output to `samtools sort`, which would avoid writing large SAM files and is usually preferable. A minimal example might look like:
  
  ```
  params.reads = './*.fastq.gz'
  params.reference = './gencode.v43.transcripts.fa'
  
  ```
  
  ```
  process minimap2 {
  
      tag { sample }
  
      cpus 8
      memory 20.GB
  
      input:
      tuple val(sample), path(fastq)
      path ref_fasta
  
      output:
      tuple val(sample), path("${sample}.bam")
  
      script:
      def cpus = task.cpus > 1 ? task.cpus - 1 : task.cpus
  
      """
      minimap2 \\
          -ax splice \\
          -uf \\
          -k14 \\
          -t ${cpus} \\
          "${ref_fasta}" \\
          "${fastq}" |
      samtools sort \\
          -o "${sample}.bam" 
      """
  }
  
  ```
  
  ```
  workflow {
  
      reads = Channel.fromFilePairs( params.reads, size: 1 )
      reference = file( params.reference )
  
      minimap2( reads, reference )
  }
  
  ```
  
  Also, be sure to avoid using absolute paths in your workflow scripts. For your workflow to be portable, processes should only access files in the working directory. The example above avoids this, by using a [value channel](https://www.nextflow.io/docs/latest/channel.html#value-channel), to stage the reference FASTA file into the working directory.
  
  In case anyone is wondering, the ternary is to ensure that we do not oversubscribe the node(s) that these jobs land on. Otherwise, each job could use (up to) 900% (800% + 100% = 900%) CPU.
  
  
  ---
  
  