Answers.Body: <p>If you're looking for the transposition, you could use the <a href="https://www.nextflow.io/docs/latest/operator.html#transpose" rel="nofollow noreferrer"><code>transpose</code></a> operator to get what you want. But to handle the variable output, you would need to first check what types you have. For this, you could use something like the following for example:</p>
  <pre><code>input_ch
      .map { pair_id, plate_id, fastq, gfa, fasta -&gt;
  
          def gfa_files = gfa instanceof Path ? [gfa] : gfa
          def fasta_files = fasta instanceof Path ? [fasta] : fasta
  
          tuple( pair_id, plate_id, fastq, gfa_files, fasta_files )
      }
      .transpose()
      .view()
  </code></pre>
  <p>Note that the old DSL1 is no longer supported.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-04-07T13:30:24.567Z
Answers.LastActivityDate: 2023-04-07T13:30:24.567Z
Answers.UserDisplayName: Steve
Body: <p>I have this output which is going to do bwa next which is from join operator</p>
  <pre><code>[DF122_SO11378_A03, DF122_SO11378, DF122_SO11378_A03.norm.merged.fq, DF122_SO11378_A03.final.gfa, DF122_SO11378_A03.final.fasta]
  </code></pre>
  <p>in this case, <code>pair_id</code> is <code>DF122_SO11378_A03</code>,
  <code>plate_id</code> is <code>DF122_SO11378</code>, <code>fq</code> is <code>DF122_SO11378_A03.norm.merged.fq</code>,
  <code>DF122_SO11378_A03.final.gfa</code>  is <code>gfa</code>
  <code>DF122_SO11378_A03.final.fasta</code>  is <code>fa</code></p>
  <p>I have following code like this</p>
  <pre><code>process bwa {
  
       input:
       tuple val(pair_id), val(plate_id), path(fq), path(gfa), path(fa) from align_in
  
       output:
       path &quot;*bam&quot; into bam
       tuple val(pair_id), val(plate_id), path (&quot;*.csv&quot;) into metrics
  
       &quot;&quot;&quot;
       if [ -s <span class="math-container">$fa ]; then
       bwa index $</span>fa
  
       bwa mem <span class="math-container">$fa $</span>fq \
        | samtools view -bh -F2048 - \
        | samtools sort &gt; ${pair_id}.bam
  
       samtools depth -a <span class="math-container">${pair_id}.bam &gt; $</span>{pair_id}.depth.csv
       samtools view -c <span class="math-container">${pair_id}.bam &gt;$</span>{pair_id}.count.csv
       
       else
       touch <span class="math-container">${pair_id}.depth.csv
       touch $</span>{pair_id}.count.csv
       touch ${pair_id}.bam
       fi
       &quot;&quot;&quot;
  }
  </code></pre>
  <p>However, now my upstream work flow changed a little, for some sample they have more than one gfa and fasta output like this</p>
  <pre><code>[DF122_SO11378_A04, DF122_SO11378, DF122_SO11378_A04.norm.merged.fq, [DF122_SO11378_A04.2.final.gfa, DF122_SO11378_A04.final.gfa], [DF122_SO11378_A04.2.final.fasta, DF122_SO11378_A04.final.fasta]]
  </code></pre>
  <p>in this case, <code>pair_id</code> is <code>DF122_SO11378_A04</code>,
  <code>plate_id</code> is <code>DF122_SO11378</code>, <code>fq</code> is <code>DF122_SO11378_A04.norm.merged.fq</code>,
  <code>[DF122_SO11378_A04.2.final.gfa, DF122_SO11378_A04.final.gfa]</code> is <code>gfa</code>,
  <code>[DF122_SO11378_A04.2.final.fasta, DF122_SO11378_A04.final.fasta]</code> is <code>fa</code></p>
  <p>But some samples still have one gfa and fasta as shown earlier.</p>
  <p>How can I adapt my bwa process to handle such situations?</p>
  <p>I am using DSL1.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-07T12:44:57.597Z
Comments.Text: I think I can use transpose operator to make the list flat, and use the downstream pipeline. So the problem is currently solved.
Comments.UserDisplayName: cautree
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-07T13:04:16.473Z
Comments.Text: You can put the details in an answer, and then accept your own answer as the solution.  That'll help anyone in the same situation later.
Comments.UserDisplayName: Jesse
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-07T13:44:05.800Z
Comments.Text: Note that there's no reference to the GFA file in the script block of the process. Also, testing if the FASTA file is empty is probably unnecessary. Ideally, this should be performed upstream so as to not execute tasks that just create empty (placeholder) files.
Comments.UserDisplayName: Steve
Tags: <nextflow>
Title: how to handle variable output from nextflow
ThreadText: # how to handle variable output from nextflow
  
  (Asked by: username_0 on 2023-04-07)
  
  I have this output which is going to do bwa next which is from join operator
  
  ```
  [DF122_SO11378_A03, DF122_SO11378, DF122_SO11378_A03.norm.merged.fq, DF122_SO11378_A03.final.gfa, DF122_SO11378_A03.final.fasta]
  
  ```
  
  in this case, `pair_id` is `DF122_SO11378_A03`,
  `plate_id` is `DF122_SO11378`, `fq` is `DF122_SO11378_A03.norm.merged.fq`,
  `DF122_SO11378_A03.final.gfa` is `gfa`
  `DF122_SO11378_A03.final.fasta` is `fa`
  
  I have following code like this
  
  ```
  process bwa {
  
       input:
       tuple val(pair_id), val(plate_id), path(fq), path(gfa), path(fa) from align_in
  
       output:
       path "*bam" into bam
       tuple val(pair_id), val(plate_id), path ("*.csv") into metrics
  
       """
       if [ -s $fa ]; then
       bwa index $fa
  
       bwa mem $fa $fq \
        | samtools view -bh -F2048 - \
        | samtools sort > ${pair_id}.bam
  
       samtools depth -a ${pair_id}.bam > ${pair_id}.depth.csv
       samtools view -c ${pair_id}.bam >${pair_id}.count.csv
       
       else
       touch ${pair_id}.depth.csv
       touch ${pair_id}.count.csv
       touch ${pair_id}.bam
       fi
       """
  }
  
  ```
  
  However, now my upstream work flow changed a little, for some sample they have more than one gfa and fasta output like this
  
  ```
  [DF122_SO11378_A04, DF122_SO11378, DF122_SO11378_A04.norm.merged.fq, [DF122_SO11378_A04.2.final.gfa, DF122_SO11378_A04.final.gfa], [DF122_SO11378_A04.2.final.fasta, DF122_SO11378_A04.final.fasta]]
  
  ```
  
  in this case, `pair_id` is `DF122_SO11378_A04`,
  `plate_id` is `DF122_SO11378`, `fq` is `DF122_SO11378_A04.norm.merged.fq`,
  `[DF122_SO11378_A04.2.final.gfa, DF122_SO11378_A04.final.gfa]` is `gfa`,
  `[DF122_SO11378_A04.2.final.fasta, DF122_SO11378_A04.final.fasta]` is `fa`
  
  But some samples still have one gfa and fasta as shown earlier.
  
  How can I adapt my bwa process to handle such situations?
  
  I am using DSL1.
  
  **Tags:** nextflow
  
  > **Comments:**
  > **username_0**: I think I can use transpose operator to make the list flat, and use the downstream pipeline. So the problem is currently solved.  
  > **username_1**: You can put the details in an answer, and then accept your own answer as the solution.  That'll help anyone in the same situation later.  
  > **username_2**: Note that there's no reference to the GFA file in the script block of the process. Also, testing if the FASTA file is empty is probably unnecessary. Ideally, this should be performed upstream so as to not execute tasks that just create empty (placeholder) files.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-04-07. Score: 1 | Accepted answer
  
  If you're looking for the transposition, you could use the [`transpose`](https://www.nextflow.io/docs/latest/operator.html#transpose) operator to get what you want. But to handle the variable output, you would need to first check what types you have. For this, you could use something like the following for example:
  
  ```
  input_ch
      .map { pair_id, plate_id, fastq, gfa, fasta ->
  
          def gfa_files = gfa instanceof Path ? [gfa] : gfa
          def fasta_files = fasta instanceof Path ? [fasta] : fasta
  
          tuple( pair_id, plate_id, fastq, gfa_files, fasta_files )
      }
      .transpose()
      .view()
  
  ```
  
  Note that the old DSL1 is no longer supported.
  
  
  ---
  
  