Answers.Body: <p>Each nextflow process looks in its working directory for the files listed in <code>output:</code> . In your <code>script:</code> you already direct the output to its eventual destination. After running the script nextflow tries to find the output files and stops, because it can't find them. Solution: keep the output files in your workdir, nextflow will detect them and use the <code>publishDir</code> directive to copy  them to their eventual destination.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-12T11:37:49.310Z
Answers.Text: ok perfect thanks
Answers.UserDisplayName: Grendel
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-12T11:25:28.027Z
Answers.LastActivityDate: 2023-09-12T11:25:28.027Z
Answers.UserDisplayName: Pallie
Body: <p>I have one file with multiple SRA accession called <code>Dataframe_with_accession.txt</code>, for this example I just put one SRA in that file called :<code>SRR8933535</code></p>
  <p>And the idea is to create a nextflow pipeline to download the sra files in the <code>Dataframe_with_accession.txt</code>, zip the files  using <code>pigz</code> and remove the previous <code>fastq file</code></p>
  <p>To do that I used this code :</p>
  <pre><code>params.inputFile = &quot;/crex/proj/Dataframe_with_accession.txt&quot;
  params.outputDir = &quot;/crex/proj/Output&quot;
  
  
  process DOWNLOAD_FASTQ {
  
      publishDir params.outputDir, mode: 'symlink'
  
      input:
      each (accession)
  
      output:
      tuple path(&quot;params.outputDir/<span class="math-container">${accession}_1.fastq.gz")
      path("params.outputDir/$</span>{accession}_2.fastq.gz&quot;)
  
      script:
      &quot;&quot;&quot;
      fasterq-dump --threads 12 --outdir <span class="math-container">${params.outputDir}  $</span>{accession}
      pigz -p12 <span class="math-container">${params.outputDir}/$</span>{accession}_1.fastq
      pigz -p12 <span class="math-container">${params.outputDir}/$</span>{accession}_2.fastq
      &quot;&quot;&quot;
  }
  
  workflow {
      // Read accessions from the input file
      accList = file(params.inputFile).readLines()
  
      DOWNLOAD_FASTQ(accList)
  }
  </code></pre>
  <p>Then I run this nextflow file.</p>
  <p>But I got the following error message :</p>
  <pre><code>[01/298f4b] process &gt; DOWNLOAD_FASTQ (1) [100%] 1 of 1, failed: 1 ✘
  Error executing process &gt; 'DOWNLOAD_FASTQ (1)'
  
  Caused by:
    Missing output file(s) `SRR8933535_1.fastq` expected by process `DOWNLOAD_FASTQ (1)`
  
  Command executed:
  
    fasterq-dump --threads 12 --outdir /crex/proj/Output  SRR8933535
  
  Command exit status:
    0
  
  Command output:
    (empty)
  
  Command error:
    spots read      : 8,389,148
    reads read      : 16,778,296
    reads written   : 16,778,296
  
  Work dir:
    /crex/proj/Output/work/01/298f4be6dde2ee1856df31db169bec
  
  Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
  </code></pre>
  <p>And I do not undestrand what is going wrong here ? It says</p>
  <pre><code>Command output:
    (empty)
  </code></pre>
  <p>But the file is created</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-09-12T13:36:15.990Z
Comments.Text: Are you aware of the fromSRA Nextflow channel factory? https://www.nextflow.io/docs/edge/channel.html#fromsra
Comments.UserDisplayName: mribeirodantas
Tags: <nextflow><fastq-dump>
Title: Error output in nextflow pipeline using fasterq-dump
ThreadText: # Error output in nextflow pipeline using fasterq-dump
  
  (Asked by: username_0 on 2023-09-11)
  
  I have one file with multiple SRA accession called `Dataframe_with_accession.txt`, for this example I just put one SRA in that file called :`SRR8933535`
  
  And the idea is to create a nextflow pipeline to download the sra files in the `Dataframe_with_accession.txt`, zip the files using `pigz` and remove the previous `fastq file`
  
  To do that I used this code :
  
  ```
  params.inputFile = "/crex/proj/Dataframe_with_accession.txt"
  params.outputDir = "/crex/proj/Output"
  
  process DOWNLOAD_FASTQ {
  
      publishDir params.outputDir, mode: 'symlink'
  
      input:
      each (accession)
  
      output:
      tuple path("params.outputDir/${accession}_1.fastq.gz")
      path("params.outputDir/${accession}_2.fastq.gz")
  
      script:
      """
      fasterq-dump --threads 12 --outdir ${params.outputDir}  ${accession}
      pigz -p12 ${params.outputDir}/${accession}_1.fastq
      pigz -p12 ${params.outputDir}/${accession}_2.fastq
      """
  }
  
  workflow {
      // Read accessions from the input file
      accList = file(params.inputFile).readLines()
  
      DOWNLOAD_FASTQ(accList)
  }
  
  ```
  
  Then I run this nextflow file.
  
  But I got the following error message :
  
  ```
  [01/298f4b] process > DOWNLOAD_FASTQ (1) [100%] 1 of 1, failed: 1 ✘
  Error executing process > 'DOWNLOAD_FASTQ (1)'
  
  Caused by:
    Missing output file(s) `SRR8933535_1.fastq` expected by process `DOWNLOAD_FASTQ (1)`
  
  Command executed:
  
    fasterq-dump --threads 12 --outdir /crex/proj/Output  SRR8933535
  
  Command exit status:
    0
  
  Command output:
    (empty)
  
  Command error:
    spots read      : 8,389,148
    reads read      : 16,778,296
    reads written   : 16,778,296
  
  Work dir:
    /crex/proj/Output/work/01/298f4be6dde2ee1856df31db169bec
  
  Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
  
  ```
  
  And I do not undestrand what is going wrong here ? It says
  
  ```
  Command output:
    (empty)
  
  ```
  
  But the file is created
  
  **Tags:** nextflow, fastq-dump
  
  > **Comments:**
  > **username_1**: Are you aware of the fromSRA Nextflow channel factory? https://www.nextflow.io/docs/edge/channel.html#fromsra  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-09-12. Score: 3 | Accepted answer
  
  Each nextflow process looks in its working directory for the files listed in `output:` . In your `script:` you already direct the output to its eventual destination. After running the script nextflow tries to find the output files and stops, because it can't find them. Solution: keep the output files in your workdir, nextflow will detect them and use the `publishDir` directive to copy them to their eventual destination.
  
  > **Comments:**
  > **username_0**: ok perfect thanks  
  
  ---
  
  