Answers.Body: <p>The Problem is, that your ID is called <code>sample.fastq.gz</code> and you tell snakemake to look for <code>{ID}.fastq.gz</code>. Therefor you get the error <code>sample.fastq.gz.fastq.gz</code> can not be found.</p>
  <p>Thus you have two options. Either remove the <em>.fastq.gz</em>-suffix in your <em>IDs.txt</em> or remove it in the snakefile.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-08-16T08:22:39.687Z
Answers.LastActivityDate: 2024-08-22T15:23:32.077Z
Answers.LastEditDate: 2024-08-22T15:23:32.077Z
Answers.UserDisplayName: kEks
Body: <p>I am trying to run a snakemake pipeline for small-RNA samples and having  problems. The <code>snakefile</code> is given below,</p>
  <pre><code>configfile: &quot;config.yaml&quot;
  #include: &quot;utils.py&quot;
  #Command line syntax:
  #snakemake -s snakefile.smk --cores 8 --config prefix= lib=
  class MyException(Exception):
      def __init__(self, message):
          self.message = message
          super().__init__(self.message)
  
  with open(config['IDs'], 'r') as file:
      # Read all the lines of the file into a list
      lines = file.read().splitlines()
  rule all: 
      input:
          expand(&quot;fastq/{prefix}.fastq.gz&quot;, prefix=lines),
          expand(&quot;trimfastq/{prefix}/{prefix}_trimmed.fq.gz&quot;, prefix = lines),
          expand(&quot;trimfastq/{prefix}&quot;, prefix = lines),
          expand(&quot;reportfalco/{prefix}&quot;, prefix=lines),
          expand(&quot;sam/{prefix}Aligned.out.bam&quot;, prefix = lines),
          expand(&quot;sam/{prefix}.s.bam&quot;, prefix=lines),
          expand(&quot;sam/{prefix}.s.bam.bai&quot;, prefix=lines) 
  
  #Checking fastq quality
  rule testquatility:
      input:
          fq = &quot;fastq/{prefix}.fastq.gz&quot;
      output:
          falrep = directory(&quot;reportfalco/{prefix}&quot;)
      shell:
          &quot;&quot;&quot;
          /home/tools/Falco/falco/bin/falco {input.fq} -o {output.falrep}
          &quot;&quot;&quot;
  #Trimming
  rule trimming:
      input:
          fq_in = &quot;fastq/{prefix}.fastq.gz&quot;
      output:
          trimdir = directory(&quot;trimfastq/{prefix}&quot;),
          fq_out = &quot;trimfastq/{prefix}/{prefix}_trimmed.fq.gz&quot;
      run:
          if config['Lib']=='nextflex':
              shell(&quot;/home/tools/TrimGalore-0.6.6/trim_galore -a TGGAATTCTCGGGTGCCAAGG --clip_R1 4 --three_prime_clip_R1 4 --length 18 {input.fq_in} --output_dir {output.trimdir}&quot;)
          elif config['Lib']=='comboseq':
              shell(&quot;/home/tools/TrimGalore-0.6.6/trim_galore -a AAAAAAAA --length 15 {input.fq_in} --output_dir {output.trimdir}&quot;)
          elif config['Lib']=='trilink':
              shell(&quot;/home/tools/TrimGalore-0.6.6/trim_galore -a TCACCGACTGCCATAGAG --small_rna {input.fq_in} --output_dir {output.trimdir}&quot;)
          else:
              raise MyException(&quot;Library strategy is expected to be either: trilink, comboseq or nextflex; but is&quot; + str(config['Lib']))
              with MyException as e:
                  print(&quot;\033[91mError when we try to execute trimmimg :&quot;, e.message, &quot;\033[0m&quot;)
  
  #Alignment
  rule align:
      input:
          ref=&quot;/references/DERFINDER_ref/STAR_INDEX&quot;,
          trim_in=&quot;trimfastq/{prefix}/{prefix}_trimmed.fq.gz&quot;
      params:
          pr=&quot;sam/{prefix}&quot;
      output:
          &quot;sam/{prefix}Aligned.out.bam&quot;
      run:
          if config['Lib']=='nextflex' or config['Lib']=='trilink':
              shell(&quot;/home/tools/STAR-2.7.9a/bin/Linux_x86_64/STAR --runThreadN 16 --genomeDir {input.ref} --readFilesIn {input.trim_in} --readFilesCommand gunzip -c --outFileNamePrefix {params.pr} --outFilterScoreMinOverLread 0 --outFilterMatchNmin 16 --outFilterMatchNminOverLread 0 --outFilterMismatchNoverLmax 0.025 --alignIntronMax 1 --alignEndsType EndToEnd --outSAMtype BAM Unsorted&quot;)
          elif config['Lib']=='comboseq':
              shell(&quot;/home/tools/STAR-2.7.9a/bin/Linux_x86_64/STAR --runThreadN 16 --genomeDir {input.ref} --readFilesIn {input.trim_in} --readFilesCommand gunzip -c --outFileNamePrefix {params.pr} --outFilterScoreMinOverLread 0 --outFilterMatchNmin 15 --outFilterMatchNminOverLread 0.9 --outFilterMismatchNmax 1 --outFilterMismatchNoverLmax 0.3 --alignIntronMax 1 --outSAMtype BAM Unsorted&quot;)
          else:
              raise MyException(&quot;Library strategy is expected to be either: trilink, comboseq or nextflex; but is&quot; + str(config['Lib']))
              with MyException as e:
                  print(&quot;\033[91mError when we try to execute trimmimg :&quot;, e.message, &quot;\033[0m&quot;)
  
  
  #Housekeeping
  rule sort:
      input:
          al_in=&quot;sam/{prefix}Aligned.out.bam&quot;,
      output:
          sambam=&quot;sam/{prefix}.s.bam&quot;
      shell:
          &quot;&quot;&quot;
          samtools sort -@ 16 -o {output.sambam} {input.al_in}; samtools index {output.sambam}; rm {input.al_in}
          &quot;&quot;&quot;
  
  #index reads
  rule index:
      input:
          al_in=&quot;sam/{prefix}.s.bam&quot;,
      output:
          sambam=&quot;sam/{prefix}.s.bam.bai&quot;
      shell:
          &quot;&quot;&quot;
          samtools index {input.al_in}
          &quot;&quot;&quot;
  </code></pre>
  <p>The <code>config.yaml file</code> is given below,</p>
  <pre><code>  IDs: IDs.txt
    Lib: 'comboseq'
  </code></pre>
  <p>The file <code>IDs.txt</code> is given below,</p>
  <pre><code>  sample.fastq.gz
  </code></pre>
  <p>When I run the above snakemake pipeline using conda environment and the command given below,</p>
  <pre><code>  snakemake --cores 8 --set-threads align=16 sort=16 index=16
  </code></pre>
  <p>then it gives the error given below,</p>
  <pre><code>  MissingInputException in rule trimming in file /mnt/data/Snakefile, 
    line 34:
    Missing input files for rule trimming:
    output: trimfastq/sample.fastq.gz, 
    trimfastq/sample.fastq.gz/sample.fastq.gz_trimmed.fq.gz
    wildcards: prefix=sample.fastq.gz
    affected files:
    fastq/sample.fastq.gz.fastq.gz
  </code></pre>
  <p>Do you have any solution for this problem?</p>
  <p>Thanks,</p>
  
Tags: <rna-seq><snakemake>
Title: snakefile not able to find files for rule trimming
ThreadText: # snakefile not able to find files for rule trimming
  
  (Asked by: username_0 on 2024-08-15)
  
  I am trying to run a snakemake pipeline for small-RNA samples and having problems. The `snakefile` is given below,
  
  ```
  configfile: "config.yaml"
  #include: "utils.py"
  #Command line syntax:
  #snakemake -s snakefile.smk --cores 8 --config prefix= lib=
  class MyException(Exception):
      def __init__(self, message):
          self.message = message
          super().__init__(self.message)
  
  with open(config['IDs'], 'r') as file:
      # Read all the lines of the file into a list
      lines = file.read().splitlines()
  rule all: 
      input:
          expand("fastq/{prefix}.fastq.gz", prefix=lines),
          expand("trimfastq/{prefix}/{prefix}_trimmed.fq.gz", prefix = lines),
          expand("trimfastq/{prefix}", prefix = lines),
          expand("reportfalco/{prefix}", prefix=lines),
          expand("sam/{prefix}Aligned.out.bam", prefix = lines),
          expand("sam/{prefix}.s.bam", prefix=lines),
          expand("sam/{prefix}.s.bam.bai", prefix=lines) 
  
  #Checking fastq quality
  rule testquatility:
      input:
          fq = "fastq/{prefix}.fastq.gz"
      output:
          falrep = directory("reportfalco/{prefix}")
      shell:
          """
          /home/tools/Falco/falco/bin/falco {input.fq} -o {output.falrep}
          """
  #Trimming
  rule trimming:
      input:
          fq_in = "fastq/{prefix}.fastq.gz"
      output:
          trimdir = directory("trimfastq/{prefix}"),
          fq_out = "trimfastq/{prefix}/{prefix}_trimmed.fq.gz"
      run:
          if config['Lib']=='nextflex':
              shell("/home/tools/TrimGalore-0.6.6/trim_galore -a TGGAATTCTCGGGTGCCAAGG --clip_R1 4 --three_prime_clip_R1 4 --length 18 {input.fq_in} --output_dir {output.trimdir}")
          elif config['Lib']=='comboseq':
              shell("/home/tools/TrimGalore-0.6.6/trim_galore -a AAAAAAAA --length 15 {input.fq_in} --output_dir {output.trimdir}")
          elif config['Lib']=='trilink':
              shell("/home/tools/TrimGalore-0.6.6/trim_galore -a TCACCGACTGCCATAGAG --small_rna {input.fq_in} --output_dir {output.trimdir}")
          else:
              raise MyException("Library strategy is expected to be either: trilink, comboseq or nextflex; but is" + str(config['Lib']))
              with MyException as e:
                  print("\033[91mError when we try to execute trimmimg :", e.message, "\033[0m")
  
  #Alignment
  rule align:
      input:
          ref="/references/DERFINDER_ref/STAR_INDEX",
          trim_in="trimfastq/{prefix}/{prefix}_trimmed.fq.gz"
      params:
          pr="sam/{prefix}"
      output:
          "sam/{prefix}Aligned.out.bam"
      run:
          if config['Lib']=='nextflex' or config['Lib']=='trilink':
              shell("/home/tools/STAR-2.7.9a/bin/Linux_x86_64/STAR --runThreadN 16 --genomeDir {input.ref} --readFilesIn {input.trim_in} --readFilesCommand gunzip -c --outFileNamePrefix {params.pr} --outFilterScoreMinOverLread 0 --outFilterMatchNmin 16 --outFilterMatchNminOverLread 0 --outFilterMismatchNoverLmax 0.025 --alignIntronMax 1 --alignEndsType EndToEnd --outSAMtype BAM Unsorted")
          elif config['Lib']=='comboseq':
              shell("/home/tools/STAR-2.7.9a/bin/Linux_x86_64/STAR --runThreadN 16 --genomeDir {input.ref} --readFilesIn {input.trim_in} --readFilesCommand gunzip -c --outFileNamePrefix {params.pr} --outFilterScoreMinOverLread 0 --outFilterMatchNmin 15 --outFilterMatchNminOverLread 0.9 --outFilterMismatchNmax 1 --outFilterMismatchNoverLmax 0.3 --alignIntronMax 1 --outSAMtype BAM Unsorted")
          else:
              raise MyException("Library strategy is expected to be either: trilink, comboseq or nextflex; but is" + str(config['Lib']))
              with MyException as e:
                  print("\033[91mError when we try to execute trimmimg :", e.message, "\033[0m")
  
  #Housekeeping
  rule sort:
      input:
          al_in="sam/{prefix}Aligned.out.bam",
      output:
          sambam="sam/{prefix}.s.bam"
      shell:
          """
          samtools sort -@ 16 -o {output.sambam} {input.al_in}; samtools index {output.sambam}; rm {input.al_in}
          """
  
  #index reads
  rule index:
      input:
          al_in="sam/{prefix}.s.bam",
      output:
          sambam="sam/{prefix}.s.bam.bai"
      shell:
          """
          samtools index {input.al_in}
          """
  
  ```
  
  The `config.yaml file` is given below,
  
  ```
    IDs: IDs.txt
    Lib: 'comboseq'
  
  ```
  
  The file `IDs.txt` is given below,
  
  ```
    sample.fastq.gz
  
  ```
  
  When I run the above snakemake pipeline using conda environment and the command given below,
  
  ```
    snakemake --cores 8 --set-threads align=16 sort=16 index=16
  
  ```
  
  then it gives the error given below,
  
  ```
    MissingInputException in rule trimming in file /mnt/data/Snakefile, 
    line 34:
    Missing input files for rule trimming:
    output: trimfastq/sample.fastq.gz, 
    trimfastq/sample.fastq.gz/sample.fastq.gz_trimmed.fq.gz
    wildcards: prefix=sample.fastq.gz
    affected files:
    fastq/sample.fastq.gz.fastq.gz
  
  ```
  
  Do you have any solution for this problem?
  
  Thanks,
  
  **Tags:** rna-seq, snakemake
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-08-16. Score: 1
  
  The Problem is, that your ID is called `sample.fastq.gz` and you tell snakemake to look for `{ID}.fastq.gz`. Therefor you get the error `sample.fastq.gz.fastq.gz` can not be found.
  
  Thus you have two options. Either remove the *.fastq.gz*-suffix in your *IDs.txt* or remove it in the snakefile.
  
  
  ---
  
  