Answers.Body: <p>It's not a problem with the scatter-gather feature per-se; Snakemake is just confusing itself with what text should match what wildcards with these filenames.  You could fix this by defining a <a href="https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#wildcards" rel="nofollow noreferrer">wildcard constraint</a> for your samples, and/or changing your filename patterns to clarify what should match what.  The <a href="https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#defining-scatter-gather-processes" rel="nofollow noreferrer">docs on scatter-gather</a> show an example with different directory names and file suffixes that prevent this from happening.  Probably the simplest fix would be to follow that idea and use different directory names for the split parts, like <code>trimmed-split</code> and <code>processed-split</code>.</p>
  <p>A hint of what's happening is the repeating <code>...-of-81-...</code> thing, and the mention of ambiguity between rules.  (See also the <code>--debug-dag</code> option.)  If you have a sample named Ornek_411 and you want to make data/processed/Ornek_411.csv, Snakemake will look for data/processed/Ornek_411_1-of-N.csv and so on, and try to figure out how to make those.  But a filename like that could be the output of either the process rule (<code>sample=Ornek_411</code> and <code>scatteritem=1-of-N</code>) or the gather rule itself again (<code>sample=Ornek_411_1-of-N</code>).  Obviously you want the former and not the latter, but it doesn't know that automatically.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-21T15:50:49.707Z
Answers.Text: Firstly thanks changing directory name pattern did trick for me. Thanks for explanation of debug-dag option too. i am aware of this logic because i got that before but with just wildcard but i couldn't set scatteritem logic
Answers.UserDisplayName: sahin
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-21T15:02:54.840Z
Answers.LastActivityDate: 2022-07-21T15:02:54.840Z
Answers.UserDisplayName: Jesse
Body: <p>My problem is when using Snakemake scatter-gather feature the documentation is basic and i modified my code according to mentioned in <a href="https://stackoverflow.com/q/64898553/19589226">this link</a></p>
  <pre><code>rule fastq_fasta:
      input:rules.trimmomatic.output.out_file
      output:&quot;data/trimmed/{sample}.fasta&quot;
      shell:&quot;sed -n '1~4s/^@/&gt;/p;2~4p' {input} &gt; {output}&quot;
  
  rule split:
      input:
          &quot;data/trimmed/{sample}.fasta&quot;
      params:
          scatter_count=config[&quot;scatter_count&quot;],
          scatter_item = lambda wildcards: wildcards.scatteritem
      output:
          temp(scatter.split(&quot;data/trimmed/{{sample}}_{scatteritem}.fasta&quot;))
      script:
          &quot;scripts/split_files.py&quot;
          
  rule process:
      input:&quot;data/trimmed/{sample}_{scatteritem}.fasta&quot;
      output:&quot;data/processed/{sample}_{scatteritem}.csv&quot;
      script:
          &quot;scripts/process.py&quot;
  
  rule gather:
      input:
          gather.split(&quot;data/processed/{{sample}}_{scatteritem}.csv&quot;)
      output:
          &quot;data/processed/{sample}.csv&quot;
      shell:
          &quot;cat {input} &gt; {output}&quot;
  </code></pre>
  <p>I added wildcard option but, i got <code>AmbiguousRuleException: Rules fastq_to_fasta(which is previous rule) and split are ambiguous for the file data/trimmed/Ornek_411-of-81-of-81-of-81-of-81-of-81-of-81-of-81-of-81-of-8.fasta</code></p>
  <p>I tried lots of things but either rules are not calling or take AmbiguousRuleException. What am i missing, can someone help?</p>
  
Tags: <python><snakemake>
Title: Snakemake scatter-gather with wildcard AmbiguousRuleException
ThreadText: # Snakemake scatter-gather with wildcard AmbiguousRuleException
  
  (Asked by: username_0 on 2022-07-21)
  
  My problem is when using Snakemake scatter-gather feature the documentation is basic and i modified my code according to mentioned in [this link](https://stackoverflow.com/q/64898553/19589226)
  
  ```
  rule fastq_fasta:
      input:rules.trimmomatic.output.out_file
      output:"data/trimmed/{sample}.fasta"
      shell:"sed -n '1~4s/^@/>/p;2~4p' {input} > {output}"
  
  rule split:
      input:
          "data/trimmed/{sample}.fasta"
      params:
          scatter_count=config["scatter_count"],
          scatter_item = lambda wildcards: wildcards.scatteritem
      output:
          temp(scatter.split("data/trimmed/{{sample}}_{scatteritem}.fasta"))
      script:
          "scripts/split_files.py"
          
  rule process:
      input:"data/trimmed/{sample}_{scatteritem}.fasta"
      output:"data/processed/{sample}_{scatteritem}.csv"
      script:
          "scripts/process.py"
  
  rule gather:
      input:
          gather.split("data/processed/{{sample}}_{scatteritem}.csv")
      output:
          "data/processed/{sample}.csv"
      shell:
          "cat {input} > {output}"
  
  ```
  
  I added wildcard option but, i got `AmbiguousRuleException: Rules fastq_to_fasta(which is previous rule) and split are ambiguous for the file data/trimmed/Ornek_411-of-81-of-81-of-81-of-81-of-81-of-81-of-81-of-81-of-8.fasta`
  
  I tried lots of things but either rules are not calling or take AmbiguousRuleException. What am i missing, can someone help?
  
  **Tags:** python, snakemake
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-07-21. Score: 1 | Accepted answer
  
  It's not a problem with the scatter-gather feature per-se; Snakemake is just confusing itself with what text should match what wildcards with these filenames. You could fix this by defining a [wildcard constraint](https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#wildcards) for your samples, and/or changing your filename patterns to clarify what should match what. The [docs on scatter-gather](https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#defining-scatter-gather-processes) show an example with different directory names and file suffixes that prevent this from happening. Probably the simplest fix would be to follow that idea and use different directory names for the split parts, like `trimmed-split` and `processed-split`.
  
  A hint of what's happening is the repeating `...-of-81-...` thing, and the mention of ambiguity between rules. (See also the `--debug-dag` option.) If you have a sample named Ornek\_411 and you want to make data/processed/Ornek\_411.csv, Snakemake will look for data/processed/Ornek\_411\_1-of-N.csv and so on, and try to figure out how to make those. But a filename like that could be the output of either the process rule (`sample=Ornek_411` and `scatteritem=1-of-N`) or the gather rule itself again (`sample=Ornek_411_1-of-N`). Obviously you want the former and not the latter, but it doesn't know that automatically.
  
  > **Comments:**
  > **username_0**: Firstly thanks changing directory name pattern did trick for me. Thanks for explanation of debug-dag option too. i am aware of this logic because i got that before but with just wildcard but i couldn't set scatteritem logic  
  
  ---
  
  