Answers.Body: <p>Although @JamesHawley answer was informative, I don't think it solved the initial issue requested by the OP. (<strong>edit</strong>: or maybe it partially did, I'm not sure as I got a bit confused with the wildcards on the initial question/answer. I'm anyway leaving this answer here, in case another way of explaining it is useful for others)</p>
  
  <h3>Solution</h3>
  
  <p>I think you only need to change the output of your <code>rule multiqc</code> from:</p>
  
  <p><code>os.path.join(qc_dir, 'multiqc_report.html')</code></p>
  
  <p>to</p>
  
  <p><code>os.path.join(qc_dir, '{sample}.html')</code></p>
  
  <p>This seemed counter-intuitive to me initially, but I think what's happening is that <code>snakemake</code> will look at <code>rule all</code> and see that you want an output called "qc/multiqc_report.html". And the only place where it can find something of that sort is in <code>rule multiqc</code>, by replacing the wildcard <code>{sample}</code> by the string <code>multiqc_report</code>. </p>
  
  <p>It felt confusing to me, because the same <code>{sample}</code> wildcard is used in other rules (namely the rules you use to generate the outputs in your <code>analysed_dir</code>). But <code>snakemake</code> will be smart to replace <code>{sample}</code> with whatever it needs to recreate the output files you're requesting. </p>
  
  <hr>
  
  <h3>Longer explanation</h3>
  
  <p>Here is a (hopefully) reproducible example, if it helps.</p>
  
  <p>First create some fake directory structure with some files:</p>
  
  <pre class="lang-sh prettyprint-override"><code>mkdir inputs
  mkdir analysed_dir
  mkdir qc_dir
  
  touch inputs/file1.txt
  touch inputs/file2.txt
  </code></pre>
  
  <p>Then, this would be the snakefile:</p>
  
  <pre class="lang-py prettyprint-override"><code># Define your variables, fetch things from config, or csv, etc...
  input_dir = "inputs"
  output_dir = "outputs"
  samples = ["file1.txt", "file2.txt"]
  
  
  # Rule all defining all the output files required
  rule all:
    input:
      expand("analysed_dir/{samples}", samples = samples),
      "qc_dir/multi_qc_report.html"
  
  # this rule is equivalent to whichever rules process your data individually
  rule analyse_each_file:
    input:
      "inputs/{sample}.txt"
    output:
      "analysed_dir/{sample}.txt"
    shell:
      "cat {input} &gt; {output}"
  
  # This rule is equivalent to your multiqc step where multiple files result in a single output
  rule multiqc:
    input:
      expand("analysed_dir/{samples}", samples = samples)
    output:
      "qc_dir/{sample}.html"
    shell:
      "cat {input} &gt; {output}"
  </code></pre>
  
  <p>to generate this pipeline:</p>
  
  <p><a href="https://i.sstatic.net/9LnDy.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/9LnDy.png" alt="dag"></a></p>
  
  <p>And then issuing:</p>
  
  <p><code>snakemake --cluster "qsub -N {wildcards.sample}-{rule}" --jobs 10</code></p>
  
  <p>Generates the following job names:</p>
  
  <ul>
  <li>file1-analyse_each_file</li>
  <li>file2-analyse_each_file</li>
  <li>multi_qc_report-multiqc</li>
  </ul>
  
  <p><strong>Edit with further explanation:</strong> </p>
  
  <p>Notice in the DAG how the <code>sample</code> wildcard takes different values for the different rules. I think the snakemake logic is as follows:</p>
  
  <ul>
  <li>Starting with <strong><code>rule all</code></strong> I see that you want three files as input (the first two come from the <code>expand()</code> call): 
  
  <ul>
  <li><code>analysed_dir/file1.txt</code></li>
  <li><code>analysed_dir/file2.txt</code></li>
  <li><code>qc_dir/multi_qc_report.html</code></li>
  </ul></li>
  <li>Let me start with the first one <code>analysed_dir/file1.txt</code>. The file doesn't exist, so let me look in the other rules and see if I can find it in their output. Actually in <strong><code>rule analyse_each_file</code></strong> I see the output is set to <code>"analysed_dir/{sample}.txt"</code>, so if I replace <code>{sample}</code> by <code>file1.txt</code> I can recreate your file from it. Therefore, the input for that rule will be <code>"inputs/file1.txt"</code> (and this file already exists, so I can issue this job).</li>
  <li>For the second file <code>analysed_dir/file2.txt</code>, well it's a similar case as with the previous one, so I'll issue a job with the same <strong><code>rule analyse_each_file</code></strong>, but this time <code>{sample}</code> = <code>file2.txt</code>.</li>
  <li>Finally, you want a file called <code>qc_dir/multi_qc_report.html</code>. Where can I create such a file from? Well, the output of <strong><code>rule multiqc</code></strong> is set to <code>"qc_dir/{sample}.html"</code>, so if I make <code>{sample}</code> = <code>multi_qc_report</code>, then I can issue the job and produce your file of interest. In this case, the rule takes as input two files <code>"analysed_dir/file1.txt"</code> and <code>"analysed_dir/file2.txt"</code>, which I already know how to create from another rule. </li>
  </ul>
  
  <p>The key thing to realise is that <code>snakemake</code> <strong>defines the wildcard <code>{sample}</code> for each rule separately</strong>. So, in this example, <code>wildcard.sample</code> in <code>rule analyse_each_file</code> does not take on the same values as in <code>rule multiqc</code>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-15T13:08:06.340Z
Answers.Text: Thank you for your comment, I still don't have a satisfying solution (meaning I now just leave out the sample wildcard from the qsub job names). For your solution I wonder, will Snakemake not complain about "qc_dir/{sample}.html" never being generated? I think the core issue is that the Multiqc rule takes in all samples so it is correct that there is no single "sample" wildcard. I was just looking for a way to set the sample wildcards to something like "all_samples" for the purpose of naming the qsubbed jobs. Am I making any sense?
Answers.UserDisplayName: Freek
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-16T11:19:30.637Z
Answers.Text: @Freek, when you say "I still don't have a satisfying solution", do you mean you tried my solution and it didn't work? (note, I've edited my solution to more correctly match your posted code using `os.path.join()` - see the **Solution** section at the top). I don't think `snakemake` will get confused, because it seems to me that _wildcards are defined within each rule individually_. See my further edit with expanded explanation at the end of the answer (below the DAG). I hope it helps.
Answers.UserDisplayName: hugot
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-14T10:54:25.777Z
Answers.LastActivityDate: 2020-01-16T11:23:18.193Z
Answers.LastEditDate: 2020-01-16T11:23:18.193Z
Answers.UserDisplayName: hugot
Answers.Body: <p>Snakemake gets <code>wildcards</code> from the parsing the input/output file names.
  For example, if you had</p>
  
  <pre><code>rule sort:
      input: "{file}.bed"
      output: "{file}.sorted.bed"
      shell: "sort -k1,1 -k2,2n {input} &gt; {output}"
  </code></pre>
  
  <p>then the <code>wildcards</code> variable would be set as <code>wildcards = {"file": ...}</code>.</p>
  
  <p>In your <code>multiqc</code> rule, you have the <code>{sample}</code> variable within your <code>expand()</code>, but this variable doesn't exist outside of <code>expand</code>'s local context. You don't actually have any wildcards there, nor in your output file (<code>qc_dir</code> and <code>rseqc_dir</code> are variables you define elsewhere, but not "wildcards" as Snakemake interprets them).</p>
  
  <p>So you're getting the error <code>'Wildcards' object has no attribute 'sample'</code> because there are no wildcards for that rule.</p>
  
  <p>Your rule is currently set up to process every sample that you have listed in  <code>samples['sample']</code>. If you want your <code>multiqc</code> rule to work for a single sample, simply remove the <code>sample=samples['sample']</code> and double brace the <code>{sample}</code> wildcard.</p>
  
  <pre><code>rule multiqc:
      input:
          expand([os.path.join(analyzed_dir, '{{sample}}.stat', '{{sample}}.cnt'),
                  os.path.join(rseqc_dir, '{{sample}}.bam_stat.txt'),
                  os.path.join(rseqc_dir, '{{sample}}.clipping_profile.xls'),
                  os.path.join(rseqc_dir, '{{sample}}.deletion_profile.txt'),
                  os.path.join(rseqc_dir, '{{sample}}.infer_experiment.txt'),
                  os.path.join(rseqc_dir, '{{sample}}.geneBodyCoverage.txt'),
                  os.path.join(rseqc_dir, '{{sample}}.inner_distance.txt'),
                  os.path.join(rseqc_dir, '{{sample}}.insertion_profile.xls'),
                  os.path.join(rseqc_dir, '{{sample}}.junction.xls'),
                  os.path.join(rseqc_dir, '{{sample}}.junctionSaturation_plot.r'),
                  os.path.join(rseqc_dir, '{{sample}}.mismatch_profile.xls'),
                  os.path.join(rseqc_dir, '{{sample}}.read_distribution.txt'),
                  os.path.join(rseqc_dir, '{{sample}}.pos.DupRate.xls'),
                  os.path.join(rseqc_dir, '{{sample}}.seq.DupRate.xls'),
                  os.path.join(rseqc_dir, '{{sample}}.GC.xls'),
                  os.path.join(rseqc_dir, '{{sample}}.NVC.xls'),
                  os.path.join(rseqc_dir, '{{sample}}.qual.r'),
                  os.path.join(rseqc_dir, '{{sample}}.RNA_fragment_size.txt'),
                  os.path.join(rseqc_dir, '{{sample}}.STAR.genome.sorted.summary.txt'),
                  os.path.join(fastqc_dir, '{{sample}}_R{read}_001_fastqc.html')],
                  read=reads)
      output:
          # this {sample} is in single {} and is recognized as a wildcard
          os.path.join(qc_dir, '{sample}', 'multiqc_report.html') 
      conda:
          "envs/multiqc.yaml"
      shell:
          '''
          #!/bin/bash
          # using {wildcards.sample} here since MultiQC likes directories
          multiqc . --outdir qc/{wildcards.sample} --ignore .snakemake --force
          '''
  </code></pre>
  
  <p>When wildcards are recognized, you'll see them in the job list.</p>
  
  <p>Running <code>snakemake 0067_P2018SEQE27R01_S1/multiqc_report.html</code>, for example, should give you something like:</p>
  
  <pre><code>rule multiqc:
      input: qc/rseqc/0067_P2018SEQE27R01_S1.geneBodyCoverage.txt, ...
      output: qc/0067_P2018SEQE27R01_S1/multiqc_report.html
      jobid: 2
      wildcards: sample=0067_P2018SEQE27R01_S1
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-15T20:02:11.857Z
Answers.Text: Thank you, nicely explained, I'll accept this answer. But, do you know of any way to temporarily set the wildcards.sample variable? So that I can use it for naming my multiqc job? I find it a shame that because of this one rule I can't have the {sample}_{rule} naming scheme anymore...
Answers.UserDisplayName: Freek
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-15T22:59:06.870Z
Answers.Text: Maybe you could try using multiple bash commands? Something like `multiqc . --outdir qc --ignore .snakemake --force && mv multiqc_report.html {wildcards.sample}_multiqc.html`?
Answers.UserDisplayName: James Hawley
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-15T23:00:28.007Z
Answers.Text: But I think that because multiqc scans the entire directory, all your files are going to get scanned anyway, and end up in each report. So I don't know if they're actually going to be any different, or if you're just going to end up with the same file under multiple names
Answers.UserDisplayName: James Hawley
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-13T16:39:38.640Z
Answers.Text: @JamesHawley, I thought the OP wanted many inputs (defined by the `expand()` call) to a single output (`qc/multiqc_report.html`), rather than your solution, which if I understand results in an output for every input. It should be possible to achieve what the OP asked for, and the [multiqc wrapper](https://snakemake-wrappers.readthedocs.io/en/stable/wrappers/multiqc.html) seems to suggest so. In which case, what was wrong with the OP's code? (I'm having the same error in a similar situation, thus my comment)
Answers.UserDisplayName: hugot
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-13T16:58:16.047Z
Answers.Text: I got it now: the problem wasn't with the rule's code itself, it was to do with the call to `--cluster`. So, there is no known solution for this problem of wanting to use the `wildcards.sample` in the call to `qsub` when there is one, but otherwise not? This would be particularly useful for log files.
Answers.UserDisplayName: hugot
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-14T23:37:39.667Z
Answers.LastActivityDate: 2019-01-15T14:46:22.620Z
Answers.LastEditDate: 2019-01-15T14:46:22.620Z
Answers.UserDisplayName: James Hawley
Body: <p>I'm using Snakemake and I'm trying to get all my QC reports into multiqc but I get the following error:</p>
  
  <pre><code>WorkflowError in line 120 of /rst1/2017-0205_illuminaseq/scratch/swo-406/test_snakemake_full/Snakefile:
  'Wildcards' object has no attribute 'sample'
  </code></pre>
  
  <p>Rule 120 is the start of my multiqc rule which looks like this:</p>
  
  <pre><code>rule multiqc:
      input:
          expand([os.path.join(analyzed_dir, '{sample}.stat', '{sample}.cnt'),
                  os.path.join(rseqc_dir, '{sample}.bam_stat.txt'),
                  os.path.join(rseqc_dir, '{sample}.clipping_profile.xls'),
                  os.path.join(rseqc_dir, '{sample}.deletion_profile.txt'),
                  os.path.join(rseqc_dir, '{sample}.infer_experiment.txt'),
                  os.path.join(rseqc_dir, '{sample}.geneBodyCoverage.txt'),
                  os.path.join(rseqc_dir, '{sample}.inner_distance.txt'),
                  os.path.join(rseqc_dir, '{sample}.insertion_profile.xls'),
                  os.path.join(rseqc_dir, '{sample}.junction.xls'),
                  os.path.join(rseqc_dir, '{sample}.junctionSaturation_plot.r'),
                  os.path.join(rseqc_dir, '{sample}.mismatch_profile.xls'),
                  os.path.join(rseqc_dir, '{sample}.read_distribution.txt'),
                  os.path.join(rseqc_dir, '{sample}.pos.DupRate.xls'),
                  os.path.join(rseqc_dir, '{sample}.seq.DupRate.xls'),
                  os.path.join(rseqc_dir, '{sample}.GC.xls'),
                  os.path.join(rseqc_dir, '{sample}.NVC.xls'),
                  os.path.join(rseqc_dir, '{sample}.qual.r'),
                  os.path.join(rseqc_dir, '{sample}.RNA_fragment_size.txt'),
                  os.path.join(rseqc_dir, '{sample}.STAR.genome.sorted.summary.txt'),
                  os.path.join(fastqc_dir, '{sample}_R{read}_001_fastqc.html')],
                  sample=samples['samples'], read=reads)
      output:
          os.path.join(qc_dir, 'multiqc_report.html') 
      conda:
          "envs/multiqc.yaml"
      shell:
          '''
          #!/bin/bash
          multiqc . --outdir qc --ignore .snakemake --force
          '''
  </code></pre>
  
  <p>My "rule all looks like this:"</p>
  
  <pre><code>rule all:
      input:
          expand([os.path.join(analyzed_dir, '{sample}.genes.results'),
                  os.path.join(qc_dir, 'multiqc_report.html')],
                  sample=samples['samples'])
  </code></pre>
  
  <p>I don't see anything wrong with it, in fact, it is very similar to my "rule all" I used before and which worked. Can anyone see an error?</p>
  
  <p>EDIT</p>
  
  <p>I limited the rule to only one input, I still have the same error:</p>
  
  <pre><code>rule multiqc:
      input:
          expand(os.path.join(rseqc_dir, '{sample}.geneBodyCoverage.txt'),
                  sample=samples['samples'])
      output:
          os.path.join(qc_dir, 'multiqc_report.html') 
      conda:
          "envs/multiqc.yaml"
      shell:
          '''
          #!/bin/bash
          multiqc . --outdir qc --ignore .snakemake --force
          '''
  </code></pre>
  
  <p>This is the ouput, you can see it read the samples correctly:</p>
  
  <pre><code>run_snakemake.sh 
  /rst1/2017-0205_illuminaseq/scratch/swo-406/snakemake/lib/python3.6/site-packages/snakemake/workflow.py:12: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'.
    from collections import OrderedDict
  Building DAG of jobs...
  /rst1/2017-0205_illuminaseq/scratch/swo-406/snakemake/lib/python3.6/site-packages/snakemake/workflow.py:12: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'.
    from collections import OrderedDict
  Building DAG of jobs...
  Using shell: /bin/bash
  Provided cluster nodes: 256
  Job counts:
      count   jobs
      1   all
      1   multiqc
      2
  
  [Mon Jan 14 17:46:40 2019]
  rule multiqc:
      input: qc/rseqc/0067_P2018SEQE27R01_S1.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R02_S2.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R03_S3.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R04_S4.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R05_S5.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R06_S6.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R07_S7.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R08_S8.geneBodyCoverage.txt
      output: qc/multiqc_report.html
      jobid: 2
  
  WorkflowError in line 119 of /rst1/2017-0205_illuminaseq/scratch/swo-406/test_snakemake_full/Snakefile:
  'Wildcards' object has no attribute 'sample'
  17:46 nlv24077@kiato /rst1/2017-0205_illuminaseq/scratch/swo-406/test_snakemake_full &gt; ll qc/rseqc/0067_P2018SEQE27R07_S7.geneBodyCoverage.txt
  -rw-rw-r-- 1 nlv24077 2017-0205_illuminaseq 303 Jan  7 16:00 qc/rseqc/0067_P2018SEQE27R07_S7.geneBodyCoverage.txt
  </code></pre>
  
  <p>EDIT 2:</p>
  
  <p>This is how I start Snakemake:</p>
  
  <pre><code>&gt; cat run_snakemake.sh
  source activate /rst1/2017-0205_illuminaseq/scratch/swo-406/snakemake
  snakemake --dag | dot -T svg &gt; dag.svg
  snakemake --cluster-config cluster.json --cluster "qsub -l nodes={cluster.nodes}:ppn={cluster.ppn} -N {wildcards.sample}_{rule}" --jobs 256 --use-conda --rerun-incomplete
  </code></pre>
  
  <p>EDIT 3:</p>
  
  <p>Ok, that last remark (by @finswimmer) triggered me, I removed the "{wildcards.sample}_" from the above command and then it works. Or, works, I get a very different error, it is multiqc specific and solved as specified here: <a href="https://multiqc.info/docs/#locale-error-messages" rel="nofollow noreferrer">https://multiqc.info/docs/#locale-error-messages</a>. When adding the two lines as suggested this error disappears and everything works as expected, even with the long inputs list.</p>
  
  <pre><code>RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult https://click.palletsprojects.com/en/7.x/python3/ for mitigation steps.
  </code></pre>
  
  <p>Anyway, I'm guessing Snakemake does not understand how to name this task when it uses an entire list of {sample}'s... Is this a bug? What should be used? I guess it is impossible to use {sample} for naming for this rule. It's a shame I have to kill the naming for all rules because using sample_rule is very nice for keeping track.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-01-14T00:09:40.980Z
Comments.Text: Can you start by limiting the input to that rule to a couple files? That will at least help debug this.
Comments.UserDisplayName: Devon Ryan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-01-14T09:03:52.900Z
Comments.Text: I don't know anything about snakemake but the problem seems to come from not declaring what a `sample` should be
Comments.UserDisplayName: llrs
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-01-14T09:21:18.013Z
Comments.Text: @llrs `sample` is defined in the `expand()` function. Oddly, there's no direct use of `wildcards` outside of that, which is usually where this sort of thing crops up.
Comments.UserDisplayName: Devon Ryan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-01-14T14:45:24.463Z
Comments.Text: Is `samples['samples']` defined somewhere?
Comments.UserDisplayName: finswimmer
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-01-14T17:32:26.997Z
Comments.Text: And how does your command look like to start the workflow?
Comments.UserDisplayName: finswimmer
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-01-14T21:07:29.417Z
Comments.Text: I think you could use it, but not that way. But I think that would be another and please post how you solved it (I want to learn to use snakemake).
Comments.UserDisplayName: llrs
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-01-14T21:47:55.143Z
Comments.Text: Hi @llrs, the answer is in EDIT 2, there is my "start snakemake" script. I used the wildcards.sample to name the jobs on our qsub cluster, but I can't do that because my MultiQC does not have a single defined wildcards.sample (it has many). So I simply removed that part (the {wildcards.sample}) and now I only have the rules as the jobname, sadly. Does that help? How do you think I could use it then?
Comments.UserDisplayName: Freek
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-01-15T08:04:57.483Z
Comments.Text: @Freek Well, to signal that you got an answer here (on the stackexchange.com sites) we post an answer! You can post an answer to your own question (it is completely normal and encouraged). In the answer please explain how it solved your problem as you did in this comment.
Comments.UserDisplayName: llrs
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-01-15T20:03:47.723Z
Comments.Text: Ok, thanx, I'll remember that, for now James Hawley worded it nicely already :)
Comments.UserDisplayName: Freek
Tags: <snakemake>
Title: 'Wildcards' object has no attribute 'sample'
ThreadText: # 'Wildcards' object has no attribute 'sample'
  
  (Asked by: username_0 on 2019-01-13)
  
  I'm using Snakemake and I'm trying to get all my QC reports into multiqc but I get the following error:
  
  ```
  WorkflowError in line 120 of /rst1/2017-0205_illuminaseq/scratch/swo-406/test_snakemake_full/Snakefile:
  'Wildcards' object has no attribute 'sample'
  
  ```
  
  Rule 120 is the start of my multiqc rule which looks like this:
  
  ```
  rule multiqc:
      input:
          expand([os.path.join(analyzed_dir, '{sample}.stat', '{sample}.cnt'),
                  os.path.join(rseqc_dir, '{sample}.bam_stat.txt'),
                  os.path.join(rseqc_dir, '{sample}.clipping_profile.xls'),
                  os.path.join(rseqc_dir, '{sample}.deletion_profile.txt'),
                  os.path.join(rseqc_dir, '{sample}.infer_experiment.txt'),
                  os.path.join(rseqc_dir, '{sample}.geneBodyCoverage.txt'),
                  os.path.join(rseqc_dir, '{sample}.inner_distance.txt'),
                  os.path.join(rseqc_dir, '{sample}.insertion_profile.xls'),
                  os.path.join(rseqc_dir, '{sample}.junction.xls'),
                  os.path.join(rseqc_dir, '{sample}.junctionSaturation_plot.r'),
                  os.path.join(rseqc_dir, '{sample}.mismatch_profile.xls'),
                  os.path.join(rseqc_dir, '{sample}.read_distribution.txt'),
                  os.path.join(rseqc_dir, '{sample}.pos.DupRate.xls'),
                  os.path.join(rseqc_dir, '{sample}.seq.DupRate.xls'),
                  os.path.join(rseqc_dir, '{sample}.GC.xls'),
                  os.path.join(rseqc_dir, '{sample}.NVC.xls'),
                  os.path.join(rseqc_dir, '{sample}.qual.r'),
                  os.path.join(rseqc_dir, '{sample}.RNA_fragment_size.txt'),
                  os.path.join(rseqc_dir, '{sample}.STAR.genome.sorted.summary.txt'),
                  os.path.join(fastqc_dir, '{sample}_R{read}_001_fastqc.html')],
                  sample=samples['samples'], read=reads)
      output:
          os.path.join(qc_dir, 'multiqc_report.html') 
      conda:
          "envs/multiqc.yaml"
      shell:
          '''
          #!/bin/bash
          multiqc . --outdir qc --ignore .snakemake --force
          '''
  
  ```
  
  My "rule all looks like this:"
  
  ```
  rule all:
      input:
          expand([os.path.join(analyzed_dir, '{sample}.genes.results'),
                  os.path.join(qc_dir, 'multiqc_report.html')],
                  sample=samples['samples'])
  
  ```
  
  I don't see anything wrong with it, in fact, it is very similar to my "rule all" I used before and which worked. Can anyone see an error?
  
  EDIT
  
  I limited the rule to only one input, I still have the same error:
  
  ```
  rule multiqc:
      input:
          expand(os.path.join(rseqc_dir, '{sample}.geneBodyCoverage.txt'),
                  sample=samples['samples'])
      output:
          os.path.join(qc_dir, 'multiqc_report.html') 
      conda:
          "envs/multiqc.yaml"
      shell:
          '''
          #!/bin/bash
          multiqc . --outdir qc --ignore .snakemake --force
          '''
  
  ```
  
  This is the ouput, you can see it read the samples correctly:
  
  ```
  run_snakemake.sh 
  /rst1/2017-0205_illuminaseq/scratch/swo-406/snakemake/lib/python3.6/site-packages/snakemake/workflow.py:12: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'.
    from collections import OrderedDict
  Building DAG of jobs...
  /rst1/2017-0205_illuminaseq/scratch/swo-406/snakemake/lib/python3.6/site-packages/snakemake/workflow.py:12: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'.
    from collections import OrderedDict
  Building DAG of jobs...
  Using shell: /bin/bash
  Provided cluster nodes: 256
  Job counts:
      count   jobs
      1   all
      1   multiqc
      2
  
  [Mon Jan 14 17:46:40 2019]
  rule multiqc:
      input: qc/rseqc/0067_P2018SEQE27R01_S1.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R02_S2.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R03_S3.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R04_S4.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R05_S5.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R06_S6.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R07_S7.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R08_S8.geneBodyCoverage.txt
      output: qc/multiqc_report.html
      jobid: 2
  
  WorkflowError in line 119 of /rst1/2017-0205_illuminaseq/scratch/swo-406/test_snakemake_full/Snakefile:
  'Wildcards' object has no attribute 'sample'
  17:46 nlv24077@kiato /rst1/2017-0205_illuminaseq/scratch/swo-406/test_snakemake_full > ll qc/rseqc/0067_P2018SEQE27R07_S7.geneBodyCoverage.txt
  -rw-rw-r-- 1 nlv24077 2017-0205_illuminaseq 303 Jan  7 16:00 qc/rseqc/0067_P2018SEQE27R07_S7.geneBodyCoverage.txt
  
  ```
  
  EDIT 2:
  
  This is how I start Snakemake:
  
  ```
  > cat run_snakemake.sh
  source activate /rst1/2017-0205_illuminaseq/scratch/swo-406/snakemake
  snakemake --dag | dot -T svg > dag.svg
  snakemake --cluster-config cluster.json --cluster "qsub -l nodes={cluster.nodes}:ppn={cluster.ppn} -N {wildcards.sample}_{rule}" --jobs 256 --use-conda --rerun-incomplete
  
  ```
  
  EDIT 3:
  
  Ok, that last remark (by @username_3) triggered me, I removed the "{wildcards.sample}\_" from the above command and then it works. Or, works, I get a very different error, it is multiqc specific and solved as specified here: <https://multiqc.info/docs/#locale-error-messages>. When adding the two lines as suggested this error disappears and everything works as expected, even with the long inputs list.
  
  ```
  RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult https://click.palletsprojects.com/en/7.x/python3/ for mitigation steps.
  
  ```
  
  Anyway, I'm guessing Snakemake does not understand how to name this task when it uses an entire list of {sample}'s... Is this a bug? What should be used? I guess it is impossible to use {sample} for naming for this rule. It's a shame I have to kill the naming for all rules because using sample\_rule is very nice for keeping track.
  
  **Tags:** snakemake
  
  > **Comments:**
  > **username_1**: Can you start by limiting the input to that rule to a couple files? That will at least help debug this.  
  > **username_2**: I don't know anything about snakemake but the problem seems to come from not declaring what a `sample` should be  
  > **username_1**: @username_2 `sample` is defined in the `expand()` function. Oddly, there's no direct use of `wildcards` outside of that, which is usually where this sort of thing crops up.  
  > **username_3**: Is `samples['samples']` defined somewhere?  
  > **username_3**: And how does your command look like to start the workflow?  
  > **username_2**: I think you could use it, but not that way. But I think that would be another and please post how you solved it (I want to learn to use snakemake).  
  > **username_0**: Hi @username_2, the answer is in EDIT 2, there is my "start snakemake" script. I used the wildcards.sample to name the jobs on our qsub cluster, but I can't do that because my MultiQC does not have a single defined wildcards.sample (it has many). So I simply removed that part (the {wildcards.sample}) and now I only have the rules as the jobname, sadly. Does that help? How do you think I could use it then?  
  > **username_2**: @username_0 Well, to signal that you got an answer here (on the stackexchange.com sites) we post an answer! You can post an answer to your own question (it is completely normal and encouraged). In the answer please explain how it solved your problem as you did in this comment.  
  > **username_0**: Ok, thanx, I'll remember that, for now username_5 username_5 worded it nicely already :)  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_5 on 2019-01-14. Score: 5 | Accepted answer
  
  Snakemake gets `wildcards` from the parsing the input/output file names.
  For example, if you had
  
  ```
  rule sort:
      input: "{file}.bed"
      output: "{file}.sorted.bed"
      shell: "sort -k1,1 -k2,2n {input} > {output}"
  
  ```
  
  then the `wildcards` variable would be set as `wildcards = {"file": ...}`.
  
  In your `multiqc` rule, you have the `{sample}` variable within your `expand()`, but this variable doesn't exist outside of `expand`'s local context. You don't actually have any wildcards there, nor in your output file (`qc_dir` and `rseqc_dir` are variables you define elsewhere, but not "wildcards" as Snakemake interprets them).
  
  So you're getting the error `'Wildcards' object has no attribute 'sample'` because there are no wildcards for that rule.
  
  Your rule is currently set up to process every sample that you have listed in `samples['sample']`. If you want your `multiqc` rule to work for a single sample, simply remove the `sample=samples['sample']` and double brace the `{sample}` wildcard.
  
  ```
  rule multiqc:
      input:
          expand([os.path.join(analyzed_dir, '{{sample}}.stat', '{{sample}}.cnt'),
                  os.path.join(rseqc_dir, '{{sample}}.bam_stat.txt'),
                  os.path.join(rseqc_dir, '{{sample}}.clipping_profile.xls'),
                  os.path.join(rseqc_dir, '{{sample}}.deletion_profile.txt'),
                  os.path.join(rseqc_dir, '{{sample}}.infer_experiment.txt'),
                  os.path.join(rseqc_dir, '{{sample}}.geneBodyCoverage.txt'),
                  os.path.join(rseqc_dir, '{{sample}}.inner_distance.txt'),
                  os.path.join(rseqc_dir, '{{sample}}.insertion_profile.xls'),
                  os.path.join(rseqc_dir, '{{sample}}.junction.xls'),
                  os.path.join(rseqc_dir, '{{sample}}.junctionSaturation_plot.r'),
                  os.path.join(rseqc_dir, '{{sample}}.mismatch_profile.xls'),
                  os.path.join(rseqc_dir, '{{sample}}.read_distribution.txt'),
                  os.path.join(rseqc_dir, '{{sample}}.pos.DupRate.xls'),
                  os.path.join(rseqc_dir, '{{sample}}.seq.DupRate.xls'),
                  os.path.join(rseqc_dir, '{{sample}}.GC.xls'),
                  os.path.join(rseqc_dir, '{{sample}}.NVC.xls'),
                  os.path.join(rseqc_dir, '{{sample}}.qual.r'),
                  os.path.join(rseqc_dir, '{{sample}}.RNA_fragment_size.txt'),
                  os.path.join(rseqc_dir, '{{sample}}.STAR.genome.sorted.summary.txt'),
                  os.path.join(fastqc_dir, '{{sample}}_R{read}_001_fastqc.html')],
                  read=reads)
      output:
          # this {sample} is in single {} and is recognized as a wildcard
          os.path.join(qc_dir, '{sample}', 'multiqc_report.html') 
      conda:
          "envs/multiqc.yaml"
      shell:
          '''
          #!/bin/bash
          # using {wildcards.sample} here since MultiQC likes directories
          multiqc . --outdir qc/{wildcards.sample} --ignore .snakemake --force
          '''
  
  ```
  
  When wildcards are recognized, you'll see them in the job list.
  
  Running `snakemake 0067_P2018SEQE27R01_S1/multiqc_report.html`, for example, should give you something like:
  
  ```
  rule multiqc:
      input: qc/rseqc/0067_P2018SEQE27R01_S1.geneBodyCoverage.txt, ...
      output: qc/0067_P2018SEQE27R01_S1/multiqc_report.html
      jobid: 2
      wildcards: sample=0067_P2018SEQE27R01_S1
  
  ```
  
  > **Comments:**
  > **username_0**: Thank you, nicely explained, I'll accept this answer. But, do you know of any way to temporarily set the wildcards.sample variable? So that I can use it for naming my multiqc job? I find it a shame that because of this one rule I can't have the {sample}_{rule} naming scheme anymore...  
  > **username_5**: Maybe you could try using multiple bash commands? Something like `multiqc . --outdir qc --ignore .snakemake --force && mv multiqc_report.html {wildcards.sample}_multiqc.html`?  
  > **username_5**: But I think that because multiqc scans the entire directory, all your files are going to get scanned anyway, and end up in each report. So I don't know if they're actually going to be any different, or if you're just going to end up with the same file under multiple names  
  > **username_4**: @username_5, I thought the OP wanted many inputs (defined by the `expand()` call) to a single output (`qc/multiqc_report.html`), rather than your solution, which if I understand results in an output for every input. It should be possible to achieve what the OP asked for, and the [multiqc wrapper](https://snakemake-wrappers.readthedocs.io/en/stable/wrappers/multiqc.html) seems to suggest so. In which case, what was wrong with the OP's code? (I'm having the same error in a similar situation, thus my comment)  
  > **username_4**: I got it now: the problem wasn't with the rule's code itself, it was to do with the call to `--cluster`. So, there is no known solution for this problem of wanting to use the `wildcards.sample` in the call to `qsub` when there is one, but otherwise not? This would be particularly useful for log files.  
  
  ---
  
  ### Answer by username_4 on 2020-01-14. Score: 0
  
  Although @username_5 answer was informative, I don't think it solved the initial issue requested by the OP. (**edit**: or maybe it partially did, I'm not sure as I got a bit confused with the wildcards on the initial question/answer. I'm anyway leaving this answer here, in case another way of explaining it is useful for others)
  
  ### Solution
  
  I think you only need to change the output of your `rule multiqc` from:
  
  `os.path.join(qc_dir, 'multiqc_report.html')`
  
  to
  
  `os.path.join(qc_dir, '{sample}.html')`
  
  This seemed counter-intuitive to me initially, but I think what's happening is that `snakemake` will look at `rule all` and see that you want an output called "qc/multiqc\_report.html". And the only place where it can find something of that sort is in `rule multiqc`, by replacing the wildcard `{sample}` by the string `multiqc_report`.
  
  It felt confusing to me, because the same `{sample}` wildcard is used in other rules (namely the rules you use to generate the outputs in your `analysed_dir`). But `snakemake` will be smart to replace `{sample}` with whatever it needs to recreate the output files you're requesting.
  
  ---
  
  ### Longer explanation
  
  Here is a (hopefully) reproducible example, if it helps.
  
  First create some fake directory structure with some files:
  
  ```sh
  mkdir inputs
  mkdir analysed_dir
  mkdir qc_dir
  
  touch inputs/file1.txt
  touch inputs/file2.txt
  
  ```
  
  Then, this would be the snakefile:
  
  ```py
  # Define your variables, fetch things from config, or csv, etc...
  input_dir = "inputs"
  output_dir = "outputs"
  samples = ["file1.txt", "file2.txt"]
  
  # Rule all defining all the output files required
  rule all:
    input:
      expand("analysed_dir/{samples}", samples = samples),
      "qc_dir/multi_qc_report.html"
  
  # this rule is equivalent to whichever rules process your data individually
  rule analyse_each_file:
    input:
      "inputs/{sample}.txt"
    output:
      "analysed_dir/{sample}.txt"
    shell:
      "cat {input} > {output}"
  
  # This rule is equivalent to your multiqc step where multiple files result in a single output
  rule multiqc:
    input:
      expand("analysed_dir/{samples}", samples = samples)
    output:
      "qc_dir/{sample}.html"
    shell:
      "cat {input} > {output}"
  
  ```
  
  to generate this pipeline:
  
  [![dag](https://i.sstatic.net/9LnDy.png)](https://i.sstatic.net/9LnDy.png)
  
  And then issuing:
  
  `snakemake --cluster "qsub -N {wildcards.sample}-{rule}" --jobs 10`
  
  Generates the following job names:
  
  * file1-analyse\_each\_file
  * file2-analyse\_each\_file
  * multi\_qc\_report-multiqc
  
  **Edit with further explanation:**
  
  Notice in the DAG how the `sample` wildcard takes different values for the different rules. I think the snakemake logic is as follows:
  
  * Starting with **`rule all`** I see that you want three files as input (the first two come from the `expand()` call):
    + `analysed_dir/file1.txt`
    + `analysed_dir/file2.txt`
    + `qc_dir/multi_qc_report.html`
  * Let me start with the first one `analysed_dir/file1.txt`. The file doesn't exist, so let me look in the other rules and see if I can find it in their output. Actually in **`rule analyse_each_file`** I see the output is set to `"analysed_dir/{sample}.txt"`, so if I replace `{sample}` by `file1.txt` I can recreate your file from it. Therefore, the input for that rule will be `"inputs/file1.txt"` (and this file already exists, so I can issue this job).
  * For the second file `analysed_dir/file2.txt`, well it's a similar case as with the previous one, so I'll issue a job with the same **`rule analyse_each_file`**, but this time `{sample}` = `file2.txt`.
  * Finally, you want a file called `qc_dir/multi_qc_report.html`. Where can I create such a file from? Well, the output of **`rule multiqc`** is set to `"qc_dir/{sample}.html"`, so if I make `{sample}` = `multi_qc_report`, then I can issue the job and produce your file of interest. In this case, the rule takes as input two files `"analysed_dir/file1.txt"` and `"analysed_dir/file2.txt"`, which I already know how to create from another rule.
  
  The key thing to realise is that `snakemake` **defines the wildcard `{sample}` for each rule separately**. So, in this example, `wildcard.sample` in `rule analyse_each_file` does not take on the same values as in `rule multiqc`.
  
  > **Comments:**
  > **username_0**: Thank you for your comment, I still don't have a satisfying solution (meaning I now just leave out the sample wildcard from the qsub job names). For your solution I wonder, will Snakemake not complain about "qc_dir/{sample}.html" never being generated? I think the core issue is that the Multiqc rule takes in all samples so it is correct that there is no single "sample" wildcard. I was just looking for a way to set the sample wildcards to something like "all_samples" for the purpose of naming the qsubbed jobs. Am I making any sense?  
  > **username_4**: @username_0, when you say "I still don't have a satisfying solution", do you mean you tried my solution and it didn't work? (note, I've edited my solution to more correctly match your posted code using `os.path.join()` - see the **Solution** section at the top). I don't think `snakemake` will get confused, because it seems to me that _wildcards are defined within each rule individually_. See my further edit with expanded explanation at the end of the answer (below the DAG). I hope it helps.  
  
  ---
  
  