Answers.Body: <p>You have defined <code>mem_gb</code> but not <code>mem_mb</code> in your resource configuration file. You can set them to same value by adding something like <code>mem_mb=config['mem_gb']['tool1']</code>. Additional information in their <a href="https://snakemake.readthedocs.io/en/stable/executing/cli.html#profiles" rel="nofollow noreferrer">documentation</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-23T10:51:11.933Z
Answers.Text: Thanks for the response. So basically, Snakemake does not understand mem_gb. So to do this correctly, I need to actually *1000 the values. Because otherwise I get too low values (e.g. 6 mb instead of 6 gb).
Answers.UserDisplayName: Dandelion
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-22T14:53:20.653Z
Answers.LastActivityDate: 2022-11-22T14:53:20.653Z
Answers.UserDisplayName: user324810
Body: <p>I am running a Snakemake (V7.14.0) pipeline on the cluster and provided resources in a resource config file like:</p>
  <pre><code>threads:
      tool1: 4
  mem_gb:
      tool1: 12
  </code></pre>
  <p>Point to it in my rules like:</p>
  <pre><code>resources:
          mem_gb=config['mem_gb']['tool1']
  threads:
          config['threads']['tool1']
  </code></pre>
  <p>And the snakemake cluster command like:</p>
  <pre><code>snakemake --configfiles \
      config/config_resources.yml \
      config/config_parameters.yml \
      --latency-wait 60 \
      --use-conda \
      --use-singularity \
      --printshellcmds \
      --jobname MyPipe{name}_{jobid} \
      --jobs 300 \
      --cluster \
      &quot;bsub -W 200 -n {threads} -M {resources.mem_gb}G \
      -R 'rusage[mem={resources.mem_gb}G]' \
      -o run_logs/{name}_%J.out \
      -e run_logs/{name}_%J.err&quot;
  </code></pre>
  <p>What I do not understand when I run the pipeline is that it shows this in the terminal:</p>
  <blockquote>
  <p>threads: 4</p>
  <p>resources: mem_mb=19442, disk_mb=19442, tmpdir=/tmp, mem_gb=12</p>
  </blockquote>
  <p>So next to the one I provided (mem_gb), there are: mem_mb, and disk_mb. They are clearly not the same value. In the cluster job it looks as though the MEMLIMIT is indeed the one I give, but the MEM usage is different (higher). Is it because Snakemake does not understand the mem_gb and I specifically need to use that word?</p>
  <p>Thanks in advance</p>
  
Tags: <snakemake><hpc>
Title: Multiple values for memory resources shown when Snakemake is run
ThreadText: # Multiple values for memory resources shown when Snakemake is run
  
  (Asked by: username_0 on 2022-11-17)
  
  I am running a Snakemake (V7.14.0) pipeline on the cluster and provided resources in a resource config file like:
  
  ```
  threads:
      tool1: 4
  mem_gb:
      tool1: 12
  
  ```
  
  Point to it in my rules like:
  
  ```
  resources:
          mem_gb=config['mem_gb']['tool1']
  threads:
          config['threads']['tool1']
  
  ```
  
  And the snakemake cluster command like:
  
  ```
  snakemake --configfiles \
      config/config_resources.yml \
      config/config_parameters.yml \
      --latency-wait 60 \
      --use-conda \
      --use-singularity \
      --printshellcmds \
      --jobname MyPipe{name}_{jobid} \
      --jobs 300 \
      --cluster \
      "bsub -W 200 -n {threads} -M {resources.mem_gb}G \
      -R 'rusage[mem={resources.mem_gb}G]' \
      -o run_logs/{name}_%J.out \
      -e run_logs/{name}_%J.err"
  
  ```
  
  What I do not understand when I run the pipeline is that it shows this in the terminal:
  
  > threads: 4
  >
  > resources: mem\_mb=19442, disk\_mb=19442, tmpdir=/tmp, mem\_gb=12
  
  So next to the one I provided (mem\_gb), there are: mem\_mb, and disk\_mb. They are clearly not the same value. In the cluster job it looks as though the MEMLIMIT is indeed the one I give, but the MEM usage is different (higher). Is it because Snakemake does not understand the mem\_gb and I specifically need to use that word?
  
  Thanks in advance
  
  **Tags:** snakemake, hpc
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-11-22. Score: 1 | Accepted answer
  
  You have defined `mem_gb` but not `mem_mb` in your resource configuration file. You can set them to same value by adding something like `mem_mb=config['mem_gb']['tool1']`. Additional information in their [documentation](https://snakemake.readthedocs.io/en/stable/executing/cli.html#profiles).
  
  > **Comments:**
  > **username_0**: Thanks for the response. So basically, Snakemake does not understand mem_gb. So to do this correctly, I need to actually *1000 the values. Because otherwise I get too low values (e.g. 6 mb instead of 6 gb).  
  
  ---
  
  