Body: <p>I am facing an issue when running snakemake with a container and conda.
  Let me explain better.
  I am developing a workflow (locally) in snakemake, where some rules use a conda env (defined in /envs/env.yaml) and other rules use wrappers (so they also are based on conda envs).</p>
  <p>An example here:</p>
  <pre><code>rule clean_spike:
      input:
          sample_ref=&quot;{}results/bam/{{id}}.tmp.bam&quot;.format(outdir),
          sample_spike=&quot;{}results/bam_spike/{{id}}_spike.tmp.bam&quot;.format(outdir),
          sample_ref_index=&quot;{}results/bam/{{id}}.tmp.bam.bai&quot;.format(outdir),
          sample_spike_index=&quot;{}results/bam_spike/{{id}}_spike.tmp.bam.bai&quot;.format(outdir),
      output:
          sample_ref=&quot;{}results/bam/{{id}}.clean.bam&quot;.format(outdir),
          sample_spike=&quot;{}results/bam_spike/{{id}}_spike.clean.bam&quot;.format(outdir),
      conda:
          &quot;../envs/pysam.yaml&quot;
      log:
          &quot;{}results/logs/spike/{{id}}.removeSpikeDups&quot;.format(outdir),
      script:
          &quot;../scripts/remove_spikeDups.py&quot;
  </code></pre>
  <p>Locally, the pipeline runs smoothly.</p>
  <p>Now, I wanted to test the pipeline on another machine (server from the institute), and so I run the command</p>
  <p><code>snakemake --containerize &gt; Dockerfile</code></p>
  <p>I got a dockerfile and  I generated an image that was uploaded on dockerhub <a href="https://hub.docker.com/repository/docker/davidebrex/chiprxsnakemake/general" rel="nofollow noreferrer">here</a>.</p>
  <p>At this point, I copied the workflow on the other machine, I added to the Snakefile the line:</p>
  <p><code>containerized: &quot;docker://davidebrex/chiprxsnakemake:latest&quot;</code></p>
  <p>and tried to run the pipeline with: <code>snakemake -c 10 --use-conda --use-singularity.</code>
  Ideally, snakemake should run the conda envs within the docker container for each rule</p>
  <p>The problem is that i get an error related to the fact that conda is missing..
  I checked the docker image and I can run conda and see the installed envs.</p>
  <p><strong>Any suggestions about what is going on?</strong></p>
  <p>Thank you!</p>
  <p>This is the error I get:</p>
  <pre><code>/usr/bin/bash: conda: command not found
  Traceback (most recent call last):
    File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/__init__.py&quot;, line 792, in snakemake
      success = workflow.execute(
                ^^^^^^^^^^^^^^^^^
    File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/workflow.py&quot;, line 1246, in execute
      raise e
    File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/workflow.py&quot;, line 1242, in execute
      success = self.scheduler.schedule()
                ^^^^^^^^^^^^^^^^^^^^^^^^^
    File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/scheduler.py&quot;, line 636, in schedule
      self.run(runjobs)
    File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/scheduler.py&quot;, line 685, in run
      executor.run_jobs(
    File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py&quot;, line 158, in run_jobs
      self.run(
    File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py&quot;, line 582, in run
      future = self.run_single_job(job)
               ^^^^^^^^^^^^^^^^^^^^^^^^
    File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py&quot;, line 635, in run_single_job
      self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)
                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py&quot;, line 623, in job_args_and_prepare
      self.workflow.conda_base_path,
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/workflow.py&quot;, line 456, in conda_base_path
      return Conda().prefix_path
             ^^^^^^^
    File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/deployment/conda.py&quot;, line 653, in __init__
      shell.check_output(self._get_cmd(&quot;conda info --json&quot;), text=True)
    File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/shell.py&quot;, line 61, in check_output
      return sp.check_output(cmd, shell=True, executable=executable, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/subprocess.py&quot;, line 466, in check_output
      return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File &quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/subprocess.py&quot;, line 571, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command 'conda info --json' returned non-zero exit status 127.
  </code></pre>
  <p>I tried to change the snakemake command and remove --use-conda but then I get an error related to the fact that the tools are missing.
  I checked out other questions on stackoverflow but they were not helpful in my case (like this one: <a href="https://stackoverflow.com/questions/64000711/how-to-use-singularity-and-conda-wrappers-in-snakemake">here</a>).</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-26T15:20:44.653Z
Comments.Text: I have zero experience with `--containerize`, but "I checked the docker image and I can run conda" versus "conda: command not found" when running the workflow really smells like a conda shell setup issue to to me, like people run into when trying to call conda in a script versus interactively.  (No idea where to point you from there, though!)
Comments.UserDisplayName: Jesse
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-30T10:03:11.097Z
Comments.Text: Thank you. At the end I changed snakemake version and it worked..quite weird
Comments.UserDisplayName: DavideBrex
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-12-07T19:11:29.807Z
Comments.Text: Actually now i still get the same error, I also opened an issue on snakemake github, I am clueless
Comments.UserDisplayName: DavideBrex
Tags: <python><snakemake><conda><singularity>
Title: Error in snakemake when running conda envs in container
ThreadText: # Error in snakemake when running conda envs in container
  
  (Asked by: username_0 on 2023-10-26)
  
  I am facing an issue when running snakemake with a container and conda.
  Let me explain better.
  I am developing a workflow (locally) in snakemake, where some rules use a conda env (defined in /envs/env.yaml) and other rules use wrappers (so they also are based on conda envs).
  
  An example here:
  
  ```
  rule clean_spike:
      input:
          sample_ref="{}results/bam/{{id}}.tmp.bam".format(outdir),
          sample_spike="{}results/bam_spike/{{id}}_spike.tmp.bam".format(outdir),
          sample_ref_index="{}results/bam/{{id}}.tmp.bam.bai".format(outdir),
          sample_spike_index="{}results/bam_spike/{{id}}_spike.tmp.bam.bai".format(outdir),
      output:
          sample_ref="{}results/bam/{{id}}.clean.bam".format(outdir),
          sample_spike="{}results/bam_spike/{{id}}_spike.clean.bam".format(outdir),
      conda:
          "../envs/pysam.yaml"
      log:
          "{}results/logs/spike/{{id}}.removeSpikeDups".format(outdir),
      script:
          "../scripts/remove_spikeDups.py"
  
  ```
  
  Locally, the pipeline runs smoothly.
  
  Now, I wanted to test the pipeline on another machine (server from the institute), and so I run the command
  
  `snakemake --containerize > Dockerfile`
  
  I got a dockerfile and I generated an image that was uploaded on dockerhub [here](https://hub.docker.com/repository/docker/username_0/chiprxsnakemake/general).
  
  At this point, I copied the workflow on the other machine, I added to the Snakefile the line:
  
  `containerized: "docker://username_0/chiprxsnakemake:latest"`
  
  and tried to run the pipeline with: `snakemake -c 10 --use-conda --use-singularity.`
  Ideally, snakemake should run the conda envs within the docker container for each rule
  
  The problem is that i get an error related to the fact that conda is missing..
  I checked the docker image and I can run conda and see the installed envs.
  
  **Any suggestions about what is going on?**
  
  Thank you!
  
  This is the error I get:
  
  ```
  /usr/bin/bash: conda: command not found
  Traceback (most recent call last):
    File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/__init__.py", line 792, in snakemake
      success = workflow.execute(
                ^^^^^^^^^^^^^^^^^
    File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/workflow.py", line 1246, in execute
      raise e
    File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/workflow.py", line 1242, in execute
      success = self.scheduler.schedule()
                ^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/scheduler.py", line 636, in schedule
      self.run(runjobs)
    File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/scheduler.py", line 685, in run
      executor.run_jobs(
    File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py", line 158, in run_jobs
      self.run(
    File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py", line 582, in run
      future = self.run_single_job(job)
               ^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py", line 635, in run_single_job
      self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)
                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py", line 623, in job_args_and_prepare
      self.workflow.conda_base_path,
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/workflow.py", line 456, in conda_base_path
      return Conda().prefix_path
             ^^^^^^^
    File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/deployment/conda.py", line 653, in __init__
      shell.check_output(self._get_cmd("conda info --json"), text=True)
    File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/shell.py", line 61, in check_output
      return sp.check_output(cmd, shell=True, executable=executable, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/subprocess.py", line 466, in check_output
      return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/subprocess.py", line 571, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command 'conda info --json' returned non-zero exit status 127.
  
  ```
  
  I tried to change the snakemake command and remove --use-conda but then I get an error related to the fact that the tools are missing.
  I checked out other questions on stackoverflow but they were not helpful in my case (like this one: [here](https://stackoverflow.com/questions/64000711/how-to-use-singularity-and-conda-wrappers-in-snakemake)).
  
  **Tags:** python, snakemake, conda, singularity
  
  > **Comments:**
  > **username_1**: I have zero experience with `--containerize`, but "I checked the docker image and I can run conda" versus "conda: command not found" when running the workflow really smells like a conda shell setup issue to to me, like people run into when trying to call conda in a script versus interactively.  (No idea where to point you from there, though!)  
  > **username_0**: Thank you. At the end I changed snakemake version and it worked..quite weird  
  > **username_0**: Actually now i still get the same error, I also opened an issue on snakemake github, I am clueless  
  
  ---
  
  ## 0 answers
  
  