Body: <p>I am using Snakemake from inside a Singularity container. Because of that, the directory that Snakemake wants to log to cannot be created, and I get an <code>OSError [Errno 30] Read-only file system: '.snakemake</code>, more specifically the traceback</p>
  <pre><code>Traceback (most recent call last):
    File &quot;/usr/local/bin/snakemake&quot;, line 8, in &lt;module&gt;
      sys.exit(main())
    File &quot;/usr/local/lib/python3.8/dist-packages/snakemake/__init__.py&quot;, line 2562, in main
      success = snakemake(
    File &quot;/usr/local/lib/python3.8/dist-packages/snakemake/__init__.py&quot;, line 525, in snakemake
      logger.setup_logfile()
    File &quot;/usr/local/lib/python3.8/dist-packages/snakemake/logging.py&quot;, line 137, in setup_logfile
      os.makedirs(os.path.join(&quot;.snakemake&quot;, &quot;log&quot;), exist_ok=True)
    File &quot;/usr/lib/python3.8/os.py&quot;, line 213, in makedirs
      makedirs(head, exist_ok=exist_ok)
    File &quot;/usr/lib/python3.8/os.py&quot;, line 223, in makedirs
      mkdir(name, mode)
  OSError: [Errno 30] Read-only file system: '.snakemake'
  </code></pre>
  <p>The Snakefile I am using is inside a Singularity-container (there's no way around that, I am on a hpc and don't have root, also using a <code>--sandbox</code>ed container doesn't work), and I call it like this (I've also played around with binding other directories without success):</p>
  <pre><code>singularity run --bind $(realpath .) --bind ~/container.sif ~/container.sif test_simple --cores 4 
  </code></pre>
  <p>The test_simple rule is a really simple one that only unzips a gzip, for testing - the problem however lies generally in Snakemake wanting to create the log at <code>&lt;snakemake_base&gt;/.snakemake/log</code> (see <a href="https://github.com/snakemake/snakemake/blob/bd5a9239b2684b57ab0143bb709e77afa1493cdc/snakemake/logging.py#L308" rel="nofollow noreferrer">https://github.com/snakemake/snakemake/blob/bd5a9239b2684b57ab0143bb709e77afa1493cdc/snakemake/logging.py#L308</a>), which in my case is inside the container.</p>
  <p>It doesn't help to use an overlay <code>--overlay overlay.img</code> for some reason, and I've also tried using my own (empty) log-handle (<code>--log-handler-script ~/loghandler.py</code>) without any success.</p>
  <p>Any ideas on how to circumvent this or disable the logging or write it to somewhere else?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-26T11:49:18.003Z
Comments.Text: I don't know what happens in your case, but I use Singularity containers with snakemake inside, and when it runs, the `.snakemake` folder is created inside the current working directory, not in a read-only folder. It might help seeing the container definition file.
Comments.UserDisplayName: bli
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-26T11:50:50.717Z
Comments.Text: Basic sanity check: do you run the container from a folder in which you have write access?
Comments.UserDisplayName: bli
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-31T15:43:38.710Z
Comments.Text: Can you write to the logs into `/tmp/` ?
Comments.UserDisplayName: Soerendip
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-31T16:10:49.080Z
Comments.Text: Or maybe don't run Snakemake in the container (e.g., `conda install -c conda-forge -c bioconda snakemake` doesn't need root).
Comments.UserDisplayName: merv
Tags: <python><snakemake><hpc>
Title: Snakemake fails because it cannot create logs inside a read-only Singularity Container
ThreadText: # Snakemake fails because it cannot create logs inside a read-only Singularity Container
  
  (Asked by: username_0 on 2021-02-25)
  
  I am using Snakemake from inside a Singularity container. Because of that, the directory that Snakemake wants to log to cannot be created, and I get an `OSError [Errno 30] Read-only file system: '.snakemake`, more specifically the traceback
  
  ```
  Traceback (most recent call last):
    File "/usr/local/bin/snakemake", line 8, in <module>
      sys.exit(main())
    File "/usr/local/lib/python3.8/dist-packages/snakemake/__init__.py", line 2562, in main
      success = snakemake(
    File "/usr/local/lib/python3.8/dist-packages/snakemake/__init__.py", line 525, in snakemake
      logger.setup_logfile()
    File "/usr/local/lib/python3.8/dist-packages/snakemake/logging.py", line 137, in setup_logfile
      os.makedirs(os.path.join(".snakemake", "log"), exist_ok=True)
    File "/usr/lib/python3.8/os.py", line 213, in makedirs
      makedirs(head, exist_ok=exist_ok)
    File "/usr/lib/python3.8/os.py", line 223, in makedirs
      mkdir(name, mode)
  OSError: [Errno 30] Read-only file system: '.snakemake'
  
  ```
  
  The Snakefile I am using is inside a Singularity-container (there's no way around that, I am on a hpc and don't have root, also using a `--sandbox`ed container doesn't work), and I call it like this (I've also played around with binding other directories without success):
  
  ```
  singularity run --bind $(realpath .) --bind ~/container.sif ~/container.sif test_simple --cores 4 
  
  ```
  
  The test\_simple rule is a really simple one that only unzips a gzip, for testing - the problem however lies generally in Snakemake wanting to create the log at `<snakemake_base>/.snakemake/log` (see <https://github.com/snakemake/snakemake/blob/bd5a9239b2684b57ab0143bb709e77afa1493cdc/snakemake/logging.py#L308>), which in my case is inside the container.
  
  It doesn't help to use an overlay `--overlay overlay.img` for some reason, and I've also tried using my own (empty) log-handle (`--log-handler-script ~/loghandler.py`) without any success.
  
  Any ideas on how to circumvent this or disable the logging or write it to somewhere else?
  
  **Tags:** python, snakemake, hpc
  
  > **Comments:**
  > **username_1**: I don't know what happens in your case, but I use Singularity containers with snakemake inside, and when it runs, the `.snakemake` folder is created inside the current working directory, not in a read-only folder. It might help seeing the container definition file.  
  > **username_1**: Basic sanity check: do you run the container from a folder in which you have write access?  
  > **username_2**: Can you write to the logs into `/tmp/` ?  
  > **username_3**: Or maybe don't run Snakemake in the container (e.g., `conda install -c conda-forge -c bioconda snakemake` doesn't need root).  
  
  ---
  
  ## 0 answers
  
  