Answers.Body: <p>As stated in <a href="https://github.com/broadinstitute/cromwell/issues/4395" rel="nofollow noreferrer">https://github.com/broadinstitute/cromwell/issues/4395</a></p>
  <p>The problem may be because you can't hardlink folders on your system for various reasons (i.e. filesystem doesn't allow hardlinking folders, only files,  or you are trying to link between different filesystems).</p>
  <p>You can try overriding the default way glob works.</p>
  <p>i.e. add this to your cromwell.config</p>
  <pre><code>glob-link-command = &quot;ls -L GLOB_PATTERN 2&gt; /dev/null | xargs -I ? ln -s ? GLOB_DIRECTORY&quot;
  </code></pre>
  <p>Like this</p>
  <pre><code>backend
  {
    default = &quot;Slurm&quot;
    providers
    {
      Htcondor {
        actor-factory = &quot;cromwell.backend.impl.sfs.config.ConfigBackendLifecycleActorFactory&quot;
  
        config
        {
          glob-link-command = &quot;find GLOB_PATTERN 2&gt; /dev/null | xargs -I ? ln -s ? GLOB_DIRECTORY&quot;
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-08T19:12:24.533Z
Answers.LastActivityDate: 2023-08-08T19:12:24.533Z
Answers.UserDisplayName: Jeff Froula
Body: <p>Using Crowell 52.  When I do a glob to get matching files, Cromwell places a copy of each matching file in the glob-* directory.</p>
  <p>As these are multi-gigabyte Bam files, I'd really prefer for Cromwell to store soft links to the files, rather than full copies.</p>
  <p>The code:</p>
  <pre><code>   output {
        Array[File] Outputs = glob(&quot;*.fastq.gz&quot;)
     }
  </code></pre>
  <p>Is it possible to make this happen?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-09T15:51:52.023Z
Comments.Text: It would help your case to show the code you're using.
Comments.UserDisplayName: Ram RS
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-10T15:12:54.457Z
Comments.Text: @RamRS added above
Comments.UserDisplayName: Greg Dougherty
Tags: <wdl>
Title: How do I keep a glob from copying a file in Cromwell / WDL
ThreadText: # How do I keep a glob from copying a file in Cromwell / WDL
  
  (Asked by: username_0 on 2021-03-09)
  
  Using Crowell 52. When I do a glob to get matching files, Cromwell places a copy of each matching file in the glob-\* directory.
  
  As these are multi-gigabyte Bam files, I'd really prefer for Cromwell to store soft links to the files, rather than full copies.
  
  The code:
  
  ```
     output {
        Array[File] Outputs = glob("*.fastq.gz")
     }
  
  ```
  
  Is it possible to make this happen?
  
  **Tags:** wdl
  
  > **Comments:**
  > **username_1**: It would help your case to show the code you're using.  
  > **username_0**: @username_1 added above  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-08-08. Score: 1
  
  As stated in <https://github.com/broadinstitute/cromwell/issues/4395>
  
  The problem may be because you can't hardlink folders on your system for various reasons (i.e. filesystem doesn't allow hardlinking folders, only files, or you are trying to link between different filesystems).
  
  You can try overriding the default way glob works.
  
  i.e. add this to your cromwell.config
  
  ```
  glob-link-command = "ls -L GLOB_PATTERN 2> /dev/null | xargs -I ? ln -s ? GLOB_DIRECTORY"
  
  ```
  
  Like this
  
  ```
  backend
  {
    default = "Slurm"
    providers
    {
      Htcondor {
        actor-factory = "cromwell.backend.impl.sfs.config.ConfigBackendLifecycleActorFactory"
  
        config
        {
          glob-link-command = "find GLOB_PATTERN 2> /dev/null | xargs -I ? ln -s ? GLOB_DIRECTORY"
  
  ```
  
  
  ---
  
  