Answers.Body: <p>Putting the placeholder inside of an input file won't work, since the contents will just be turned into a string when read with <code>read_lines</code> and won't get treated like WDL code. Since the lines from the <code>program_file</code> are simply getting pasted into the command, you can move the contents into the command section. In a placeholder (<code>~{}</code>), adding a string to an optional value will result in an empty string if that optional value is not supplied (see the spec <a href="https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#prepending-a-string-to-an-optional-parameter" rel="nofollow noreferrer">here</a>). So you can use optional inputs to make the different lines non-mandatory:</p>
  <pre><code>task trimmomatic {
      input {
          File fastq1
          File fastq2
          String outfile1 = “trimmed_R1.fastq.gz”
          String outfile2 = “trimmed_R2.fastq.gz”
          File? adapter_file
          Int? minlen
          String? slidingwindow
          Int? leading
          Int? trailing
       }
   
       command {
          TrimmomaticPE \
          ~{fastq1} ~{fastq2} \
          ~{outfile1} ~{outfile2} \
          ~{&quot;ILLUMINACLIP:&quot; + adapter_file + &quot;:2:30:10:7:keepBothReads&quot;} \
          ~{&quot;MINLEN:&quot; + minlen} \
          ~{&quot;SLIDINGWINDOW:&quot; + slidingwindow} \
          ~{&quot;LEADING:&quot; + leading} \
          ~{&quot;TRAILING:&quot; + trailing}
       }
   
       output {
          File out1 = outfile1
          File out2 = outfile2
       }
  }
  </code></pre>
  <p>If, say, <code>adapter_file</code> were to be omitted in the inputs, then the produced command should look something like this:</p>
  <pre><code>TrimmomaticPE \
  R1.fq R2.fq \
  trimmed_R1.fastq.gz trimmed_R2.fastq.gz \
   \
  MINLEN:36 \
  SLIDINGWINDOW:4:15 \
  LEADING:3 \
  TRAILING:3
  </code></pre>
  <p>Also note that the output file paths in the input section should be <code>String</code>s. Since they don't exist yet, the execution engine will complain that it can't find them if they are <code>File</code>s</p>
  <p>Ps. I'm not familiar with the actual trimmomatic command options, so please do check that this produces the desired command. I suspect that the <code>:2:30:10:7:keepBothReads</code> bit could be split up into additional inputs as well.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-04T08:34:33.483Z
Answers.LastActivityDate: 2022-10-04T08:34:33.483Z
Answers.UserDisplayName: DavyCats
Body: <p>I am developing a gatk pipeline for variant calling and want to include trimmomatic as a preprocessing step. I would like the user to specify a program file of parameters to feed into trimmomatic, and a file of adapter sequences to provide to the ILLUMINACLIP parameter. My goal is to include a variable (“~{adapter_file}” below) in the program file that will expand to include the adapter filename, like so:</p>
  <p>Adapter file:</p>
  <pre><code>&gt;IL_Multiplex_five_prime_fwd
  AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT
  &gt;IL_Multiplex_three_prime_rev
  AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC
  </code></pre>
  <p>Program file:</p>
  <pre><code>ILLUMINACLIP:~{adapter_file}:2:30:10:7:keepBothReads
  MINLEN:36
  SLIDINGWINDOW:4:15
  LEADING:3
  TRAILING:3
  </code></pre>
  <p>In this way, the user can change the adaptor file without having to alter the program file each time. However, I can’t seem to get the ~{adapter_file} variable to expand inside the ~{trim_program_array} variable. Here’s a simplified version of my trimmomatic WDL file:</p>
  <p>Sample workflow:</p>
  <pre><code>version 1.0
   
  Workflow trim {
      input {
          File fastq1 = “R1.fastq.gz”
          File fastq2 = “R2.fastq.gz”
          File program_file = “trim_program.txt”
          File adapter_file = “adapters.fa”
      }
   
      Array[String] trim_program_array = read_lines(program_file)
   
      call trimmomatic {
          input:
              fastq1 = fastq1,
              fastq2 = fastq2,
              trim_program_array = trim_program_array,
              adapter_file = adaptor_file
      }
  }
   
  Task trimmomatic {
      input {
          File fastq1
          File fastq2
          File outfile1 = “trimmed_R1.fastq.gz”
          File outfile2 = “trimmed_R2.fastq.gz”
          Array[String] trim_program_array
          File adapter_file
       }
   
       command {
          TrimmomaticPE \
          ~{fastq1} ~{fastq2} \
          ~{outfile1} ~{outfile2} \
          ~{sep = “ “ trim_program_array}
       }
   
       output {
          File out1 = outfile1
          File out2 = outfile2
       }
  }
  </code></pre>
  <p>My goal is for the first element of ~{sep = &quot; &quot; trim_program_array} (aka the first line of trim_program.txt) to evaluate to “ILLUMINACLIP:adapters.fa:2:30:10:7:keepBothReads” rather than “ILLUMINACLIP:~{adapter_file}:2:30:10:7:keepBothReads”. I know this is because ~{adapter_file} is nested inside trim_program_array[0], so it’s only expanding the “outer” variable. How can I convince WDL to expand the variable nested inside the first element of that array before executing the bash command?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-27T07:48:41.710Z
Comments.Text: Hi, I've corrected your question to make the spelling of the word 'adapter' consistent, but noticed that some of those changes were in the configuration file. Can you please re-check to make sure this spelling inconsistency is not causing your problems?
Comments.UserDisplayName: gringer
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-27T07:59:10.613Z
Comments.Text: The ~{} syntax only works within the wdl command section or wdl string literals. Since the string is getting read from a file in this case, it is probably getting escaped. It looks like this file is just getting used to provide arguments in the command. Can't you just include the contents of the program file in the command section directly?
Comments.UserDisplayName: DavyCats
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-29T02:11:32.140Z
Comments.Text: @DavyCats Yes, that's what I eventually figured out, but it would be nice to have it in the program file so that users of the pipeline aren't left wondering what ":2:30:10:7:keepBothReads" in the first line of the program file refers to, since 'ILLUMINACLIP:~{adaptor_file}' is missing. There are worse problems to have though, so I'm fine with that work-around. Thanks for the suggestion.
Comments.UserDisplayName: user16196
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-30T20:43:17.733Z
Comments.Text: @DavyCats Upon thinking about this further, I realized that one advantage to supplying a program file to Trimmomatic is that the user gets to determine which parameters to trim by, rather than just choosing the value for each predetermined parameter. By encoding "ILLUMINACLIP:~{adaptor_file}" into the code block, I'm in effect making that a required parameter. I've seen many cases where a user wouldn't want to supply that parameter to Trimmomatic at all. I'm hoping that someone will come along and help me with a better solution to this problem so that I don't have to hard code any parameters.
Comments.UserDisplayName: user16196
Tags: <wdl>
Title: Nested variable expansion in WDL code block
ThreadText: # Nested variable expansion in WDL code block
  
  (Asked by: username_0 on 2022-09-26)
  
  I am developing a gatk pipeline for variant calling and want to include trimmomatic as a preprocessing step. I would like the user to specify a program file of parameters to feed into trimmomatic, and a file of adapter sequences to provide to the ILLUMINACLIP parameter. My goal is to include a variable (“~{adapter\_file}” below) in the program file that will expand to include the adapter filename, like so:
  
  Adapter file:
  
  ```
  >IL_Multiplex_five_prime_fwd
  AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT
  >IL_Multiplex_three_prime_rev
  AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC
  
  ```
  
  Program file:
  
  ```
  ILLUMINACLIP:~{adapter_file}:2:30:10:7:keepBothReads
  MINLEN:36
  SLIDINGWINDOW:4:15
  LEADING:3
  TRAILING:3
  
  ```
  
  In this way, the user can change the adaptor file without having to alter the program file each time. However, I can’t seem to get the ~{adapter\_file} variable to expand inside the ~{trim\_program\_array} variable. Here’s a simplified version of my trimmomatic WDL file:
  
  Sample workflow:
  
  ```
  version 1.0
   
  Workflow trim {
      input {
          File fastq1 = “R1.fastq.gz”
          File fastq2 = “R2.fastq.gz”
          File program_file = “trim_program.txt”
          File adapter_file = “adapters.fa”
      }
   
      Array[String] trim_program_array = read_lines(program_file)
   
      call trimmomatic {
          input:
              fastq1 = fastq1,
              fastq2 = fastq2,
              trim_program_array = trim_program_array,
              adapter_file = adaptor_file
      }
  }
   
  Task trimmomatic {
      input {
          File fastq1
          File fastq2
          File outfile1 = “trimmed_R1.fastq.gz”
          File outfile2 = “trimmed_R2.fastq.gz”
          Array[String] trim_program_array
          File adapter_file
       }
   
       command {
          TrimmomaticPE \
          ~{fastq1} ~{fastq2} \
          ~{outfile1} ~{outfile2} \
          ~{sep = “ “ trim_program_array}
       }
   
       output {
          File out1 = outfile1
          File out2 = outfile2
       }
  }
  
  ```
  
  My goal is for the first element of ~{sep = " " trim\_program\_array} (aka the first line of trim\_program.txt) to evaluate to “ILLUMINACLIP:adapters.fa:2:30:10:7:keepBothReads” rather than “ILLUMINACLIP:~{adapter\_file}:2:30:10:7:keepBothReads”. I know this is because ~{adapter\_file} is nested inside trim\_program\_array[0], so it’s only expanding the “outer” variable. How can I convince WDL to expand the variable nested inside the first element of that array before executing the bash command?
  
  **Tags:** wdl
  
  > **Comments:**
  > **username_1**: Hi, I've corrected your question to make the spelling of the word 'adapter' consistent, but noticed that some of those changes were in the configuration file. Can you please re-check to make sure this spelling inconsistency is not causing your problems?  
  > **username_2**: The ~{} syntax only works within the wdl command section or wdl string literals. Since the string is getting read from a file in this case, it is probably getting escaped. It looks like this file is just getting used to provide arguments in the command. Can't you just include the contents of the program file in the command section directly?  
  > **username_0**: @username_2 Yes, that's what I eventually figured out, but it would be nice to have it in the program file so that users of the pipeline aren't left wondering what ":2:30:10:7:keepBothReads" in the first line of the program file refers to, since 'ILLUMINACLIP:~{adaptor_file}' is missing. There are worse problems to have though, so I'm fine with that work-around. Thanks for the suggestion.  
  > **username_0**: @username_2 Upon thinking about this further, I realized that one advantage to supplying a program file to Trimmomatic is that the user gets to determine which parameters to trim by, rather than just choosing the value for each predetermined parameter. By encoding "ILLUMINACLIP:~{adaptor_file}" into the code block, I'm in effect making that a required parameter. I've seen many cases where a user wouldn't want to supply that parameter to Trimmomatic at all. I'm hoping that someone will come along and help me with a better solution to this problem so that I don't have to hard code any parameters.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-10-04. Score: 1
  
  Putting the placeholder inside of an input file won't work, since the contents will just be turned into a string when read with `read_lines` and won't get treated like WDL code. Since the lines from the `program_file` are simply getting pasted into the command, you can move the contents into the command section. In a placeholder (`~{}`), adding a string to an optional value will result in an empty string if that optional value is not supplied (see the spec [here](https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#prepending-a-string-to-an-optional-parameter)). So you can use optional inputs to make the different lines non-mandatory:
  
  ```
  task trimmomatic {
      input {
          File fastq1
          File fastq2
          String outfile1 = “trimmed_R1.fastq.gz”
          String outfile2 = “trimmed_R2.fastq.gz”
          File? adapter_file
          Int? minlen
          String? slidingwindow
          Int? leading
          Int? trailing
       }
   
       command {
          TrimmomaticPE \
          ~{fastq1} ~{fastq2} \
          ~{outfile1} ~{outfile2} \
          ~{"ILLUMINACLIP:" + adapter_file + ":2:30:10:7:keepBothReads"} \
          ~{"MINLEN:" + minlen} \
          ~{"SLIDINGWINDOW:" + slidingwindow} \
          ~{"LEADING:" + leading} \
          ~{"TRAILING:" + trailing}
       }
   
       output {
          File out1 = outfile1
          File out2 = outfile2
       }
  }
  
  ```
  
  If, say, `adapter_file` were to be omitted in the inputs, then the produced command should look something like this:
  
  ```
  TrimmomaticPE \
  R1.fq R2.fq \
  trimmed_R1.fastq.gz trimmed_R2.fastq.gz \
   \
  MINLEN:36 \
  SLIDINGWINDOW:4:15 \
  LEADING:3 \
  TRAILING:3
  
  ```
  
  Also note that the output file paths in the input section should be `String`s. Since they don't exist yet, the execution engine will complain that it can't find them if they are `File`s
  
  Ps. I'm not familiar with the actual trimmomatic command options, so please do check that this produces the desired command. I suspect that the `:2:30:10:7:keepBothReads` bit could be split up into additional inputs as well.
  
  
  ---
  
  