Answers.Body: <p>Unfortunately generating inputs for Cromwell is not something that is handled by Cromwell itself or by GATK, and generally needs to be scripted to fit your custom needs. </p>
  
  <p>Despite this being a bit arduous, there is a very good reason for this. WDL itself is a highly polymorphic workflow language, where inputs can be vastly different between each workflow. Cromwell <em>ONLY</em> understands how to parse the workflow and the inputs, but does not know the context of what it is running (ie this is a paired mapping job), nor does GATK know that it is running within a WDL. </p>
  
  <p>Additionally, I've encountered many different cases where there are not just 2 fastq files but, due to older sequencing technologies, 10 or more for the same sample. Unless GATK/Cromwell knows how your specific samples are arranged there is no easy way for them to generate your inputs.</p>
  
  <p>There are tools that would possibly help you simplify scripting better. <a href="https://cromwell.readthedocs.io/en/stable/WOMtool/" rel="nofollow noreferrer">WOMTool</a> developed by the Broad has an <code>inputs</code> command which will take an  arbitrary workflow and generate a <code>JSON</code> template of the inputs, which you can then use in your own scripts. </p>
  
  <p>As an example, imagine you have this WDL workflow in a file called <code>my.wdl</code>:</p>
  
  <pre><code>version 1.0
  workflow foo {
    input {
      Array[Pair[File,File]] fastqFiles
    }
  }
  </code></pre>
  
  <p>When you run WOMtool's <code>inputs</code> command, it produces a template JSON listing all inputs and specifying the variable types expected for each:</p>
  
  <pre class="lang-sh prettyprint-override"><code>$ java -jar womtool.jar inputs my.wdl
  {
    "foo.fastqFiles": "Array[Pair[File, File]]"
  }
  </code></pre>
  
  <p>You can use this template to produce actual JSONs for each of your samples.</p>
  
  <p>Note that you can also get the template written to a file as follows: </p>
  
  <pre><code>$ java -jar womtool.jar inputs my.wdl &gt; my.inputs.json
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-24T15:54:27.630Z
Answers.LastActivityDate: 2020-04-24T17:26:18.813Z
Answers.LastEditDate: 2020-04-24T17:26:18.813Z
Answers.UserDisplayName: Patrick Magee
Body: <p>I am starting to use Cromwell WDL scripts for running GATK workflows on a slurm cluster. When using the paired-fastq-to-unmapped-bam script I need to define the fastq sepcifics in the appropriate json file. This needs to be written separately for each pair of fastqs. My question is: <strong>Is there a way of generating the json files on a batch of fastq-files by scripting?</strong> Of course, I could start writing scripts myself, but since I assume that GATK is used in a batch environment, I have the hope that something already exists.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-18T09:37:39.307Z
Comments.Text: Hi @luckyman, if you post the non-batch code you have written I'm pretty sure someone will loop it for you.
Comments.UserDisplayName: M__
Tags: <gatk><wdl>
Title: How to generate the json files for Cromwell workflow execution?
ThreadText: # How to generate the json files for Cromwell workflow execution?
  
  (Asked by: username_0 on 2020-04-18)
  
  I am starting to use Cromwell WDL scripts for running GATK workflows on a slurm cluster. When using the paired-fastq-to-unmapped-bam script I need to define the fastq sepcifics in the appropriate json file. This needs to be written separately for each pair of fastqs. My question is: **Is there a way of generating the json files on a batch of fastq-files by scripting?** Of course, I could start writing scripts myself, but since I assume that GATK is used in a batch environment, I have the hope that something already exists.
  
  **Tags:** gatk, wdl
  
  > **Comments:**
  > **username_1**: Hi @luckyman, if you post the non-batch code you have written I'm pretty sure someone will loop it for you.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-04-24. Score: 2
  
  Unfortunately generating inputs for Cromwell is not something that is handled by Cromwell itself or by GATK, and generally needs to be scripted to fit your custom needs.
  
  Despite this being a bit arduous, there is a very good reason for this. WDL itself is a highly polymorphic workflow language, where inputs can be vastly different between each workflow. Cromwell *ONLY* understands how to parse the workflow and the inputs, but does not know the context of what it is running (ie this is a paired mapping job), nor does GATK know that it is running within a WDL.
  
  Additionally, I've encountered many different cases where there are not just 2 fastq files but, due to older sequencing technologies, 10 or more for the same sample. Unless GATK/Cromwell knows how your specific samples are arranged there is no easy way for them to generate your inputs.
  
  There are tools that would possibly help you simplify scripting better. [WOMTool](https://cromwell.readthedocs.io/en/stable/WOMtool/) developed by the Broad has an `inputs` command which will take an arbitrary workflow and generate a `JSON` template of the inputs, which you can then use in your own scripts.
  
  As an example, imagine you have this WDL workflow in a file called `my.wdl`:
  
  ```
  version 1.0
  workflow foo {
    input {
      Array[Pair[File,File]] fastqFiles
    }
  }
  
  ```
  
  When you run WOMtool's `inputs` command, it produces a template JSON listing all inputs and specifying the variable types expected for each:
  
  ```sh
  $ java -jar womtool.jar inputs my.wdl
  {
    "foo.fastqFiles": "Array[Pair[File, File]]"
  }
  
  ```
  
  You can use this template to produce actual JSONs for each of your samples.
  
  Note that you can also get the template written to a file as follows:
  
  ```
  $ java -jar womtool.jar inputs my.wdl > my.inputs.json
  
  ```
  
  
  ---
  
  