Answers.Body: <p>Somebody helped me with this one through other channels, but I wanted to make it publicly available:</p>
  <p>Make a tarball of the STAR index. The option <code>--strip-components=1</code> strips the first level of the path from the untarred items; thus, the expected tarball was made with something like</p>
  <pre><code>tar -cvf ${star_index} the_star_index_directory/
  </code></pre>
  <p>Then point Cromwell/WDL to the tarball, e.g. on S3. The GTEx container will unwrap the tarball on the fly during execution.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-14T02:41:15.123Z
Answers.LastActivityDate: 2020-09-14T02:41:15.123Z
Answers.UserDisplayName: Kevin L. Keys
Body: <p>I'm trying to run the WDL tasks from the <a href="https://github.com/broadinstitute/gtex-pipeline" rel="nofollow noreferrer">GTEx RNA-Seq pipeline</a> with Cromwell using AWS Batch as a backend.</p>
  <p>I store the STAR alignment index on an Elastic Block Store since my Cromwell server instance lacks sufficient disk space. The EBS Block is mounted to my Cromwell server at <code>/data</code>. The STAR index is located at <code>/data/genomic-references/rnaseq/star_index_oh50</code>.</p>
  <p>Try as I might, I cannot get Cromwell to run the STAR task correctly.</p>
  <p>My JSON inputs file:</p>
  <pre><code> {
     &quot;rnaseq_pipeline_fastq_workflow.prefix&quot;: &quot;s3://mybucket/test/CL100148068&quot;,
     &quot;rnaseq_pipeline_fastq_workflow.star.fastq1&quot;: &quot;s3://anotherbucket/genotypedata/sample1.fq.gz&quot;,
     &quot;rnaseq_pipeline_fastq_workflow.star.star_index&quot;: &quot;/data/genomic-references/rnaseq/star_index_oh50/&quot;,
     &quot;rnaseq_pipeline_fastq_workflow.star.memory&quot;: &quot;32&quot;,
     &quot;rnaseq_pipeline_fastq_workflow.star.num_threads&quot;: &quot;4&quot;,
     &quot;rnaseq_pipeline_fastq_workflow.star.disk_space&quot;: &quot;200&quot;,
     &quot;rnaseq_pipeline_fastq_workflow.star.num_preempt&quot;: &quot;0&quot; 
   }
  </code></pre>
  <p>Error message:</p>
  <pre><code>tar: /data/genomic-references/rnaseq/star_index_oh50/: Cannot open: No such file or directory
  </code></pre>
  <p>Note that <code>/data/genomic-references/rnaseq/star_index_oh50/</code> does exist and is readable.</p>
  <p>The relevant command in the GTEx container is <a href="https://github.com/broadinstitute/gtex-pipeline/blob/b53b734a9b096caed237952b34cbce88b38485bd/rnaseq/star.wdl#L69" rel="nofollow noreferrer">here</a>:</p>
  <pre><code>tar -xvvf ${star_index} -C star_index --strip-components=1
  </code></pre>
  <p>If the files are stored locally, then why does <code>tar</code> fail to wrap the directory?</p>
  
Tags: <rna-seq><genomics><wdl><gtex><aws>
Title: Pointing Cromwell to local EBS mounted storage on AWS Batch
ThreadText: # Pointing Cromwell to local EBS mounted storage on AWS Batch
  
  (Asked by: username_0 on 2020-09-11)
  
  I'm trying to run the WDL tasks from the [GTEx RNA-Seq pipeline](https://github.com/broadinstitute/gtex-pipeline) with Cromwell using AWS Batch as a backend.
  
  I store the STAR alignment index on an Elastic Block Store since my Cromwell server instance lacks sufficient disk space. The EBS Block is mounted to my Cromwell server at `/data`. The STAR index is located at `/data/genomic-references/rnaseq/star_index_oh50`.
  
  Try as I might, I cannot get Cromwell to run the STAR task correctly.
  
  My JSON inputs file:
  
  ```
   {
     "rnaseq_pipeline_fastq_workflow.prefix": "s3://mybucket/test/CL100148068",
     "rnaseq_pipeline_fastq_workflow.star.fastq1": "s3://anotherbucket/genotypedata/sample1.fq.gz",
     "rnaseq_pipeline_fastq_workflow.star.star_index": "/data/genomic-references/rnaseq/star_index_oh50/",
     "rnaseq_pipeline_fastq_workflow.star.memory": "32",
     "rnaseq_pipeline_fastq_workflow.star.num_threads": "4",
     "rnaseq_pipeline_fastq_workflow.star.disk_space": "200",
     "rnaseq_pipeline_fastq_workflow.star.num_preempt": "0" 
   }
  
  ```
  
  Error message:
  
  ```
  tar: /data/genomic-references/rnaseq/star_index_oh50/: Cannot open: No such file or directory
  
  ```
  
  Note that `/data/genomic-references/rnaseq/star_index_oh50/` does exist and is readable.
  
  The relevant command in the GTEx container is [here](https://github.com/broadinstitute/gtex-pipeline/blob/b53b734a9b096caed237952b34cbce88b38485bd/rnaseq/star.wdl#L69):
  
  ```
  tar -xvvf ${star_index} -C star_index --strip-components=1
  
  ```
  
  If the files are stored locally, then why does `tar` fail to wrap the directory?
  
  **Tags:** rna-seq, genomics, wdl, gtex, aws
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2020-09-14. Score: 1 | Accepted answer
  
  Somebody helped me with this one through other channels, but I wanted to make it publicly available:
  
  Make a tarball of the STAR index. The option `--strip-components=1` strips the first level of the path from the untarred items; thus, the expected tarball was made with something like
  
  ```
  tar -cvf ${star_index} the_star_index_directory/
  
  ```
  
  Then point Cromwell/WDL to the tarball, e.g. on S3. The GTEx container will unwrap the tarball on the fly during execution.
  
  
  ---
  
  