Answers.Body: <p>A member of the SRA submission staff pointed out that using</p>
  
  <pre><code>prefetch --type all SRR5253957
  </code></pre>
  
  <p>will download the original files. In this case, it means running the above within an EC2 instance colocated with the S3 bucket (so, <strong>us-east-1</strong>) and having installed and configured SRA Toolkit to work from AWS (<a href="https://www.ncbi.nlm.nih.gov/sra/docs/sra-aws-download/" rel="nofollow noreferrer">as per this documentation</a>).</p>
  
  <p>Unfortunately, the particular files I am concerned with are not currently accessible, but this should generically work in most situations.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-17T22:52:52.590Z
Answers.LastActivityDate: 2020-04-17T22:52:52.590Z
Answers.UserDisplayName: merv
Body: <p>I want to download the original BAM files that the authors had uploaded to SRA. Normally, I would just use <code>sam-dump</code>, but the files are having issues that seem related to <a href="https://github.com/ncbi/sra-tools/issues/23" rel="nofollow noreferrer">this issue</a>. Since according to the entry, AWS S3 also hosts the original BAM files, I thought I could download these directly.</p>
  
  <h2>Example SRA Entry</h2>
  
  <p><img src="https://i.sstatic.net/LP8Y7.png" alt="enter image description here"></p>
  
  <p>NCBI documentation implies that I can't download this directly, but I can freely copy to other AWS locations within the region. To this end, I created my own S3 bucket (<code>mm-mneuron</code>) and am now trying to copy from the SRA bucket to mine.  Here's what I try:</p>
  
  <pre><code>import boto3
  import botocore
  
  s3 = boto3.resource('s3')
  
  bam_file = {
    'Bucket': 'sra-pub-src-6',
    'Key': 'SRR5253957/RPI25_0.bam'
  }
  
  my_bucket = s3.Bucket('mm-mneuron')
  
  my_bucket.copy(bam_file, 'RPI25_0.bam')
  </code></pre>
  
  <p>This fails with:</p>
  
  <blockquote>
    <p>botocore.exceptions.ClientError: An error occurred (403) when calling
    the HeadObject operation: Forbidden</p>
  </blockquote>
  
  <p>That is, it sounds like I can't access the SRA bucket. I've tested downloading and uploading to my bucket, so I know I have write permissions. Not sure what else to try here.</p>
  
  <p><strong>How can I access the SRA data on S3?</strong> </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-17T23:57:55.610Z
Comments.Text: It seems like is not a public bucket. You should download inside an (SRA AMI)[https://www.ncbi.nlm.nih.gov/sra/docs/sra-aws-download/] from sra-pub-src-6 to your instance, and then, in you local terminal download from SRA AMI in EC2 to your computer.
Comments.UserDisplayName: zorbax
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-18T04:21:34.847Z
Comments.Text: @zorbax yes that's what I did end up doing ([see answer](https://bioinformatics.stackexchange.com/a/12962/900)), but there is something specifically wrong with the deposits I'm trying to download. In general, are SRA buckets only accessible through SRA tools? i.e., one has to run an EC2 instance? It seems like direct transfers across S3 buckets would an obvious interface to provide.
Comments.UserDisplayName: merv
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-23T13:35:00.220Z
Comments.Text: You only have access to the public data sets with s3, and the key prj_phs710EA_test.ngc in EC2 only bring acces to 1000 Genomes data. https://www.biostars.org/p/393172/
Comments.UserDisplayName: zorbax
Tags: <rna-seq><python><phylogenetics><aws>
Title: Downloading SRA Files from AWS
ThreadText: # Downloading SRA Files from AWS
  
  (Asked by: username_0 on 2020-04-16)
  
  I want to download the original BAM files that the authors had uploaded to SRA. Normally, I would just use `sam-dump`, but the files are having issues that seem related to [this issue](https://github.com/ncbi/sra-tools/issues/23). Since according to the entry, AWS S3 also hosts the original BAM files, I thought I could download these directly.
  
  ## Example SRA Entry
  
  ![enter image description here](https://i.sstatic.net/LP8Y7.png)
  
  NCBI documentation implies that I can't download this directly, but I can freely copy to other AWS locations within the region. To this end, I created my own S3 bucket (`mm-mneuron`) and am now trying to copy from the SRA bucket to mine. Here's what I try:
  
  ```
  import boto3
  import botocore
  
  s3 = boto3.resource('s3')
  
  bam_file = {
    'Bucket': 'sra-pub-src-6',
    'Key': 'SRR5253957/RPI25_0.bam'
  }
  
  my_bucket = s3.Bucket('mm-mneuron')
  
  my_bucket.copy(bam_file, 'RPI25_0.bam')
  
  ```
  
  This fails with:
  
  > botocore.exceptions.ClientError: An error occurred (403) when calling
  > the HeadObject operation: Forbidden
  
  That is, it sounds like I can't access the SRA bucket. I've tested downloading and uploading to my bucket, so I know I have write permissions. Not sure what else to try here.
  
  **How can I access the SRA data on S3?**
  
  **Tags:** rna-seq, python, phylogenetics, aws
  
  > **Comments:**
  > **username_1**: It seems like is not a public bucket. You should download inside an (SRA AMI)[https://www.ncbi.nlm.nih.gov/sra/docs/sra-aws-download/] from sra-pub-src-6 to your instance, and then, in you local terminal download from SRA AMI in EC2 to your computer.  
  > **username_0**: @username_1 yes that's what I did end up doing ([see answer](https://bioinformatics.stackexchange.com/a/12962/900)), but there is something specifically wrong with the deposits I'm trying to download. In general, are SRA buckets only accessible through SRA tools? i.e., one has to run an EC2 instance? It seems like direct transfers across S3 buckets would an obvious interface to provide.  
  > **username_1**: You only have access to the public data sets with s3, and the key prj_phs710EA_test.ngc in EC2 only bring acces to 1000 Genomes data. https://www.biostars.org/p/393172/  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2020-04-17. Score: 2
  
  A member of the SRA submission staff pointed out that using
  
  ```
  prefetch --type all SRR5253957
  
  ```
  
  will download the original files. In this case, it means running the above within an EC2 instance colocated with the S3 bucket (so, **us-east-1**) and having installed and configured SRA Toolkit to work from AWS ([as per this documentation](https://www.ncbi.nlm.nih.gov/sra/docs/sra-aws-download/)).
  
  Unfortunately, the particular files I am concerned with are not currently accessible, but this should generically work in most situations.
  
  
  ---
  
  