Answers.Body: <p>SRA moved all of the data to the cloud per <a href="https://ncbiinsights.ncbi.nlm.nih.gov/2020/02/24/sra-cloud/" rel="nofollow noreferrer">https://ncbiinsights.ncbi.nlm.nih.gov/2020/02/24/sra-cloud/</a></p>
  <p>You can still download data for free using <code>sratoolkit</code> as described in the official documentation here: <a href="https://www.ncbi.nlm.nih.gov/sra/docs/sradownload/" rel="nofollow noreferrer">https://www.ncbi.nlm.nih.gov/sra/docs/sradownload/</a></p>
  <p>You can download the data using sratoolkit without having to pay cloud providers. There are some user-pays data on the cloud but for FASTQ files you don't have to pay.</p>
  <p>For documentation on how to download data from the cloud, see <a href="https://www.ncbi.nlm.nih.gov/sra/docs/sra-aws-download/" rel="nofollow noreferrer">https://www.ncbi.nlm.nih.gov/sra/docs/sra-aws-download/</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-02-08T17:40:56.540Z
Answers.Text: Cool and thanks
Answers.UserDisplayName: M__
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-02-08T18:31:05.210Z
Answers.Text: This is certainly useful and relevant, but does not truly answers the question. The article does not say anything about discontinuing the ftp servers, rather offering a new way to access the data.
Answers.UserDisplayName: Kamil S Jaron
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-02-08T18:37:36.350Z
Answers.Text: I don't think downloading from the FTP was never an officially advertised method. The entire transition from NCBI FTP to cloud is not a concern for the users of `sratoolkit`, the preferred method to download data.
Answers.UserDisplayName: vkkodali
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-02-08T17:39:47.763Z
Answers.LastActivityDate: 2022-02-08T21:11:03.597Z
Answers.LastEditDate: 2022-02-08T21:11:03.597Z
Answers.UserDisplayName: vkkodali
Answers.Body: <p>NCBI appeared at least in part to shift onto Google Cloud buckets, one previous OP raised a question about access via GCP (I think the question was closed). Fundamentally, traditional FTP is not sustainable against data growtch. GCP is accessed via gsutil. It would be good if you could contact NCBI and post their official response here. They will response when their database provision is in a state of flux.</p>
  <p>sratoolkit appears to be a front-end to GCPs gsutil. Just to point out, it would be very unusual for NCBI to charge the user for e.g. direct gsutil download, because usually the provider will charge e.g. NCBI on a per download basis. This will likely work out cheaper overall for NCBI.</p>
  <p>Just to formalise my bit, if SRA are hosting on a GCP bucket, Google will not permit ftp access which is at least part the reason behind your question.</p>
  <p>They do have their own ftp equivalent called <code>gsutil</code> which will permit direct download if you know the bucket name. <code>gsutil</code> is part of the ClouD SDK package and can be downloaded via conda/anaconda <code>conda install gcloud-sdk</code>. This will give you command line access if you know the bucket names. There is a past post on this BTW.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-02-08T19:50:07.717Z
Answers.Text: You don't really need to use `gsutil`. For example, if you run `srapath SRR13397412` you will see the path as `https://storage.googleapis.com/nih-sequence-read-archive/run/SRR13397412/SRR13397412` which can be downloaded using `wget`.
Answers.UserDisplayName: vkkodali
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-16T02:10:10.877Z
Answers.Text: `update`:  conda install -c conda-forge google-cloud-sdk
Answers.UserDisplayName: envs_h_gang_5
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-02-08T19:12:07.810Z
Answers.LastActivityDate: 2022-02-08T21:10:08.963Z
Answers.LastEditDate: 2022-02-08T21:10:08.963Z
Answers.UserDisplayName: M__
Body: <p>My whole (professional) life I am downloading sequences from SRA and ENA databases. Long time ago, I used prefetch from sra-tools, but later I switched to simply using wget from ftp servers of the two respective servers (used to work for both SRA and ENA). But it seems that SRA ftp server does not have the same data in there anymore and I kind of worry about reproducibility of my old pipelines - so what happened to sra ftp server?</p>
  <p>The paths that used to work were formatted as follows</p>
  <pre><code>ftp://ftp.ncbi.nlm.nih.gov/sra/sra-instant/reads/ByRun/sra/${ACCESION::3}/${ACCESION::6}/&quot;$ACCESION&quot;/&quot;$ACCESION&quot;.sra
  </code></pre>
  <p>but it seems <code>sra-instant</code> directory is not accessible anymore. Did the files move? Or were the access been just restricted?</p>
  <p>The alternative workaround is to dl the files using <code>prefetch</code> instead of wgetting. Prefetch is part of <a href="https://hpc.nih.gov/apps/sratoolkit.html" rel="nofollow noreferrer">sra-tools</a> and available on <a href="https://anaconda.org/bioconda/sra-tools" rel="nofollow noreferrer">conda</a> and it's not such a bad solution. I guess I am mostly curious what happened to the ftp server and if anyone knows what is coming next, is ENA going to restrict the access too some time soon?</p>
  
Tags: <public-databases><data-download><sratoolkit><sra>
Title: What happened to sra ftp server?
ThreadText: # What happened to sra ftp server?
  
  (Asked by: username_0 on 2022-02-08)
  
  My whole (professional) life I am downloading sequences from SRA and ENA databases. Long time ago, I used prefetch from sra-tools, but later I switched to simply using wget from ftp servers of the two respective servers (used to work for both SRA and ENA). But it seems that SRA ftp server does not have the same data in there anymore and I kind of worry about reproducibility of my old pipelines - so what happened to sra ftp server?
  
  The paths that used to work were formatted as follows
  
  ```
  ftp://ftp.ncbi.nlm.nih.gov/sra/sra-instant/reads/ByRun/sra/${ACCESION::3}/${ACCESION::6}/"$ACCESION"/"$ACCESION".sra
  
  ```
  
  but it seems `sra-instant` directory is not accessible anymore. Did the files move? Or were the access been just restricted?
  
  The alternative workaround is to dl the files using `prefetch` instead of wgetting. Prefetch is part of [sra-tools](https://hpc.nih.gov/apps/sratoolkit.html) and available on [conda](https://anaconda.org/bioconda/sra-tools) and it's not such a bad solution. I guess I am mostly curious what happened to the ftp server and if anyone knows what is coming next, is ENA going to restrict the access too some time soon?
  
  **Tags:** public-databases, data-download, sratoolkit, sra
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-02-08. Score: 3
  
  SRA moved all of the data to the cloud per <https://ncbiinsights.ncbi.nlm.nih.gov/2020/02/24/sra-cloud/>
  
  You can still download data for free using `sratoolkit` as described in the official documentation here: <https://www.ncbi.nlm.nih.gov/sra/docs/sradownload/>
  
  You can download the data using sratoolkit without having to pay cloud providers. There are some user-pays data on the cloud but for FASTQ files you don't have to pay.
  
  For documentation on how to download data from the cloud, see <https://www.ncbi.nlm.nih.gov/sra/docs/sra-aws-download/>
  
  > **Comments:**
  > **username_2**: Cool and thanks  
  > **username_0**: This is certainly useful and relevant, but does not truly answers the question. The article does not say anything about discontinuing the ftp servers, rather offering a new way to access the data.  
  > **username_1**: I don't think downloading from the FTP was never an officially advertised method. The entire transition from NCBI FTP to cloud is not a concern for the users of `sratoolkit`, the preferred method to download data.  
  
  ---
  
  ### Answer by username_2 on 2022-02-08. Score: 2
  
  NCBI appeared at least in part to shift onto Google Cloud buckets, one previous OP raised a question about access via GCP (I think the question was closed). Fundamentally, traditional FTP is not sustainable against data growtch. GCP is accessed via gsutil. It would be good if you could contact NCBI and post their official response here. They will response when their database provision is in a state of flux.
  
  sratoolkit appears to be a front-end to GCPs gsutil. Just to point out, it would be very unusual for NCBI to charge the user for e.g. direct gsutil download, because usually the provider will charge e.g. NCBI on a per download basis. This will likely work out cheaper overall for NCBI.
  
  Just to formalise my bit, if SRA are hosting on a GCP bucket, Google will not permit ftp access which is at least part the reason behind your question.
  
  They do have their own ftp equivalent called `gsutil` which will permit direct download if you know the bucket name. `gsutil` is part of the ClouD SDK package and can be downloaded via conda/anaconda `conda install gcloud-sdk`. This will give you command line access if you know the bucket names. There is a past post on this BTW.
  
  > **Comments:**
  > **username_1**: You don't really need to use `gsutil`. For example, if you run `srapath SRR13397412` you will see the path as `https://storage.googleapis.com/nih-sequence-read-archive/run/SRR13397412/SRR13397412` which can be downloaded using `wget`.  
  > **username_3**: `update`:  conda install -c conda-forge google-cloud-sdk  
  
  ---
  
  