Answers.Body: <p>You could try <a href="https://github.com/pmenzel/download-refseq-genomes" rel="nofollow noreferrer">download-refseq-genomes</a>. It will fetch all genomes from the NCBI FTP server that are in a specific subtree of the phylogeny. </p>
  
  <p>For example, downloading the amino acid sequences from all proteins in each genome assembly for all species belonging to Chloroflexi (NCBI taxon ID = <a href="https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?mode=Tree&amp;id=200795&amp;lvl=3&amp;keep=1&amp;srchmode=1&amp;unlock" rel="nofollow noreferrer">200795</a>):</p>
  
  <pre><code>download-refseq-genomes.pl -t faa 200795
  </code></pre>
  
  <p>This will download all assemblies with status=Complete. Use option <code>-a</code> to download all assemblies with any status.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-25T09:01:52.020Z
Answers.LastActivityDate: 2020-02-25T09:01:52.020Z
Answers.UserDisplayName: Peter Menzel
Answers.Body: <p>You can use NCBI resources to download these data directly. For the entire proteome, you can download the sequences of all proteins annotated on the genome assembly in FASTA format. In the NCBI FTP directories, these files have the names with the suffix <code>_protein.faa.gz</code>. More details about the files in the FTP directories are described in the <a href="https://ftp.ncbi.nlm.nih.gov/genomes/all/README.txt" rel="nofollow noreferrer">README</a>. </p>
  
  <h2>Command-line method using EDirect</h2>
  
  <p>This method uses <a href="https://www.ncbi.nlm.nih.gov/books/NBK179288/" rel="nofollow noreferrer">Entrez Direct</a> to first query the Assembly database for all available genomes, parses the output to extract the ftp path and downloads data using wget command. </p>
  
  <pre><code>esearch -db assembly -query 'txid200795[organism] AND lastest_refseq[filter]' \
    | esummary \
    | xtract -pattern DocumentSummary -element FtpPath_RefSeq \
    | while read -r url ; do 
      path=$(echo $url | perl -pe 's/(GC[FA]_\d+.*)/\1\/\1_protein.faa.gz/g') ; 
      wget -q --show-progress "$path" -P protein_data ; 
    done
  </code></pre>
  
  <p>Here, I am restricting the results to latest RefSeq data by using the <code>latest_refseq[filter]</code> but you can use any other Entrez filters to further narrow down your results. Along the same lines, you can modify the code minimally as shown below to download multiple file types: </p>
  
  <pre><code>esearch -db assembly -query 'txid200795[organism] AND latest_refseq[filter]' \
    | esummary \
    | xtract -pattern DocumentSummary -element FtpPath_RefSeq \
    | while read -r url ; do 
      base_path=$(echo $url | perl -pe 's/(GCF_\d+.*)/\1\/\1_/g') ; 
      for suf in genomic.fna.gz protein.faa.gz ; do
        wget -q --show-progress "$base_path$suf" -P ncbi_data ; 
      done ;
    done
  </code></pre>
  
  <h2>GUI method</h2>
  
  <p>You can use the same query as shown above in the <a href="https://www.ncbi.nlm.nih.gov/assembly/" rel="nofollow noreferrer">NCBI Assembly</a> portal and then use the 'Download Assemblies' button to pick and choose the type of files you would like to download. By default, if you do not select any assemblies in the results, this interface downloads files for all results.
  <a href="https://i.sstatic.net/ZbQY9.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ZbQY9.png" alt="enter image description here"></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-26T14:30:56.180Z
Answers.LastActivityDate: 2020-02-26T15:02:06.597Z
Answers.LastEditDate: 2020-02-26T15:02:06.597Z
Answers.UserDisplayName: vkkodali
Body: <p>I'm looking for a way to download multiple proteomes at once from one clade, as fasta. Probably from the NCBI, because it looks more user-friendly than Ensembl, but Ensembl is okay too. In the best case scenario, I would like to constrain the download to a specific clade (eg. Actinobacteria, Chloroflexi etc.), maybe with a specific RefSeq Category (eg. reference/representative genome) or even depending on the genome assembly level. Command line would probably be the best option, but if there's a direct download option it would be nice to know it as well.
  Does anything like this exist? Help would be much appreciated!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-24T19:35:20.217Z
Comments.Text: The FTP has it: ftp://ftp.ncbi.nlm.nih.gov/genomes/ therein look for the `fa.gz` file
Comments.UserDisplayName: Matteo Ferla
Tags: <phylogenetics><ensembl><data-download><proteomics>
Title: How to download multiple proteomes at once?
ThreadText: # How to download multiple proteomes at once?
  
  (Asked by: username_0 on 2020-02-24)
  
  I'm looking for a way to download multiple proteomes at once from one clade, as fasta. Probably from the NCBI, because it looks more user-friendly than Ensembl, but Ensembl is okay too. In the best case scenario, I would like to constrain the download to a specific clade (eg. Actinobacteria, Chloroflexi etc.), maybe with a specific RefSeq Category (eg. reference/representative genome) or even depending on the genome assembly level. Command line would probably be the best option, but if there's a direct download option it would be nice to know it as well.
  Does anything like this exist? Help would be much appreciated!
  
  **Tags:** phylogenetics, ensembl, data-download, proteomics
  
  > **Comments:**
  > **username_1**: The FTP has it: ftp://ftp.ncbi.nlm.nih.gov/genomes/ therein look for the `fa.gz` file  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2020-02-25. Score: 1 | Accepted answer
  
  You could try [download-refseq-genomes](https://github.com/pmenzel/download-refseq-genomes). It will fetch all genomes from the NCBI FTP server that are in a specific subtree of the phylogeny.
  
  For example, downloading the amino acid sequences from all proteins in each genome assembly for all species belonging to Chloroflexi (NCBI taxon ID = [200795](https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?mode=Tree&id=200795&lvl=3&keep=1&srchmode=1&unlock)):
  
  ```
  download-refseq-genomes.pl -t faa 200795
  
  ```
  
  This will download all assemblies with status=Complete. Use option `-a` to download all assemblies with any status.
  
  
  ---
  
  ### Answer by username_3 on 2020-02-26. Score: 0
  
  You can use NCBI resources to download these data directly. For the entire proteome, you can download the sequences of all proteins annotated on the genome assembly in FASTA format. In the NCBI FTP directories, these files have the names with the suffix `_protein.faa.gz`. More details about the files in the FTP directories are described in the [README](https://ftp.ncbi.nlm.nih.gov/genomes/all/README.txt).
  
  ## Command-line method using EDirect
  
  This method uses [Entrez Direct](https://www.ncbi.nlm.nih.gov/books/NBK179288/) to first query the Assembly database for all available genomes, parses the output to extract the ftp path and downloads data using wget command.
  
  ```
  esearch -db assembly -query 'txid200795[organism] AND lastest_refseq[filter]' \
    | esummary \
    | xtract -pattern DocumentSummary -element FtpPath_RefSeq \
    | while read -r url ; do 
      path=$(echo $url | perl -pe 's/(GC[FA]_\d+.*)/\1\/\1_protein.faa.gz/g') ; 
      wget -q --show-progress "$path" -P protein_data ; 
    done
  
  ```
  
  Here, I am restricting the results to latest RefSeq data by using the `latest_refseq[filter]` but you can use any other Entrez filters to further narrow down your results. Along the same lines, you can modify the code minimally as shown below to download multiple file types:
  
  ```
  esearch -db assembly -query 'txid200795[organism] AND latest_refseq[filter]' \
    | esummary \
    | xtract -pattern DocumentSummary -element FtpPath_RefSeq \
    | while read -r url ; do 
      base_path=$(echo $url | perl -pe 's/(GCF_\d+.*)/\1\/\1_/g') ; 
      for suf in genomic.fna.gz protein.faa.gz ; do
        wget -q --show-progress "$base_path$suf" -P ncbi_data ; 
      done ;
    done
  
  ```
  
  ## GUI method
  
  You can use the same query as shown above in the [NCBI Assembly](https://www.ncbi.nlm.nih.gov/assembly/) portal and then use the 'Download Assemblies' button to pick and choose the type of files you would like to download. By default, if you do not select any assemblies in the results, this interface downloads files for all results.
  [![enter image description here](https://i.sstatic.net/ZbQY9.png)](https://i.sstatic.net/ZbQY9.png)
  
  
  ---
  
  