Answers.Body: <p>Outline of solution:</p>
  
  <ul>
  <li>get this file: <a href="ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq/bacteria/assembly_summary.txt" rel="nofollow noreferrer">ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq/bacteria/assembly_summary.txt</a></li>
  <li>filter for lines having "Complete Genome" in column 12</li>
  <li>filter for lines having a taxid (column 6) that is a descendant of taxon id 1117 (phylum Cyanobacteria), you can use nodes.dmp from <a href="ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump.tar.gz" rel="nofollow noreferrer">ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump.tar.gz</a></li>
  <li>The file download URL is the concatenation of column 20 and the last field of column 20, after separating it at '/', followed by appending '_genomic.gbff.gz'.  For example: <a href="ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/007/365/GCF_000007365.1_ASM736v1/GCF_000007365.1_ASM736v1_genomic.gbff.gz" rel="nofollow noreferrer">ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/007/365/GCF_000007365.1_ASM736v1/GCF_000007365.1_ASM736v1_genomic.gbff.gz</a></li>
  </ul>
  
  <p>Edit:<br>
  Since I will need something like that soon, I made a <a href="https://github.com/pmenzel/download-refseq-genomes" rel="nofollow noreferrer">Perl script for downloading genomes</a>.<br>
  For Cyanobacteria, you would do <code>./download-refseq-genomes.pl 1117</code></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-28T19:35:37.397Z
Answers.LastActivityDate: 2018-09-29T07:24:04.260Z
Answers.LastEditDate: 2018-09-29T07:24:04.260Z
Answers.UserDisplayName: Peter Menzel
Answers.Body: <p>As an alternative, you can use the <a href="https://www.ncbi.nlm.nih.gov/datasets/docs/command-line-start/" rel="nofollow noreferrer">NCBI Datasets CLI</a>, in a two step process to generate a dehydrated dataset for later <a href="https://www.ncbi.nlm.nih.gov/datasets/docs/rehydrate/" rel="nofollow noreferrer">hydration</a>:</p>
  <pre><code># Download metadata for all Cyanobacteria 
  # And pull out accession for 'Complete' genomes
  ./datasets assembly-descriptors taxon Cyanobacteria --limit ALL \
       | jq -r '.assemblies[].assembly | &quot;\(.assembly_accession) \(.assembly_level)&quot;' \
       | grep Complete \
       | cut -f 1 -d ' ' &gt; accs.txt
  
  # Download a dehydrated dataset for the provided set of accessions
  ./datasets download assembly -i accs.txt -f 1117_complete.zip \
       --dehydrated \
       --exclude-protein \
       --exclude-rna \
       --exclude-seq \
       --exclude-gff3 \
       --include-gbff 
  </code></pre>
  <p>You may be interested in some of the other files that are excluded.</p>
  <p>Once you have the dehydrated dataset, you can rehydrate it when you are ready to analyze the data with:</p>
  <pre><code>unzip 1117_complete.zip -d 1117_complete
  ./datasets rehydrate -f 1117_complete
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-29T20:44:38.243Z
Answers.LastActivityDate: 2020-09-29T20:44:38.243Z
Answers.UserDisplayName: Brad Holmes
Answers.Body: <p>Use the NCBI Datasets command line tool to download genomes by taxon (NCBI Taxonomy ID, scientific or common name at any tax rank).</p>
  <pre><code>datasets download genome taxon cyanobacterial \
    --assembly-source refseq --assembly-level complete_genome \
    --exclude-genomic-cds --exclude-gff3 --exclude-protein \
    --exclude-rna --exclude-seq --include-gbff \
    --filename genomes.zip
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-06T07:40:04.940Z
Answers.LastActivityDate: 2022-12-06T07:40:04.940Z
Answers.UserDisplayName: Forrest Vigor
Body: <p>I need to download all the <strong>completely assembled cyanobacterial genome</strong>'s GenBank file(.gbff) from NCBI(RefSeq or INSDC ftp data).</p>
  
  <p>For this I think, the steps are:</p>
  
  <ol>
  <li>Need to find the completely assembled genomes.</li>
  <li>find the GenBank file URL based on the taxonomic name.</li>
  <li>download the GenBank file(.gbff file)</li>
  </ol>
  
  <p>Is their any way to do this by using any python module or any other idea??</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-09-05T07:23:08.480Z
Comments.Text: Biopython has tools that could help you for steps 2 and 3, but I'm not sure how to get the information of your step 1.
Comments.UserDisplayName: bli
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-09-05T07:25:53.587Z
Comments.Text: can you please tell the tool name for step 2.
Comments.UserDisplayName: Arijit Panda
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-09-05T07:31:45.677Z
Comments.Text: @Arijit Searching for "Download biopython Genebank" lead you [here](http://biopython.org/DIST/docs/tutorial/Tutorial.html#htoc140)
Comments.UserDisplayName: llrs
Tags: <python><phylogenetics>
Title: How to automate NCBI genome download
ThreadText: # How to automate NCBI genome download
  
  (Asked by: username_0 on 2018-09-04)
  
  I need to download all the **completely assembled cyanobacterial genome**'s GenBank file(.gbff) from NCBI(RefSeq or INSDC ftp data).
  
  For this I think, the steps are:
  
  1. Need to find the completely assembled genomes.
  2. find the GenBank file URL based on the taxonomic name.
  3. download the GenBank file(.gbff file)
  
  Is their any way to do this by using any python module or any other idea??
  
  **Tags:** python, phylogenetics
  
  > **Comments:**
  > **username_1**: Biopython has tools that could help you for steps 2 and 3, but I'm not sure how to get the information of your step 1.  
  > **username_0**: can you please tell the tool name for step 2.  
  > **username_2**: @username_0 Searching for "Download biopython Genebank" lead you [here](http://biopython.org/DIST/docs/tutorial/Tutorial.html#htoc140)  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_3 on 2018-09-28. Score: 3 | Accepted answer
  
  Outline of solution:
  
  * get this file: <ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq/bacteria/assembly_summary.txt>
  * filter for lines having "Complete Genome" in column 12
  * filter for lines having a taxid (column 6) that is a descendant of taxon id 1117 (phylum Cyanobacteria), you can use nodes.dmp from <ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump.tar.gz>
  * The file download URL is the concatenation of column 20 and the last field of column 20, after separating it at '/', followed by appending '\_genomic.gbff.gz'. For example: <ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/007/365/GCF_000007365.1_ASM736v1/GCF_000007365.1_ASM736v1_genomic.gbff.gz>
  
  Edit:  
  Since I will need something like that soon, I made a [Perl script for downloading genomes](https://github.com/pmenzel/download-refseq-genomes).  
  For Cyanobacteria, you would do `./download-refseq-genomes.pl 1117`
  
  
  ---
  
  ### Answer by username_4 on 2020-09-29. Score: 2
  
  As an alternative, you can use the [NCBI Datasets CLI](https://www.ncbi.nlm.nih.gov/datasets/docs/command-line-start/), in a two step process to generate a dehydrated dataset for later [hydration](https://www.ncbi.nlm.nih.gov/datasets/docs/rehydrate/):
  
  ```
  # Download metadata for all Cyanobacteria 
  # And pull out accession for 'Complete' genomes
  ./datasets assembly-descriptors taxon Cyanobacteria --limit ALL \
       | jq -r '.assemblies[].assembly | "\(.assembly_accession) \(.assembly_level)"' \
       | grep Complete \
       | cut -f 1 -d ' ' > accs.txt
  
  # Download a dehydrated dataset for the provided set of accessions
  ./datasets download assembly -i accs.txt -f 1117_complete.zip \
       --dehydrated \
       --exclude-protein \
       --exclude-rna \
       --exclude-seq \
       --exclude-gff3 \
       --include-gbff 
  
  ```
  
  You may be interested in some of the other files that are excluded.
  
  Once you have the dehydrated dataset, you can rehydrate it when you are ready to analyze the data with:
  
  ```
  unzip 1117_complete.zip -d 1117_complete
  ./datasets rehydrate -f 1117_complete
  
  ```
  
  
  ---
  
  ### Answer by username_5 on 2022-12-06. Score: 0
  
  Use the NCBI Datasets command line tool to download genomes by taxon (NCBI Taxonomy ID, scientific or common name at any tax rank).
  
  ```
  datasets download genome taxon cyanobacterial \
    --assembly-source refseq --assembly-level complete_genome \
    --exclude-genomic-cds --exclude-gff3 --exclude-protein \
    --exclude-rna --exclude-seq --include-gbff \
    --filename genomes.zip
  
  ```
  
  
  ---
  
  