Answers.Body: <p>You should check out <a href="https://www.ncbi.nlm.nih.gov/datasets" rel="nofollow noreferrer">NCBI Datasets</a> for this sort of thing. Since this is bacteria, you won't be able to use the web interface but there's an <a href="https://www.ncbi.nlm.nih.gov/datasets/docs/datasets-api/" rel="nofollow noreferrer">API</a> and a <a href="https://www.ncbi.nlm.nih.gov/datasets/docs/command-line-start/" rel="nofollow noreferrer">command-line tool</a> that you can use.</p>
  <p>For example, you can use the <code>datasets</code> command line tool to download the E. coli RefSeq genome assemblies as follows:</p>
  <pre><code>## see how many genomes are in scope
  $ datasets summary genome taxon 562 --limit none --refseq
  {&quot;total_count&quot;:22287}
  
  ## download data excluding gff3, rna and genome seq files
  ## as an example but drop --exclude-seq to download genome seq as well
  $ datasets download genome taxon 562 --refseq --exclude-gff3 --exclude-rna --exclude-seq --filename ecoli_genomes.zip
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-01T14:15:59.020Z
Answers.LastActivityDate: 2021-04-01T14:15:59.020Z
Answers.UserDisplayName: vkkodali
Answers.Body: <p>Do you mean the NCBI taxonomy IDs? If so, I've personally always preferred the way <a href="https://www.uniprot.org/taxonomy/" rel="noreferrer">UniProt displays NCBI taxonomy</a>, and it's easier to download IDs in mass.</p>
  <p>For example, if you find <em>E.coli</em>'s entry: <a href="https://www.uniprot.org/taxonomy/562" rel="noreferrer">https://www.uniprot.org/taxonomy/562</a></p>
  <p>You can then click on &quot;all lower taxonomy nodes&quot;, which will show all <em>E.coli</em> strains. If you click 'Preview first 10' before downloading, you'll see that the first column is the NCBI taxonomy ID of each strain of <em>E.coli</em>.</p>
  <p>For example, <em>E.coli</em> strain 0.1197 is reported to have the ID <strong>1005443</strong> from this preview page. Indeed, if we search this ID on NCBI's taxonomy page, we do get the correct entry: <a href="https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?id=1005443" rel="noreferrer">https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?id=1005443</a></p>
  <p>If this is what you want, click 'download all' to get a nice tab-separated file of all the strains of <em>E.coli</em> and their NCBI taxonomy IDs. This file could then be read and processed before you send API requests to Entrez.</p>
  <p>Hope this helps!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-04-01T11:25:41.290Z
Answers.LastActivityDate: 2021-04-01T11:33:07.760Z
Answers.LastEditDate: 2021-04-01T11:33:07.760Z
Answers.UserDisplayName: Bioinfomrat
Body: <p>I would like to gather a list of ALL E coli NCBI (Entrez/Nucleotide) IDs so that I can download fna and faa files, through a function like this:</p>
  <pre><code># python code #
  from Bio import Entrez
  from glob import glob
  
  def dl_genome(id, folder='genomes'):
      files=glob('%s/*.fna'%folder)
      out_file = '%s/%s.fna'%(folder, id)
  
      if out_file in files:
          print (out_file, 'already downloaded')
          return
      else:
          print ('downloading %s from NCBI'%id)
          
      Entrez.email = &quot;&quot;
      handle = Entrez.efetch(db=&quot;nucleotide&quot;, id=id, rettype=&quot;fna&quot;, retmode=&quot;text&quot;)
      with open(out_file, 'w') as fout:
          fout.write(handle.read())
  
  for strain in targetStrainIDs:
      dl_genome(strain, folder='genomes')
  </code></pre>
  <p>My problem is that I don't have a good way to get the E coli strain names and their corresponding NCBI IDs for Entrez. Is there a way to get these NCBI IDs without manually searching for them?</p>
  <p><strong>Context</strong></p>
  <p>I have been trying to develop an E coli pan-genome for one of my projects. I initially just downloaded the relevant genomes (fna, faa, &amp; features.tab files) from the PATRIC database. However, I've come to realize that the reference sequence on PATRIC has issues. Specifically, the reference sequence on PATRIC's fna/faa files for E coli K12 MG1655 notably do NOT list genes like b2031 or b2033, despite both being present in this strain.</p>
  <p>The NCBI reference genome DOES include these genes (with proper annotations), so I'd like to just download what I need from NCBI instead. However, I'd like a systematic way to download all the fna/faa files since my pan-genome is eventually going to be built from hundreds of of these fna/faa files.</p>
  <p>Any help would be appreciated.</p>
  
Tags: <phylogenetics><entrez>
Title: How to obtain NCBI Nucleotide (Entrez) IDs for all E coli sequences?
ThreadText: # How to obtain NCBI Nucleotide (Entrez) IDs for all E coli sequences?
  
  (Asked by: username_0 on 2021-04-01)
  
  I would like to gather a list of ALL E coli NCBI (Entrez/Nucleotide) IDs so that I can download fna and faa files, through a function like this:
  
  ```
  # python code #
  from Bio import Entrez
  from glob import glob
  
  def dl_genome(id, folder='genomes'):
      files=glob('%s/*.fna'%folder)
      out_file = '%s/%s.fna'%(folder, id)
  
      if out_file in files:
          print (out_file, 'already downloaded')
          return
      else:
          print ('downloading %s from NCBI'%id)
          
      Entrez.email = ""
      handle = Entrez.efetch(db="nucleotide", id=id, rettype="fna", retmode="text")
      with open(out_file, 'w') as fout:
          fout.write(handle.read())
  
  for strain in targetStrainIDs:
      dl_genome(strain, folder='genomes')
  
  ```
  
  My problem is that I don't have a good way to get the E coli strain names and their corresponding NCBI IDs for Entrez. Is there a way to get these NCBI IDs without manually searching for them?
  
  **Context**
  
  I have been trying to develop an E coli pan-genome for one of my projects. I initially just downloaded the relevant genomes (fna, faa, & features.tab files) from the PATRIC database. However, I've come to realize that the reference sequence on PATRIC has issues. Specifically, the reference sequence on PATRIC's fna/faa files for E coli K12 MG1655 notably do NOT list genes like b2031 or b2033, despite both being present in this strain.
  
  The NCBI reference genome DOES include these genes (with proper annotations), so I'd like to just download what I need from NCBI instead. However, I'd like a systematic way to download all the fna/faa files since my pan-genome is eventually going to be built from hundreds of of these fna/faa files.
  
  Any help would be appreciated.
  
  **Tags:** phylogenetics, entrez
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2021-04-01. Score: 1 | Accepted answer
  
  You should check out [NCBI Datasets](https://www.ncbi.nlm.nih.gov/datasets) for this sort of thing. Since this is bacteria, you won't be able to use the web interface but there's an [API](https://www.ncbi.nlm.nih.gov/datasets/docs/datasets-api/) and a [command-line tool](https://www.ncbi.nlm.nih.gov/datasets/docs/command-line-start/) that you can use.
  
  For example, you can use the `datasets` command line tool to download the E. coli RefSeq genome assemblies as follows:
  
  ```
  ## see how many genomes are in scope
  $ datasets summary genome taxon 562 --limit none --refseq
  {"total_count":22287}
  
  ## download data excluding gff3, rna and genome seq files
  ## as an example but drop --exclude-seq to download genome seq as well
  $ datasets download genome taxon 562 --refseq --exclude-gff3 --exclude-rna --exclude-seq --filename ecoli_genomes.zip
  
  ```
  
  
  ---
  
  ### Answer by username_2 on 2021-04-01. Score: 6
  
  Do you mean the NCBI taxonomy IDs? If so, I've personally always preferred the way [UniProt displays NCBI taxonomy](https://www.uniprot.org/taxonomy/), and it's easier to download IDs in mass.
  
  For example, if you find *E.coli*'s entry: <https://www.uniprot.org/taxonomy/562>
  
  You can then click on "all lower taxonomy nodes", which will show all *E.coli* strains. If you click 'Preview first 10' before downloading, you'll see that the first column is the NCBI taxonomy ID of each strain of *E.coli*.
  
  For example, *E.coli* strain 0.1197 is reported to have the ID **1005443** from this preview page. Indeed, if we search this ID on NCBI's taxonomy page, we do get the correct entry: <https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?id=1005443>
  
  If this is what you want, click 'download all' to get a nice tab-separated file of all the strains of *E.coli* and their NCBI taxonomy IDs. This file could then be read and processed before you send API requests to Entrez.
  
  Hope this helps!
  
  
  ---
  
  