Answers.Body: <p>The general procedure is explained <a href="https://www.ncbi.nlm.nih.gov/guide/howto/retrieve-seq-org/" rel="nofollow noreferrer">here</a>:</p>
  <p>First step is to search the <a href="https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi" rel="nofollow noreferrer">Taxonomy database</a> with the organism name. Accepted common names usually work at all taxonomic levels. Use the scientific name or formal name if no results are obtained with the common name.</p>
  <p>In code:</p>
  <pre><code>from Bio import Entrez
  
  Entrez.email = 'user@example.org' # Put your email here
  
  def get_tax_id(species):
      species = species.replace('_', '+').strip()
      search = Entrez.esearch(term=species, db='taxonomy', retmode='xml')
      record = Entrez.read(search)
      return record['IdList'][0]
  
  if __name__ == '__main__':
      organisms = ['lophodermium_seditiosum', 'trichoderma_harzianum']
      taxids = [get_tax_id(organism) for organism in organisms]
      print(taxids)
  </code></pre>
  <p>This gives us the <code>taxid</code>s for our species, respectively <code>128023</code> and <code>5544</code>. Now we can use these to get the proteomes. For <em>Lophodermium seditiosum</em> we find two results, but <a href="https://www.ncbi.nlm.nih.gov/protein/?term=txid5544%5BOrganism:exp%5D" rel="nofollow noreferrer">for <em>Trichoderma harzianum</em> there are 65900 results</a>, so you might want to tweak the query to filter the results. For example, if you only want the sequences appearing in PDB you add <code>AND pdb[filter]</code> so that the query becomes <code>txid5544[Organism:exp] AND pdb[filter]</code>.</p>
  <pre><code>def get_proteomes(taxid):
      query = 'txid{}[Organism:exp]'.format(taxid)
      handle = Entrez.esearch(db='protein', term=query)
      result = Entrez.read(handle)
      if int(result['Count']) &lt;= 2000:
          ids = ','.join(result['IdList'])
          proteomes = Entrez.efetch(db='protein', id=ids, rettype='fasta').read()
          return proteomes
      else:
          raise RuntimeError('Too many results!')
  
  if __name__ == '__main__':
      print(get_proteomes('128023'))
  </code></pre>
  <p>Returns:</p>
  <pre><code>&gt;ADI44294.1 actin, partial [Lophodermium seditiosum]
  CKAGFAGDDAPRAVFPSIVGRPRHHGIMIGMGQKDS
  
  &gt;ADI44293.1 actin, partial [Lophodermium seditiosum]
  CKAGFAGDDAPRAVFPSIVGRPRHHGIMIGMGQKDS
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-22T09:27:06.803Z
Answers.Text: So I'm getting a 'Supplied ID parameter is empty' error from this code, the functions are defined at the beginning and are the same as in the answer
  
  `listoforganisms = [x.split('\t')[0] for x in open("OGTlist.csv").readlines()]
  
  IDlist = []
  
  
  #convert to id
  if __name__ == '__main__':
      organisms = listoforganisms
      for organism in organisms:
          taxid = get_tax_id(organism)
          IDlist.append(taxid)
          print (taxid)
          print(get_proteomes((taxid)))
  `
Answers.UserDisplayName: Biomage
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-22T09:40:57.353Z
Answers.Text: Can you post a small sample of your `OGTlist.csv`file?
Answers.UserDisplayName: BioGeek
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-22T13:24:19.443Z
Answers.Text: Got it working with, `if __name__ == '__main__':
      organisms = listoforganisms
      for organism in organisms:
          taxid = get_tax_id(organism)
          print(organism)
          print (taxid)
          stringid = str(taxid)
  
          print(get_proteomes(stringid.strip("'[]'")))`
  
  Just needed to strip the list elements to the right form
Answers.UserDisplayName: Biomage
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-19T08:40:47.010Z
Answers.LastActivityDate: 2021-10-27T07:55:20.457Z
Answers.LastEditDate: 2021-10-27T07:55:20.457Z
Answers.UserDisplayName: BioGeek
Answers.Body: <p>You can use the <a href="https://www.ncbi.nlm.nih.gov/datasets" rel="nofollow noreferrer">NCBI Datasets</a> to download the protein sequences of all annotated protein-coding genes for an organism starting with the organism name. There is a web-interface that allows exploring by taxonomy or specific organism search as well as a command line utility and an API. For example, the command line utility can be used as follows:</p>
  <pre class="lang-bash prettyprint-override"><code>$ datasets download genome taxon 'trichoderma harzianum' \
      --annotated --exclude-genomic-cds --exclude-gff3 \
      --exclude-rna --exclude-seq --reference
  Collecting 1 genome accessions [================================================] 100% 1/1
  Downloading: ncbi_dataset.zip    3.92MB done
  </code></pre>
  <p>This downloads a zip archive containing the <code>protein.faa</code> file for the <em>Trichoderma harzianum</em> reference assembly.</p>
  <p>Note, unlike the Entrez method described above, this will not download all proteins for a given organism from the <a href="https://www.ncbi.nlm.nih.gov/protein" rel="nofollow noreferrer">NCBI Protein</a> portal. Thus, <em>Lophodermium seditiosum</em> does not return any results as there is no available genome for this assembly in the <a href="https://www.ncbi.nlm.nih.gov/assembly" rel="nofollow noreferrer">NCBI Assembly</a> database.</p>
  <p>On the other hand, the NCBI Datasets method will be significantly faster and return cleaner results. I would argue that if you are interested in obtaining the 'proteomes' of a given set of organisms, this is a better choice as it includes only those cases where a complete proteome is available based on the annotation of the genome and does not include cases where only a few proteins from an organism are available.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-27T11:04:17.727Z
Answers.LastActivityDate: 2021-10-27T11:04:17.727Z
Answers.UserDisplayName: vkkodali
Body: <p>I'm trying to download around 2000 proteomes from NCBI, but all I have is the genus and species e.g: <code>lophodermium_seditiosum trichoderma_harzianum</code></p>
  
  <p>Does anyone know a good way of doing this in script form?<br>
  My code at the moment is:</p>
  
  <pre><code>from Bio import Entrez
  from Bio import SeqIO
  import os
  
  file_name = "proteome1.faa"
  net_handle = Entrez.efetch(db="protein", id="trichoderma_harzianum", rettype="gp", retmode="text")
  out_handle = open(file_name, "w")
  out_handle.write(net_handle.read())
  out_handle.close()
  net_handle.close()
  print("Saved")
  
  
  for seq_record in SeqIO.parse("proteome1.faa", "genbank"):
      print(seq_record.id)
      print(repr(seq_record.seq))
      print(len(seq_record))
  </code></pre>
  
  <p>But I think it wants a number as the id, I don't know if it's possible to convert species name to a numerical ID within the script, or if that's the best way of going about this.</p>
  
Tags: <proteins><python><phylogenetics>
Title: Download proteomes from NCBI based only on binomial names
ThreadText: # Download proteomes from NCBI based only on binomial names
  
  (Asked by: username_0 on 2018-06-18)
  
  I'm trying to download around 2000 proteomes from NCBI, but all I have is the genus and species e.g: `lophodermium_seditiosum trichoderma_harzianum`
  
  Does anyone know a good way of doing this in script form?  
  My code at the moment is:
  
  ```
  from Bio import Entrez
  from Bio import SeqIO
  import os
  
  file_name = "proteome1.faa"
  net_handle = Entrez.efetch(db="protein", id="trichoderma_harzianum", rettype="gp", retmode="text")
  out_handle = open(file_name, "w")
  out_handle.write(net_handle.read())
  out_handle.close()
  net_handle.close()
  print("Saved")
  
  for seq_record in SeqIO.parse("proteome1.faa", "genbank"):
      print(seq_record.id)
      print(repr(seq_record.seq))
      print(len(seq_record))
  
  ```
  
  But I think it wants a number as the id, I don't know if it's possible to convert species name to a numerical ID within the script, or if that's the best way of going about this.
  
  **Tags:** proteins, python, phylogenetics
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2018-06-19. Score: 3 | Accepted answer
  
  The general procedure is explained [here](https://www.ncbi.nlm.nih.gov/guide/howto/retrieve-seq-org/):
  
  First step is to search the [Taxonomy database](https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi) with the organism name. Accepted common names usually work at all taxonomic levels. Use the scientific name or formal name if no results are obtained with the common name.
  
  In code:
  
  ```
  from Bio import Entrez
  
  Entrez.email = 'user@example.org' # Put your email here
  
  def get_tax_id(species):
      species = species.replace('_', '+').strip()
      search = Entrez.esearch(term=species, db='taxonomy', retmode='xml')
      record = Entrez.read(search)
      return record['IdList'][0]
  
  if __name__ == '__main__':
      organisms = ['lophodermium_seditiosum', 'trichoderma_harzianum']
      taxids = [get_tax_id(organism) for organism in organisms]
      print(taxids)
  
  ```
  
  This gives us the `taxid`s for our species, respectively `128023` and `5544`. Now we can use these to get the proteomes. For *Lophodermium seditiosum* we find two results, but [for *Trichoderma harzianum* there are 65900 results](https://www.ncbi.nlm.nih.gov/protein/?term=txid5544%5BOrganism:exp%5D), so you might want to tweak the query to filter the results. For example, if you only want the sequences appearing in PDB you add `AND pdb[filter]` so that the query becomes `txid5544[Organism:exp] AND pdb[filter]`.
  
  ```
  def get_proteomes(taxid):
      query = 'txid{}[Organism:exp]'.format(taxid)
      handle = Entrez.esearch(db='protein', term=query)
      result = Entrez.read(handle)
      if int(result['Count']) <= 2000:
          ids = ','.join(result['IdList'])
          proteomes = Entrez.efetch(db='protein', id=ids, rettype='fasta').read()
          return proteomes
      else:
          raise RuntimeError('Too many results!')
  
  if __name__ == '__main__':
      print(get_proteomes('128023'))
  
  ```
  
  Returns:
  
  ```
  >ADI44294.1 actin, partial [Lophodermium seditiosum]
  CKAGFAGDDAPRAVFPSIVGRPRHHGIMIGMGQKDS
  
  >ADI44293.1 actin, partial [Lophodermium seditiosum]
  CKAGFAGDDAPRAVFPSIVGRPRHHGIMIGMGQKDS
  
  ```
  
  > **Comments:**
  > **username_0**: So I'm getting a 'Supplied ID parameter is empty' error from this code, the functions are defined at the beginning and are the same as in the answer  `listoforganisms = [x.split('\t')[0] for x in open("OGTlist.csv").readlines()]  IDlist = []   #convert to id if __name__ == '__main__':     organisms = listoforganisms     for organism in organisms:         taxid = get_tax_id(organism)         IDlist.append(taxid)         print (taxid)         print(get_proteomes((taxid))) `  
  > **username_1**: Can you post a small sample of your `OGTlist.csv`file?  
  > **username_0**: Got it working with, `if __name__ == '__main__':     organisms = listoforganisms     for organism in organisms:         taxid = get_tax_id(organism)         print(organism)         print (taxid)         stringid = str(taxid)          print(get_proteomes(stringid.strip("'[]'")))`  Just needed to strip the list elements to the right form  
  
  ---
  
  ### Answer by username_2 on 2021-10-27. Score: 1
  
  You can use the [NCBI Datasets](https://www.ncbi.nlm.nih.gov/datasets) to download the protein sequences of all annotated protein-coding genes for an organism starting with the organism name. There is a web-interface that allows exploring by taxonomy or specific organism search as well as a command line utility and an API. For example, the command line utility can be used as follows:
  
  ```bash
  $ datasets download genome taxon 'trichoderma harzianum' \
      --annotated --exclude-genomic-cds --exclude-gff3 \
      --exclude-rna --exclude-seq --reference
  Collecting 1 genome accessions [================================================] 100% 1/1
  Downloading: ncbi_dataset.zip    3.92MB done
  
  ```
  
  This downloads a zip archive containing the `protein.faa` file for the *Trichoderma harzianum* reference assembly.
  
  Note, unlike the Entrez method described above, this will not download all proteins for a given organism from the [NCBI Protein](https://www.ncbi.nlm.nih.gov/protein) portal. Thus, *Lophodermium seditiosum* does not return any results as there is no available genome for this assembly in the [NCBI Assembly](https://www.ncbi.nlm.nih.gov/assembly) database.
  
  On the other hand, the NCBI Datasets method will be significantly faster and return cleaner results. I would argue that if you are interested in obtaining the 'proteomes' of a given set of organisms, this is a better choice as it includes only those cases where a complete proteome is available based on the annotation of the genome and does not include cases where only a few proteins from an organism are available.
  
  
  ---
  
  