Answers.Body: <p>Did a little more searching and found the answer in the README on BLAST's ftp site: <a href="ftp://ftp.ncbi.nlm.nih.gov/blast/db/README" rel="nofollow noreferrer">ftp://ftp.ncbi.nlm.nih.gov/blast/db/README</a></p>
  <pre><code>6. Non-redundant defline syntax
  
  The non-redundant databases are nr, nt and pataa. Identical sequences are 
  merged into one entry in these databases. To be merged two sequences must
  have identical lengths and every residue at every position must be the 
  same.  The FASTA deflines for the different entries that belong to one 
  record are separated by control-A characters invisible to most 
  programs. In the example below both entries Q57293.1 and AAB05030.1
  have the same sequence, in every respect:
  
  &gt;Q57293.1 RecName: Full=Fe(3+) ions import ATP-binding protein FbpC ^AAAB05030.1 afuC 
  [Actinobacillus pleuropneumoniae] ^AAAB17216.1 afuC [Actinobacillus pleuropneumoniae]
  MNNDFLVLKNITKSFGKATVIDNLDLVIKRGTMVTLLGPSGCGKTTVLRLVAGLENPTSGQIFIDGEDVTKSSIQNRDIC
  IVFQSYALFPHMSIGDNVGYGLRMQGVSNEERKQRVKEALELVDLAGFADRFVDQISGGQQQRVALARALVLKPKVLILD
  EPLSNLDANLRRSMREKIRELQQRLGITSLYVTHDQTEAFAVSDEVIVMNKGTIMQKARQKIFIYDRILYSLRNFMGEST
  ICDGNLNQGTVSIGDYRFPLHNAADFSVADGACLVGVRPEAIRLTATGETSQRCQIKSAVYMGNHWEIVANWNGKDVLIN
  ANPDQFDPDATKAFIHFTEQGIFLLNKE
  
  Individual sequences are now identifed simply by their accession.version.  
  
  For databases whose entries are not from official NCBI sequence databases, 
  such as Trace database, the gnl| convention is used. For custom databases, 
  this convention should be followed and the id for each sequence must be 
  unique, if one would like to take the advantage of indexed database, 
  which enables specific sequence retrieval using blastdbcmd program included 
  in the blast executable package.  One should refer to documents 
  distributed in the standalone BLAST package for more details.
  </code></pre>
  <p>Landed on that README from this question on biostars.org: <a href="https://www.biostars.org/p/217456/" rel="nofollow noreferrer">https://www.biostars.org/p/217456/</a></p>
  <p><strong>Edit</strong></p>
  <p>In that same README file is some information on the origin of the sequences in the non-redundant sets:</p>
  <pre><code>+-----------------------+-----------------------------------------------------+
  |File Name              | Content Description                                 |
  +-----------------------+-----------------------------------------------------+
  nr.gz*                  | non-redundant protein sequence database with entries
                             from GenPept, Swissprot, PIR, PDF, PDB, and RefSeq
  nt.gz*                  | nucleotide sequence database, with entries from all
                            traditional divisions of GenBank, EMBL, and DDBJ;
                            excluding bulk divisions (gss, sts, pat, est, htg)
                            and wgs entries. Partially non-redundant.
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-30T14:31:07.220Z
Answers.Text: Great job @juniper! It would be useful for the community if you could add to your answer that `nr` is a mix of different databases (not just RefSeq) and then accept your own answer. Not so humble... but that would clarify the outcome and save time for everyone I think.
Answers.UserDisplayName: gui11aume
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-30T19:56:03.597Z
Answers.Text: Sure thing. It is done.
Answers.UserDisplayName: juniper-
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-07T16:35:23.487Z
Answers.LastActivityDate: 2020-12-30T19:55:29.377Z
Answers.LastEditDate: 2020-12-30T19:55:29.377Z
Answers.UserDisplayName: juniper-
Answers.Body: <p>The Refseq team and also the NCBI resource coordinators team publish a new paper every few years, so check out the many papers (e.g. <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5753331/" rel="nofollow noreferrer">here</a> or <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4383943/" rel="nofollow noreferrer">here</a>), but to answer your 2nd question, non-redundancy here is (I think) defined very strictly as proteins that are identical in terms of sequence and length, so the clustering is trivial, without the need for a sophisticated clustering algorithm as required to detect more remote homologs.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-06T15:51:40.813Z
Answers.Text: This is certainly helpful reading, but RefSeq non-redundant protein sequences (a recent development, if I'm not mistaken) are not the same thing as the nr database (which has been around a while).
Answers.UserDisplayName: Daniel Standage
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-06T16:30:30.530Z
Answers.Text: @DanielStandage agreed, "nr" is used ambiguously to describe non-redundant proteins (nr) or nucleotides (nt) or these Refseq records. But yes the BLAST nr databases are not just Refseq records, but also Genbank+PDB etc. i think they still use the strict identical non-redundant definition though?
Answers.UserDisplayName: Chris_Rands
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-06T08:39:33.353Z
Answers.LastActivityDate: 2019-05-06T16:35:55.953Z
Answers.LastEditDate: 2019-05-06T16:35:55.953Z
Answers.UserDisplayName: Chris_Rands
Body: <p>Is there a paper or web page describing the procedure for creating the nr database used by NCBI's BLAST implementation?</p>
  
  <p>I presume it's some type of clustering, but I'm curious about how exactly sequences are condensed into non-redundant representatives.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-06T07:22:05.443Z
Comments.Text: Hi, what where your search terms that didn't work? (It would help to prevent other's people time). What do you expect them to do differently than what you do?
Comments.UserDisplayName: llrs
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-06T18:55:21.310Z
Comments.Text: Did you perhaps mean to leave this comment for a different question? I'm not sure what search terms you are referring to.
Comments.UserDisplayName: juniper-
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-07T07:13:17.630Z
Comments.Text: No, I didn't confuse your question with another. I am sure you tried to find yourself the answer to your question. I was referring to the search terms you used to find the answer to the question. I hoped we could know what did you search and suggest other search terms or explain how did we arrive to the answer (to help you in your future searches).
Comments.UserDisplayName: llrs
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-07T16:32:24.230Z
Comments.Text: You are right, I did try to find the answer to my question myself. I searched for the terms in the question itself: NCBI BLAST nr, BLAST non-redundant, etc... But your comment got me to search some more and I ended up finding my answer. It's in an answer below :-)
Comments.UserDisplayName: juniper-
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-08T08:47:11.883Z
Comments.Text: So your answer appears to confirm my answer- good- i can delete mine if it's not helpful/needed?
Comments.UserDisplayName: Chris_Rands
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-09T04:13:18.913Z
Comments.Text: Please don't delete your answer! The links to the papers are helpful and it provides more context.
Comments.UserDisplayName: juniper-
Tags: <blast><makeblastdb>
Title: How is BLAST's nr database created?
ThreadText: # How is BLAST's nr database created?
  
  (Asked by: username_0 on 2019-05-06)
  
  Is there a paper or web page describing the procedure for creating the nr database used by NCBI's BLAST implementation?
  
  I presume it's some type of clustering, but I'm curious about how exactly sequences are condensed into non-redundant representatives.
  
  **Tags:** blast, makeblastdb
  
  > **Comments:**
  > **username_1**: Hi, what where your search terms that didn't work? (It would help to prevent other's people time). What do you expect them to do differently than what you do?  
  > **username_0**: Did you perhaps mean to leave this comment for a different question? I'm not sure what search terms you are referring to.  
  > **username_1**: No, I didn't confuse your question with another. I am sure you tried to find yourself the answer to your question. I was referring to the search terms you used to find the answer to the question. I hoped we could know what did you search and suggest other search terms or explain how did we arrive to the answer (to help you in your future searches).  
  > **username_0**: You are right, I did try to find the answer to my question myself. I searched for the terms in the question itself: NCBI BLAST nr, BLAST non-redundant, etc... But your comment got me to search some more and I ended up finding my answer. It's in an answer below :-)  
  > **username_2**: So your answer appears to confirm my answer- good- i can delete mine if it's not helpful/needed?  
  > **username_0**: Please don't delete your answer! The links to the papers are helpful and it provides more context.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_0 on 2019-05-07. Score: 2 | Accepted answer
  
  Did a little more searching and found the answer in the README on BLAST's ftp site: <ftp://ftp.ncbi.nlm.nih.gov/blast/db/README>
  
  ```
  6. Non-redundant defline syntax
  
  The non-redundant databases are nr, nt and pataa. Identical sequences are 
  merged into one entry in these databases. To be merged two sequences must
  have identical lengths and every residue at every position must be the 
  same.  The FASTA deflines for the different entries that belong to one 
  record are separated by control-A characters invisible to most 
  programs. In the example below both entries Q57293.1 and AAB05030.1
  have the same sequence, in every respect:
  
  >Q57293.1 RecName: Full=Fe(3+) ions import ATP-binding protein FbpC ^AAAB05030.1 afuC 
  [Actinobacillus pleuropneumoniae] ^AAAB17216.1 afuC [Actinobacillus pleuropneumoniae]
  MNNDFLVLKNITKSFGKATVIDNLDLVIKRGTMVTLLGPSGCGKTTVLRLVAGLENPTSGQIFIDGEDVTKSSIQNRDIC
  IVFQSYALFPHMSIGDNVGYGLRMQGVSNEERKQRVKEALELVDLAGFADRFVDQISGGQQQRVALARALVLKPKVLILD
  EPLSNLDANLRRSMREKIRELQQRLGITSLYVTHDQTEAFAVSDEVIVMNKGTIMQKARQKIFIYDRILYSLRNFMGEST
  ICDGNLNQGTVSIGDYRFPLHNAADFSVADGACLVGVRPEAIRLTATGETSQRCQIKSAVYMGNHWEIVANWNGKDVLIN
  ANPDQFDPDATKAFIHFTEQGIFLLNKE
  
  Individual sequences are now identifed simply by their accession.version.  
  
  For databases whose entries are not from official NCBI sequence databases, 
  such as Trace database, the gnl| convention is used. For custom databases, 
  this convention should be followed and the id for each sequence must be 
  unique, if one would like to take the advantage of indexed database, 
  which enables specific sequence retrieval using blastdbcmd program included 
  in the blast executable package.  One should refer to documents 
  distributed in the standalone BLAST package for more details.
  
  ```
  
  Landed on that README from this question on biostars.org: <https://www.biostars.org/p/217456/>
  
  **Edit**
  
  In that same README file is some information on the origin of the sequences in the non-redundant sets:
  
  ```
  +-----------------------+-----------------------------------------------------+
  |File Name              | Content Description                                 |
  +-----------------------+-----------------------------------------------------+
  nr.gz*                  | non-redundant protein sequence database with entries
                             from GenPept, Swissprot, PIR, PDF, PDB, and RefSeq
  nt.gz*                  | nucleotide sequence database, with entries from all
                            traditional divisions of GenBank, EMBL, and DDBJ;
                            excluding bulk divisions (gss, sts, pat, est, htg)
                            and wgs entries. Partially non-redundant.
  
  ```
  
  > **Comments:**
  > **username_3**: Great job @username_0! It would be useful for the community if you could add to your answer that `nr` is a mix of different databases (not just RefSeq) and then accept your own answer. Not so humble... but that would clarify the outcome and save time for everyone I think.  
  > **username_0**: Sure thing. It is done.  
  
  ---
  
  ### Answer by username_2 on 2019-05-06. Score: 4
  
  The Refseq team and also the NCBI resource coordinators team publish a new paper every few years, so check out the many papers (e.g. [here](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5753331/) or [here](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4383943/)), but to answer your 2nd question, non-redundancy here is (I think) defined very strictly as proteins that are identical in terms of sequence and length, so the clustering is trivial, without the need for a sophisticated clustering algorithm as required to detect more remote homologs.
  
  > **Comments:**
  > **username_4**: This is certainly helpful reading, but RefSeq non-redundant protein sequences (a recent development, if I'm not mistaken) are not the same thing as the nr database (which has been around a while).  
  > **username_2**: @username_4 agreed, "nr" is used ambiguously to describe non-redundant proteins (nr) or nucleotides (nt) or these Refseq records. But yes the BLAST nr databases are not just Refseq records, but also Genbank+PDB etc. i think they still use the strict identical non-redundant definition though?  
  
  ---
  
  