Answers.Body: <p>Yes, Biopython deliberately does not allow duplicate keys with <code>.to_dict()</code>, e.g.</p>
  <p>Input file:</p>
  <pre><code>$ cat test.fa 
  &gt;1
  ATG
  &gt;2
  AAA
  &gt;1
  CG
  </code></pre>
  <p>Code:</p>
  <pre><code>from Bio import SeqIO
  
  print(SeqIO.to_dict(SeqIO.parse(&quot;test.fa&quot;, &quot;fasta&quot;)))
  </code></pre>
  <p>Traceback:</p>
  <pre><code>  File &quot;script.py&quot;, line 3, in &lt;module&gt;
      print(SeqIO.to_dict(SeqIO.parse(&quot;test.fa&quot;, &quot;fasta&quot;)))
    File &quot;/path/lib/python3.7/site-packages/Bio/SeqIO/__init__.py&quot;, line 789, in to_dict
      raise ValueError(&quot;Duplicate key '%s'&quot; % key)
  ValueError: Duplicate key '1'
  </code></pre>
  <p>However, you can easily write your own function, e.g.:</p>
  <pre><code>from Bio import SeqIO
  
  def to_dict_remove_dups(sequences):
      return {record.id: record for record in sequences}
  
  print(to_dict_remove_dups(SeqIO.parse(&quot;test.fa&quot;, &quot;fasta&quot;)))
  </code></pre>
  <p>Output:</p>
  <pre><code>{'1': SeqRecord(seq=Seq('CG', SingleLetterAlphabet()), id='1', name='1', description='1', dbxrefs=[]), '2': SeqRecord(seq=Seq('AAA', SingleLetterAlphabet()), id='2', name='2', description='2', dbxrefs=[])}
  </code></pre>
  <p>Alternatively, you Biopython's <code>.to_dict()</code> accepts a <code>key_function</code> argument, e.g. if you wish to make all the headers unique</p>
  <pre><code>from Bio import SeqIO
  from itertools import count
  
  c = count()
  
  print(SeqIO.to_dict(SeqIO.parse(&quot;test.fa&quot;, &quot;fasta&quot;),
                      key_function = lambda x: f&quot;{x.id}_seqNum{next(c)}&quot;))
  </code></pre>
  <p>Output:</p>
  <pre><code>{'1_seqNum0': SeqRecord(seq=Seq('ATG', SingleLetterAlphabet()), id='1', name='1', description='1', dbxrefs=[]), '2_seqNum1': SeqRecord(seq=Seq('AAA', SingleLetterAlphabet()), id='2', name='2', description='2', dbxrefs=[]), '1_seqNum2': SeqRecord(seq=Seq('CG', SingleLetterAlphabet()), id='1', name='1', description='1', dbxrefs=[])}
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-31T08:50:22.717Z
Answers.Text: thank you; return {record.id: record for record in sequences} so the expression garners us only unique IDs it seems, is this the case?
Answers.UserDisplayName: Mike 
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-31T09:27:23.497Z
Answers.Text: Well, obviously it does, but I guess my question is, what is determining the order (in the first solution)?
Answers.UserDisplayName: Mike 
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-02T17:05:51.527Z
Answers.Text: @Mike The last key,value pair is nearly* always retained *https://stackoverflow.com/questions/39678672/is-a-python-dict-comprehension-always-last-wins-if-there-are-duplicate-keys
Answers.UserDisplayName: Chris_Rands
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-03T07:37:52.523Z
Answers.Text: Hi Chris, got it thank you!
Answers.UserDisplayName: Mike 
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-08-30T20:57:07.067Z
Answers.LastActivityDate: 2020-08-30T20:57:07.067Z
Answers.UserDisplayName: Chris_Rands
Body: <p>I got the following error:</p>
  <blockquote>
  <p>attempting to use to_dict to index a multiple sequence FASTA file with duplicate keys.</p>
  </blockquote>
  <p>Are there methods for working around duplicate keys in biopython? I was thinking to simply remove the duplicate entries, but I would need to delete the entire entry associated with the keys.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-08-30T12:41:25.450Z
Comments.Text: Hi @Mike are you looking to rename the duplicate keys?
Comments.UserDisplayName: M__
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-08-30T12:52:56.030Z
Comments.Text: yes, I was thinking of appending the keys with digits incrementally or something? The keys are uniprot IDs, but the renamed duplicates wouldn't do anything harmful to my output. Might just be best to remove them using a set method or something. Still quite new to this.
Comments.UserDisplayName: Mike 
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-08-30T17:19:29.957Z
Comments.Text: Removing keys via a dictionary is easy, it should be automatic (they will overwrite). Pandas will enable duplicate keys are preserved by shifting the index
Comments.UserDisplayName: M__
Tags: <python>
Title: Dealing with duplicate keys
ThreadText: # Dealing with duplicate keys
  
  (Asked by: username_0 on 2020-08-30)
  
  I got the following error:
  
  > attempting to use to\_dict to index a multiple sequence FASTA file with duplicate keys.
  
  Are there methods for working around duplicate keys in biopython? I was thinking to simply remove the duplicate entries, but I would need to delete the entire entry associated with the keys.
  
  **Tags:** python
  
  > **Comments:**
  > **username_1**: Hi @username_0are you looking to rename the duplicate keys?  
  > **username_0**: yes, I was thinking of appending the keys with digits incrementally or something? The keys are uniprot IDs, but the renamed duplicates wouldn't do anything harmful to my output. Might just be best to remove them using a set method or something. Still quite new to this.  
  > **username_1**: Removing keys via a dictionary is easy, it should be automatic (they will overwrite). Pandas will enable duplicate keys are preserved by shifting the index  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-08-30. Score: 2
  
  Yes, Biopython deliberately does not allow duplicate keys with `.to_dict()`, e.g.
  
  Input file:
  
  ```
  $ cat test.fa 
  >1
  ATG
  >2
  AAA
  >1
  CG
  
  ```
  
  Code:
  
  ```
  from Bio import SeqIO
  
  print(SeqIO.to_dict(SeqIO.parse("test.fa", "fasta")))
  
  ```
  
  Traceback:
  
  ```
    File "script.py", line 3, in <module>
      print(SeqIO.to_dict(SeqIO.parse("test.fa", "fasta")))
    File "/path/lib/python3.7/site-packages/Bio/SeqIO/__init__.py", line 789, in to_dict
      raise ValueError("Duplicate key '%s'" % key)
  ValueError: Duplicate key '1'
  
  ```
  
  However, you can easily write your own function, e.g.:
  
  ```
  from Bio import SeqIO
  
  def to_dict_remove_dups(sequences):
      return {record.id: record for record in sequences}
  
  print(to_dict_remove_dups(SeqIO.parse("test.fa", "fasta")))
  
  ```
  
  Output:
  
  ```
  {'1': SeqRecord(seq=Seq('CG', SingleLetterAlphabet()), id='1', name='1', description='1', dbxrefs=[]), '2': SeqRecord(seq=Seq('AAA', SingleLetterAlphabet()), id='2', name='2', description='2', dbxrefs=[])}
  
  ```
  
  Alternatively, you Biopython's `.to_dict()` accepts a `key_function` argument, e.g. if you wish to make all the headers unique
  
  ```
  from Bio import SeqIO
  from itertools import count
  
  c = count()
  
  print(SeqIO.to_dict(SeqIO.parse("test.fa", "fasta"),
                      key_function = lambda x: f"{x.id}_seqNum{next(c)}"))
  
  ```
  
  Output:
  
  ```
  {'1_seqNum0': SeqRecord(seq=Seq('ATG', SingleLetterAlphabet()), id='1', name='1', description='1', dbxrefs=[]), '2_seqNum1': SeqRecord(seq=Seq('AAA', SingleLetterAlphabet()), id='2', name='2', description='2', dbxrefs=[]), '1_seqNum2': SeqRecord(seq=Seq('CG', SingleLetterAlphabet()), id='1', name='1', description='1', dbxrefs=[])}
  
  ```
  
  > **Comments:**
  > **username_0**: thank you; return {record.id: record for record in sequences} so the expression garners us only unique IDs it seems, is this the case?  
  > **username_0**: Well, obviously it does, but I guess my question is, what is determining the order (in the first solution)?  
  > **username_2**: @username_0The last key,value pair is nearly* always retained *https://stackoverflow.com/questions/39678672/is-a-python-dict-comprehension-always-last-wins-if-there-are-duplicate-keys  
  > **username_0**: Hi Chris, got it thank you!  
  
  ---
  
  