title: unique clones finding using SeqIO module of biopython
body: <p>I am working on Next Generation Sequencing (NGS) analysis of DNA. I am using SeqIO Biopython module to parse the DNA libraries in Fasta format. I want to filter the unique clones (unique records) only. I am using the following python code for this purpose.</p>
  <pre><code>seen=[]
  unique_clones=[]
  records=list(SeqIO.parse('DNA_library', 'fasta'))
  for record in records:
    if str(record.seq) not in seen:
      seen.append(str(record.seq))
      unique_clones.append(record)
  SeqIO.write(unique_clones, 'unique_clones.fasta', 'fasta')
  </code></pre>
  <p>one DNA library has around 1 million records, and I have more than 100 libraries to be analyzed. 1 Library is taking more than 2 Hours to be filtered by this code. This code seems very slow to extract unique clones. Is there any other way to filter unique records??</p>
  <hr />
  <p><strong>for python coders who don't have any experience in working with bioinformatics</strong></p>
  <p>clones in fasta format has two parameters, ID (&gt;id-number) and record(ATCG) and looks like below:</p>
  <pre><code>&gt;id-No-1
  ATCGGGCTAAATTCGACTGCAGT
  
  &gt;id-No-2
  ATCGGGCTAAATTCGACTGCAGT
  </code></pre>
  <p>I just want to filter unique clones based on their record and want to print unique clones (id and record).</p>
  <p>Please let me know if I am not enough clear in my question or explanation.</p>