Answers.Body: <p>And here’s a solution using R (with the Bioconductor):</p>
  
  <pre class="lang-r prettyprint-override"><code>fa = ShortRead::readFasta(infile)
  ids = as.character(ShortRead::id(fa))
  fa@id = Biostrings::BStringSet(make.unique(ids, sep = '_'))
  ShortRead::writeFasta(fa, outfile)
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-04T13:10:58.163Z
Answers.LastActivityDate: 2017-07-04T17:14:57.440Z
Answers.LastEditDate: 2017-07-04T17:14:57.440Z
Answers.UserDisplayName: Konrad Rudolph
Answers.Body: <p>Sure, using biopython:</p>
  
  <pre><code>#!/usr/bin/env python
  from Bio import SeqIO
  
  records = set()
  of = open("output.fa", "w")
  for record in SeqIO.parse("foo.fa", "fasta"):
      ID = record.id
      num = 1
      while ID in records:
          ID = "{}_{}".format(record.id, num)
          num += 1
      records.add(ID)
      record.id = ID
      record.name = ID
      record.description = ID
      SeqIO.write(record, of, "fasta")
  of.close()
  </code></pre>
  
  <p>Change <code>output.fa</code> and <code>foo.fa</code>. One doesn't need to explicitly change the <code>.name</code> and <code>.description</code>, but that's handy to prevent the original ID from not appearing still (after a space).</p>
  
  <h1>Explanation</h1>
  
  <ul>
  <li><code>records = set()</code> This will create a lookup table of all IDs <strong>written</strong></li>
  <li><code>for record in SeqIO.parse("foo.fa", "fasta"):</code> Open <code>foo.fa</code> as a fasta file and iterate over the <strong>entries</strong> in it. These are objects with an ID, name, description, and sequence attribute (the name and description are the same as the ID if not present).</li>
  <li><code>ID = record.id</code> <a href="https://en.wikipedia.org/wiki/Memoization" rel="nofollow noreferrer">Memoize</a> the entry ID (e.g., <code>1_duplicateName</code>)</li>
  <li><code>while ID in records:</code> As long as the ID has already been seen, keep looping.</li>
  <li><code>ID = "{}_{}".format(record.id, num)</code> Start adding increasing numbers after the ID, such as <code>1_duplicateName_1</code> and <code>1_duplicateName_2</code>. This will continue until the ID has not been seen.</li>
  <li><code>records.add(ID)</code> Add the unseen ID to the set.</li>
  <li><code>record.id = ID</code> Update the ID, the <code>.name</code> and <code>.description</code> are the same. If you don't do that, then you get output like <code>&gt;1_duplicateName_1 1_duplicateName</code>. That's not really a problem, but it's excessive.</li>
  <li><code>SeqIO.write(record, of, "fasta")</code> Write the record to the output file in fasta format.</li>
  </ul>
  
  <p>A benefit of biopython is the easy ability to change formats, so instead of fasta one could have substituted Genbank or another format if needed.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-04T12:02:17.610Z
Answers.LastActivityDate: 2017-07-05T15:49:58.337Z
Answers.LastEditDate: 2017-07-05T15:49:58.337Z
Answers.UserDisplayName: Devon Ryan
Answers.Body: <p>Sure, this little Perl snippet should do it:</p>
  
  <pre><code>$ perl -pe 's/$/_$seen{$_}/ if ++$seen{$_}&gt;1 and /^&gt;/; ' file.fa 
  &gt;1_uniqueGeneName
  atgc
  &gt;1_anotherUniqueGeneName
  atgc_2
  &gt;1_duplicateName
  atgc_3
  &gt;1_duplicateName_2
  atgc_4
  </code></pre>
  
  <p>Or, to make the changes in the original file, use <code>-i</code>:    </p>
  
  <pre><code>perl -i.bak -pe 's/$/_$seen{$_}/ if ++$seen{$_}&gt;1 and /^&gt;/; ' file.fa 
  </code></pre>
  
  <p>Note that the first occurrence of a duplicate name isn't changed, the second will become <code>_2</code>, the third <code>_3</code> etc. </p>
  
  <h3>Explanation</h3>
  
  <ul>
  <li><code>perl -pe</code> : <strong>p</strong>rint each input line after applying the script given by <code>-e</code> to it. </li>
  <li><code>++$seen{$_}&gt;1</code> : increment the current value stored in the hash <code>%seen</code> for this line (<code>$_</code>) by 1 and compare it to <code>1</code>. </li>
  <li><code>s/$/_$seen{$_}/ if ++$seen{$_}&gt;1 and /^&gt;/</code> : if the current line starts with a <code>&gt;</code> and the  value stored in the hash <code>%seen</code> for this line is greater than 1 (if this isn't the first time we see this line), replace the end of the line (<code>$</code>) with a <code>_</code> and the current value in the hash</li>
  </ul>
  
  <hr>
  
  <p>Alternatively, here's the same idea in <code>awk</code>:</p>
  
  <pre><code>$ awk '(/^&gt;/ &amp;&amp; s[$0]++){$0=$0"_"s[$0]}1;' file.fa 
  &gt;1_uniqueGeneName
  atgc
  &gt;1_anotherUniqueGeneName
  atgc
  &gt;1_duplicateName
  atgc
  &gt;1_duplicateName_2
  atgc
  </code></pre>
  
  <p>To make the changes in the original file (assuming you are using GNU <code>awk</code> which is the default on most Linux versions), use <code>-i inplace</code>:</p>
  
  <pre><code>awk -iinplace '(/^&gt;/ &amp;&amp; s[$0]++){$0=$0"_"s[$0]}1;' file.fa 
  </code></pre>
  
  <h3>Explanation</h3>
  
  <p>In awk, the special variable <code>$0</code> is the current line. </p>
  
  <ul>
  <li><code>(/^&gt;/ &amp;&amp; s[$0]++)</code> : if this line starts with a <code>&gt;</code> and incrementing the value stored in the array <code>s</code> for this line by 1 evaluates to true (is greater than 0).</li>
  <li><code>$0=$0"_"s[$0]</code> :  make the current line be itself with a <code>_</code> and the value from <code>s</code> appended.</li>
  <li><code>1;</code> : this is just shorthand for "print this line". If an expression evaluates to true, awk will print the current line. Since <code>1</code> is always true, this will print every line. </li>
  </ul>
  
  <hr>
  
  <p>If you want all of the duplicates to be marked, you need to read the file twice. Once to collect the names and a second to mark them:</p>
  
  <pre><code>$ awk '{
      if (NR==FNR){
          if(/^&gt;/){
              s[$0]++
          }
          next;
      }
      if(/^&gt;/){
          k[$0]++;
          if(s[$0]&gt;1){
              $0=$0"_"k[$0]
          }
      }
      print
  }' file.fa file.fa
  &gt;1_uniqueGeneName
  atgc
  &gt;1_anotherUniqueGeneName
  atgc
  &gt;1_duplicateName_1
  atgc
  &gt;1_duplicateName_2
  atgc
  </code></pre>
  
  <hr>
  
  <p><strong>IMPORTANT</strong>: note that all of these approaches assume you don't already have sequence names ending with <code>_N</code> where <code>N</code> is a number. If your input file has 2 sequences called <code>foo</code> and one called <code>foo_2</code>, then you will end up with two <code>foo_2</code>:</p>
  
  <pre><code>$ cat test.fa
  &gt;foo_2
  actg
  &gt;foo
  actg
  &gt;foo
  actg
  $ perl -pe 's/$/_$seen{$_}/ if ++$seen{$_}&gt;1 and /^&gt;/; ' test.fa 
  &gt;foo_2
  actg
  &gt;foo
  actg
  &gt;foo_2
  actg
  </code></pre>
  
  <p>If this can be an issue for you, use one of the more sophisticated approaches suggested by the other answers.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-04T13:37:43.387Z
Answers.Text: In the `awk` example I worry that duplicate IDs could be created if you have contigs like `>foo` and `>foo_1` in the original file. `foo_1` wouldn't be in `s`, but might already have been printed out if `foo` had already been encountered.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-04T13:56:06.430Z
Answers.Text: @DevonRyan ah, yes that's a very valid point. Not just the awk one either, all approaches here break if a duplicate sequence already exists with a `_N`. In that case, the extra verbiage added by python is actually worth it :P. I added a disclaimer, thanks.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-04T11:58:43.597Z
Answers.LastActivityDate: 2017-07-04T13:59:59.517Z
Answers.LastEditDate: 2017-07-04T13:59:59.517Z
Answers.UserDisplayName: terdon
Body: <p>I have a reference database with contains 100s of sequences in fasta format. Some of these sequences have duplicate names like so:</p>
  
  <pre><code>&gt;1_uniqueGeneName
  atgc
  &gt;1_anotherUniqueGeneName
  atgc
  &gt;1_duplicateName
  atgc
  &gt;1_duplicateName
  atgc
  </code></pre>
  
  <p>Is is possible to run through a large file like this and change the names of only the duplicates?</p>
  
  <pre><code>&gt;1_uniqueGeneName
  atgc
  &gt;1_anotherUniqueGeneName
  atgc
  &gt;1_duplicateName_1
  atgc
  &gt;1_duplicateName_2
  atgc
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-04T11:57:44.083Z
Comments.Text: Do you want to only change the names of the duplicates, I mean of the second, third etc seqs with the same name or also of the 1st as you have done? I mean, do you really need `duplicate_1`, `duplicate_2` or is `duplicate`, `duplicate_1` enough? The latter is far simpler since the former will require you to read the file twice.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-04T12:32:35.593Z
Comments.Text: Cool. Id prefer the `duplicate_1 duplicate_2`format but if it is much more complex then I am fine with the `duplicate duplicate_1`format.
Comments.UserDisplayName: AudileF
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-04T16:08:06.887Z
Comments.Text: @AudileF: I'm having a hard time coming up with a good reason why I would want the first duplicate to be changed as well.  Can you give us a bit more insight into the reason for this request?
Comments.UserDisplayName: winni2k
Tags: <fasta><text>
Title: How to append numbers only on duplicates sequence names?
ThreadText: # How to append numbers only on duplicates sequence names?
  
  (Asked by: username_0 on 2017-07-04)
  
  I have a reference database with contains 100s of sequences in fasta format. Some of these sequences have duplicate names like so:
  
  ```
  >1_uniqueGeneName
  atgc
  >1_anotherUniqueGeneName
  atgc
  >1_duplicateName
  atgc
  >1_duplicateName
  atgc
  
  ```
  
  Is is possible to run through a large file like this and change the names of only the duplicates?
  
  ```
  >1_uniqueGeneName
  atgc
  >1_anotherUniqueGeneName
  atgc
  >1_duplicateName_1
  atgc
  >1_duplicateName_2
  atgc
  
  ```
  
  **Tags:** fasta, text
  
  > **Comments:**
  > **username_1**: Do you want to only change the names of the duplicates, I mean of the second, third etc seqs with the same name or also of the 1st as you have done? I mean, do you really need `duplicate_1`, `duplicate_2` or is `duplicate`, `duplicate_1` enough? The latter is far simpler since the former will require you to read the file twice.  
  > **username_0**: Cool. Id prefer the `duplicate_1 duplicate_2`format but if it is much more complex then I am fine with the `duplicate duplicate_1`format.  
  > **username_2**: @username_0: I'm having a hard time coming up with a good reason why I would want the first duplicate to be changed as well.  Can you give us a bit more insight into the reason for this request?  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_1 on 2017-07-04. Score: 5 | Accepted answer
  
  Sure, this little Perl snippet should do it:
  
  ```
  $ perl -pe 's/$/_$seen{$_}/ if ++$seen{$_}>1 and /^>/; ' file.fa 
  >1_uniqueGeneName
  atgc
  >1_anotherUniqueGeneName
  atgc_2
  >1_duplicateName
  atgc_3
  >1_duplicateName_2
  atgc_4
  
  ```
  
  Or, to make the changes in the original file, use `-i`:
  
  ```
  perl -i.bak -pe 's/$/_$seen{$_}/ if ++$seen{$_}>1 and /^>/; ' file.fa 
  
  ```
  
  Note that the first occurrence of a duplicate name isn't changed, the second will become `_2`, the third `_3` etc.
  
  ### Explanation
  
  * `perl -pe` : **p**rint each input line after applying the script given by `-e` to it.
  * `++$seen{$_}>1` : increment the current value stored in the hash `%seen` for this line (`$_`) by 1 and compare it to `1`.
  * `s/$/_$seen{$_}/ if ++$seen{$_}>1 and /^>/` : if the current line starts with a `>` and the value stored in the hash `%seen` for this line is greater than 1 (if this isn't the first time we see this line), replace the end of the line (`$`) with a `_` and the current value in the hash
  
  ---
  
  Alternatively, here's the same idea in `awk`:
  
  ```
  $ awk '(/^>/ && s[$0]++){$0=$0"_"s[$0]}1;' file.fa 
  >1_uniqueGeneName
  atgc
  >1_anotherUniqueGeneName
  atgc
  >1_duplicateName
  atgc
  >1_duplicateName_2
  atgc
  
  ```
  
  To make the changes in the original file (assuming you are using GNU `awk` which is the default on most Linux versions), use `-i inplace`:
  
  ```
  awk -iinplace '(/^>/ && s[$0]++){$0=$0"_"s[$0]}1;' file.fa 
  
  ```
  
  ### Explanation
  
  In awk, the special variable `$0` is the current line.
  
  * `(/^>/ && s[$0]++)` : if this line starts with a `>` and incrementing the value stored in the array `s` for this line by 1 evaluates to true (is greater than 0).
  * `$0=$0"_"s[$0]` : make the current line be itself with a `_` and the value from `s` appended.
  * `1;` : this is just shorthand for "print this line". If an expression evaluates to true, awk will print the current line. Since `1` is always true, this will print every line.
  
  ---
  
  If you want all of the duplicates to be marked, you need to read the file twice. Once to collect the names and a second to mark them:
  
  ```
  $ awk '{
      if (NR==FNR){
          if(/^>/){
              s[$0]++
          }
          next;
      }
      if(/^>/){
          k[$0]++;
          if(s[$0]>1){
              $0=$0"_"k[$0]
          }
      }
      print
  }' file.fa file.fa
  >1_uniqueGeneName
  atgc
  >1_anotherUniqueGeneName
  atgc
  >1_duplicateName_1
  atgc
  >1_duplicateName_2
  atgc
  
  ```
  
  ---
  
  **IMPORTANT**: note that all of these approaches assume you don't already have sequence names ending with `_N` where `N` is a number. If your input file has 2 sequences called `foo` and one called `foo_2`, then you will end up with two `foo_2`:
  
  ```
  $ cat test.fa
  >foo_2
  actg
  >foo
  actg
  >foo
  actg
  $ perl -pe 's/$/_$seen{$_}/ if ++$seen{$_}>1 and /^>/; ' test.fa 
  >foo_2
  actg
  >foo
  actg
  >foo_2
  actg
  
  ```
  
  If this can be an issue for you, use one of the more sophisticated approaches suggested by the other answers.
  
  > **Comments:**
  > **username_4**: In the `awk` example I worry that duplicate IDs could be created if you have contigs like `>foo` and `>foo_1` in the original file. `foo_1` wouldn't be in `s`, but might already have been printed out if `foo` had already been encountered.  
  > **username_1**: @username_4 ah, yes that's a very valid point. Not just the awk one either, all approaches here break if a duplicate sequence already exists with a `_N`. In that case, the extra verbiage added by python is actually worth it :P. I added a disclaimer, thanks.  
  
  ---
  
  ### Answer by username_3 on 2017-07-04. Score: 7
  
  And here’s a solution using R (with the Bioconductor):
  
  ```r
  fa = ShortRead::readFasta(infile)
  ids = as.character(ShortRead::id(fa))
  fa@id = Biostrings::BStringSet(make.unique(ids, sep = '_'))
  ShortRead::writeFasta(fa, outfile)
  
  ```
  
  
  ---
  
  ### Answer by username_4 on 2017-07-04. Score: 6
  
  Sure, using biopython:
  
  ```
  #!/usr/bin/env python
  from Bio import SeqIO
  
  records = set()
  of = open("output.fa", "w")
  for record in SeqIO.parse("foo.fa", "fasta"):
      ID = record.id
      num = 1
      while ID in records:
          ID = "{}_{}".format(record.id, num)
          num += 1
      records.add(ID)
      record.id = ID
      record.name = ID
      record.description = ID
      SeqIO.write(record, of, "fasta")
  of.close()
  
  ```
  
  Change `output.fa` and `foo.fa`. One doesn't need to explicitly change the `.name` and `.description`, but that's handy to prevent the original ID from not appearing still (after a space).
  
  # Explanation
  
  * `records = set()` This will create a lookup table of all IDs **written**
  * `for record in SeqIO.parse("foo.fa", "fasta"):` Open `foo.fa` as a fasta file and iterate over the **entries** in it. These are objects with an ID, name, description, and sequence attribute (the name and description are the same as the ID if not present).
  * `ID = record.id` [Memoize](https://en.wikipedia.org/wiki/Memoization) the entry ID (e.g., `1_duplicateName`)
  * `while ID in records:` As long as the ID has already been seen, keep looping.
  * `ID = "{}_{}".format(record.id, num)` Start adding increasing numbers after the ID, such as `1_duplicateName_1` and `1_duplicateName_2`. This will continue until the ID has not been seen.
  * `records.add(ID)` Add the unseen ID to the set.
  * `record.id = ID` Update the ID, the `.name` and `.description` are the same. If you don't do that, then you get output like `>1_duplicateName_1 1_duplicateName`. That's not really a problem, but it's excessive.
  * `SeqIO.write(record, of, "fasta")` Write the record to the output file in fasta format.
  
  A benefit of biopython is the easy ability to change formats, so instead of fasta one could have substituted Genbank or another format if needed.
  
  
  ---
  
  