Answers.Body: <p>After modifying the code above in the original post, this one works the best for me. I got it working after putting 'not' in the second last line instead of &quot;if record.id in to keep&quot; to &quot;if record.id not in to keep&quot;:</p>
  <pre><code>from Bio import SeqIO
  with open('headers.txt', 'r') as fin:
      to_remove = set([])
      for line in fin:
          to_remove.add(line.strip().split(' ')[0])
  with open('output.fa', 'w') as fout:
      for record in SeqIO.parse('sample.fa','fasta'):
          if record.id in to_remove:
              pass
          else:
              SeqIO.write(record, fout, 'fasta')
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-21T05:41:14.237Z
Answers.LastActivityDate: 2023-12-21T09:15:31.863Z
Answers.LastEditDate: 2023-12-21T09:15:31.863Z
Answers.UserDisplayName: Irfan
Answers.Body: <p>You can use something like seqkit: <a href="https://bioinf.shenwei.me/seqkit/" rel="nofollow noreferrer">https://bioinf.shenwei.me/seqkit/</a></p>
  <p>seqkit has a <code>grep</code> sub-command that is one of the most flexible I've ever seen. Your use case is quite simple, you can use <code>seqkit grep -v</code> like so:</p>
  <pre><code>seqkit grep -v -n -f list_of_IDs_to_exclude.txt in.fasta &gt; out.fasta
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-20T19:23:48.367Z
Answers.LastActivityDate: 2023-12-21T09:16:21.220Z
Answers.LastEditDate: 2023-12-21T09:16:21.220Z
Answers.UserDisplayName: Ram RS
Answers.Body: <p>This should work</p>
  <pre><code>import argparse
  
  ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.ArgumentDefaultsHelpFormatter)
  ap.add_argument('--fasta', required=True, type=str, help='Fasta file to extract from')
  ap.add_argument('--headers', required=True, type=str, help='Headers to remove, one per line.')
  
  conf = ap.parse_args()
  f = conf.fasta
  h = conf.headers
  header_set = set()
  
  with open(h) as header_file:
      for line in header_file:
          line = line.rstrip()
          header_set.add(line)
  
  with open(f) as fasta_file:
      remove_sequence = False
      for line in fasta_file:
          if '&gt;' in line:
              header_id = line.split()[0].replace(&quot;&gt;&quot;, &quot;&quot;)
              if header_id in header_set:
                  remove_sequence = True
              else:
                  remove_sequence = False
                  print(line.rstrip())
          elif not remove_sequence:
              print(line.rstrip())
  
  exit()
  </code></pre>
  <p>Save it as python script and then run</p>
  <p><code>python remove_fasta.py --fasta sample.fa --headers listofheaders.txt &gt; cleaned.fasta</code></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-21T02:23:29.083Z
Answers.Text: Thanks. I tried this first thing in the morning. First it ran but didn't remove anything, then I removed '>' character from the headers list and it gave the correct expected cleaned file.
  What if I want to run it directly form the python IDLE and not from cmd?
Answers.UserDisplayName: Irfan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-21T10:29:09.060Z
Answers.Text: I'm glad it worked. As you can see, there are other options that will get the job done, but I understood that you want to learn how to do this in Python. I don't personally use the Python IDLE on Windows, but I believe you can save your code, open it in IDLE, and click on "run module". Make sure to modify the file locations as needed. I hope this helps, and good luck with your coding
Answers.UserDisplayName: CorteZero
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-20T10:05:33.050Z
Answers.LastActivityDate: 2023-12-20T10:12:09.957Z
Answers.LastEditDate: 2023-12-20T10:12:09.957Z
Answers.UserDisplayName: CorteZero
Answers.Body: <p>I would argue writing a whole python script for this is overkill. Assuming you are using a*nix based operating system like Linux or macOS, you can use the two scripts <a href="https://bioinformatics.stackexchange.com/a/2651/298">I have posted here</a>, <code>FastaToTbl</code> and <code>TblToFasta</code>, and just <code>grep</code> out the ones you do not want:</p>
  <pre><code>FastaToTbl sample.fa | grep -v -xFf listofheaders.txt | TblToFasta &gt; cleaned.fa
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-21T01:57:25.487Z
Answers.Text: Hi terdon, I am only using windows 10 PC with python. I am trying the above suggestions as well. Writing my own code is a big opportunity to learn as well.
Answers.UserDisplayName: Irfan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-21T09:43:36.170Z
Answers.Text: Oh. Windows? OK. But then please add that to your question. Windows isn't really used by bioinformaticians at all so nobody would guess you are trying this in windows. If you want to learn bioinformatics, I would urge you to use Linux or macOS or basically anything other than Windows.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-22T00:59:53.343Z
Answers.Text: I agree with you. I am already on to it. ")
Answers.UserDisplayName: Irfan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-20T13:53:21.690Z
Answers.LastActivityDate: 2023-12-20T13:53:21.690Z
Answers.UserDisplayName: terdon
Body: <p>a python beginner here.
  I have a fasta file with 2500+ sequences, and after doing some analysis I want to remove around 200+ sequences based on the matching IDs. Now, I have one fasta file (as sample.fa) and a text file with a list of IDs for the sequences that I want to remove from this fasta file. Below is an example of the fasta sequences. For simplicity, I have shortened the sequences and their respective IDs. The ID in the text file (for the sequences to remove, and in the fast file will be exactly same). The sequences were originally downloaded form NCBI and had very long headers so I have them shortened as at this point I don't need all that info, usually just the sequence ID in the headers is enough.</p>
  <p>Sample fasta sequences in fasta file:</p>
  <pre><code>&gt;WP_017417394.1-Bacillus_amyloliquefaciens_gp.
  MRGKKVWISLLFALALIFTMAFGSTSPAQAAGKSNGEKKYIVGFKQTMSTMSAAKKKDVISEKGGKVQKQFKYVDAASAT
  &gt;WP_268291751.1-Bacillus_inaquosorum
  MRRKKLWISLLFALTLIFTMAFSNMSAQAAGKSSTEKKYIVGFKETMSAMSSAKKKDVISEKGGKVQKQFKYVNAATATL
  &gt;WP_003327717.1-Bacillus_atrophaeus
  MRSRKLWIGLLFAFTLVFTMAFSSMSPAQAAGKSSEEKKYIVGFKQTMGAMSAAAKKDVIAENGGKLQKQFKYVDAATAT
  &gt;WP_206702175.1-Bacillus_vallismortis
  MAFSNMSAQAAGKSSTEKKYIVGFKQTMSVMSSAKKKDVISEKGGKVQKQFKYVNAAAATLDAKAVKELKQDPSVAYVEE
  &gt;KAF1680298.1-Bacillus_sp.
  MKSKKLWISLLFALTLVFTMAFSNMSAQASGKNSTEKKYIVGFKQTMSTAKKKDIISEKGGKVQKQFKYVNAASATLDQK
  &gt;WP_172293960.1-Bacillus_sp.
  MGKKRLWYSLLTVLLLVFSMAFSHPANAVKSSAKDVKKDYIVGFKSSLKTTSAKKDVIKENGGKVDKHFRSINAAKVTLD
  &gt;WP_082194748.1-Bacillus_xiamenensis
  MCVKKKNVMTSVLLAVPLLFSAGFGGSMANAETASKSESEKSYIVGFKASATTNSSKKQAVTQNGGTLEKQYRLINAAQV
  &gt;WP_048353666.1-Bacillus_glycinifermentans
  MKKRSLWLSVLTALLLVFTMAFSNPASAEQPAKDVEKDYIVGFKSSVKTASVKKDVIKESGGKVDKQFKIINAAKATLDQ
  &gt;WP_206702858.1-Bacillus_sonorensis
  MTALLLVLSTAFSSPASAAQPAKDVEKDYIVGFKSSVKTAAVKKDVIKENGGKVDKQFKIINAAKATLDQEEVKALKKD
  </code></pre>
  <p>The list of IDs in a separate text file that need to be removed from the above fasta file:</p>
  <pre><code>&gt;WP_017417394.1-Bacillus_amyloliquefaciens_gp.
  &gt;WP_003327717.1-Bacillus_atrophaeus
  &gt;KAF1680298.1-Bacillus_sp.
  &gt;WP_048353666.1-Bacillus_glycinifermentans
  &gt;WP_206702858.1-Bacillus_sonorensis
  </code></pre>
  <p>I searched and tried a few scripts shared elsewhere (e.g. from <a href="https://bioinformatics.stackexchange.com/questions/15326/how-to-remove-sequences-from-a-fasta-file-using-a-sequence-id-list-which-contain">here</a>), but I couldn't make it work with my files. So all the help will be really appreciated. Thanks in advance.</p>
  <p>PS. I am using using python usually with its own IDLE on windows 10 PC.</p>
  <p>I did one thing with <a href="https://bioinformatics.stackexchange.com/questions/15393/filtering-sequences-entries-by-headers-id-from-a-fasta-file-database">this code</a>. I removed the '&gt;' character from the headers file and then this code works but it keeps those headers and their respective sequences. I think it works as intended in this code, but I want to remove those from the sequence file not to keep.</p>
  <p>Update: as asked below, I copied this code from another post.</p>
  <pre><code>from Bio import SeqIO
  with open('headers.txt', 'r') as fin:
      to_keep = set([])
      for line in fin:
          to_keep.add(line.strip().split(' ')[0])
  with open('output.fa', 'w') as fout:
      for record in SeqIO.parse('sample.fa','fasta'):
          if record.id in to_keep:
              SeqIO.write(record, fout, 'fasta')
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-12-20T17:29:39.290Z
Comments.Text: And how did they not work? I was about to write an answer but it is essentially what  had already written as an answer to the other question. Please [edit] and clarify why those other solutions didn't work, since right now, this looks like a duplicate question.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-12-21T16:49:10.823Z
Comments.Text: Does this answer your question? [Filtering Sequences (entries) by headers ID from a Fasta file database](https://bioinformatics.stackexchange.com/questions/15393/filtering-sequences-entries-by-headers-id-from-a-fasta-file-database)
Comments.UserDisplayName: Ram RS
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-12-25T20:15:41.473Z
Comments.Text: This is not a good FASTA file format. IDs and description should be separate by space. I suggest to take a look at the official documentation. Good practice will make your life easier here an [example](https://www.ncbi.nlm.nih.gov/genbank/fastaformat/) [or](https://bioperl.org/formats/sequence_formats/FASTA_sequence_format)
Comments.UserDisplayName: Supertech
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-12-28T05:55:18.377Z
Comments.Text: yes, I agree. but I do that on purpose b/c after some manipulation, I make a phylogenetic tree and visualize (usually on ITOL) which messes up the formatting, sometimes it is very long and I am interested to see only the accession ID and the bacterial name. So I delete the rest and make it as one continuous string so everything I need stays in and still within a short limit to be visible.
Comments.UserDisplayName: Irfan
Tags: <python><fasta><sequence>
Title: Remove sequences from a fasta file with IDs from a text file using Python
ThreadText: # Remove sequences from a fasta file with IDs from a text file using Python
  
  (Asked by: username_0 on 2023-12-20)
  
  a python beginner here.
  I have a fasta file with 2500+ sequences, and after doing some analysis I want to remove around 200+ sequences based on the matching IDs. Now, I have one fasta file (as sample.fa) and a text file with a list of IDs for the sequences that I want to remove from this fasta file. Below is an example of the fasta sequences. For simplicity, I have shortened the sequences and their respective IDs. The ID in the text file (for the sequences to remove, and in the fast file will be exactly same). The sequences were originally downloaded form NCBI and had very long headers so I have them shortened as at this point I don't need all that info, usually just the sequence ID in the headers is enough.
  
  Sample fasta sequences in fasta file:
  
  ```
  >WP_017417394.1-Bacillus_amyloliquefaciens_gp.
  MRGKKVWISLLFALALIFTMAFGSTSPAQAAGKSNGEKKYIVGFKQTMSTMSAAKKKDVISEKGGKVQKQFKYVDAASAT
  >WP_268291751.1-Bacillus_inaquosorum
  MRRKKLWISLLFALTLIFTMAFSNMSAQAAGKSSTEKKYIVGFKETMSAMSSAKKKDVISEKGGKVQKQFKYVNAATATL
  >WP_003327717.1-Bacillus_atrophaeus
  MRSRKLWIGLLFAFTLVFTMAFSSMSPAQAAGKSSEEKKYIVGFKQTMGAMSAAAKKDVIAENGGKLQKQFKYVDAATAT
  >WP_206702175.1-Bacillus_vallismortis
  MAFSNMSAQAAGKSSTEKKYIVGFKQTMSVMSSAKKKDVISEKGGKVQKQFKYVNAAAATLDAKAVKELKQDPSVAYVEE
  >KAF1680298.1-Bacillus_sp.
  MKSKKLWISLLFALTLVFTMAFSNMSAQASGKNSTEKKYIVGFKQTMSTAKKKDIISEKGGKVQKQFKYVNAASATLDQK
  >WP_172293960.1-Bacillus_sp.
  MGKKRLWYSLLTVLLLVFSMAFSHPANAVKSSAKDVKKDYIVGFKSSLKTTSAKKDVIKENGGKVDKHFRSINAAKVTLD
  >WP_082194748.1-Bacillus_xiamenensis
  MCVKKKNVMTSVLLAVPLLFSAGFGGSMANAETASKSESEKSYIVGFKASATTNSSKKQAVTQNGGTLEKQYRLINAAQV
  >WP_048353666.1-Bacillus_glycinifermentans
  MKKRSLWLSVLTALLLVFTMAFSNPASAEQPAKDVEKDYIVGFKSSVKTASVKKDVIKESGGKVDKQFKIINAAKATLDQ
  >WP_206702858.1-Bacillus_sonorensis
  MTALLLVLSTAFSSPASAAQPAKDVEKDYIVGFKSSVKTAAVKKDVIKENGGKVDKQFKIINAAKATLDQEEVKALKKD
  
  ```
  
  The list of IDs in a separate text file that need to be removed from the above fasta file:
  
  ```
  >WP_017417394.1-Bacillus_amyloliquefaciens_gp.
  >WP_003327717.1-Bacillus_atrophaeus
  >KAF1680298.1-Bacillus_sp.
  >WP_048353666.1-Bacillus_glycinifermentans
  >WP_206702858.1-Bacillus_sonorensis
  
  ```
  
  I searched and tried a few scripts shared elsewhere (e.g. from [here](https://bioinformatics.stackexchange.com/questions/15326/how-to-remove-sequences-from-a-fasta-file-using-a-sequence-id-list-which-contain)), but I couldn't make it work with my files. So all the help will be really appreciated. Thanks in advance.
  
  PS. I am using using python usually with its own IDLE on windows 10 PC.
  
  I did one thing with [this code](https://bioinformatics.stackexchange.com/questions/15393/filtering-sequences-entries-by-headers-id-from-a-fasta-file-database). I removed the '>' character from the headers file and then this code works but it keeps those headers and their respective sequences. I think it works as intended in this code, but I want to remove those from the sequence file not to keep.
  
  Update: as asked below, I copied this code from another post.
  
  ```
  from Bio import SeqIO
  with open('headers.txt', 'r') as fin:
      to_keep = set([])
      for line in fin:
          to_keep.add(line.strip().split(' ')[0])
  with open('output.fa', 'w') as fout:
      for record in SeqIO.parse('sample.fa','fasta'):
          if record.id in to_keep:
              SeqIO.write(record, fout, 'fasta')
  
  ```
  
  **Tags:** python, fasta, sequence
  
  > **Comments:**
  > **username_1**: And how did they not work? I was about to write an answer but it is essentially what  had already written as an answer to the other question. Please [edit] and clarify why those other solutions didn't work, since right now, this looks like a duplicate question.  
  > **username_2**: Does this answer your question? [Filtering Sequences (entries) by headers ID from a Fasta file database](https://bioinformatics.stackexchange.com/questions/15393/filtering-sequences-entries-by-headers-id-from-a-fasta-file-database)  
  > **username_3**: This is not a good FASTA file format. IDs and description should be separate by space. I suggest to take a look at the official documentation. Good practice will make your life easier here an [example](https://www.ncbi.nlm.nih.gov/genbank/fastaformat/) [or](https://bioperl.org/formats/sequence_formats/FASTA_sequence_format)  
  > **username_0**: yes, I agree. but I do that on purpose b/c after some manipulation, I make a phylogenetic tree and visualize (usually on ITOL) which messes up the formatting, sometimes it is very long and I am interested to see only the accession ID and the bacterial name. So I delete the rest and make it as one continuous string so everything I need stays in and still within a short limit to be visible.  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_0 on 2023-12-21. Score: 2 | Accepted answer
  
  After modifying the code above in the original post, this one works the best for me. I got it working after putting 'not' in the second last line instead of "if record.id in to keep" to "if record.id not in to keep":
  
  ```
  from Bio import SeqIO
  with open('headers.txt', 'r') as fin:
      to_remove = set([])
      for line in fin:
          to_remove.add(line.strip().split(' ')[0])
  with open('output.fa', 'w') as fout:
      for record in SeqIO.parse('sample.fa','fasta'):
          if record.id in to_remove:
              pass
          else:
              SeqIO.write(record, fout, 'fasta')
  
  ```
  
  
  ---
  
  ### Answer by username_2 on 2023-12-20. Score: 5
  
  You can use something like seqkit: <https://bioinf.shenwei.me/seqkit/>
  
  seqkit has a `grep` sub-command that is one of the most flexible I've ever seen. Your use case is quite simple, you can use `seqkit grep -v` like so:
  
  ```
  seqkit grep -v -n -f list_of_IDs_to_exclude.txt in.fasta > out.fasta
  
  ```
  
  
  ---
  
  ### Answer by username_4 on 2023-12-20. Score: 2
  
  This should work
  
  ```
  import argparse
  
  ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.ArgumentDefaultsHelpFormatter)
  ap.add_argument('--fasta', required=True, type=str, help='Fasta file to extract from')
  ap.add_argument('--headers', required=True, type=str, help='Headers to remove, one per line.')
  
  conf = ap.parse_args()
  f = conf.fasta
  h = conf.headers
  header_set = set()
  
  with open(h) as header_file:
      for line in header_file:
          line = line.rstrip()
          header_set.add(line)
  
  with open(f) as fasta_file:
      remove_sequence = False
      for line in fasta_file:
          if '>' in line:
              header_id = line.split()[0].replace(">", "")
              if header_id in header_set:
                  remove_sequence = True
              else:
                  remove_sequence = False
                  print(line.rstrip())
          elif not remove_sequence:
              print(line.rstrip())
  
  exit()
  
  ```
  
  Save it as python script and then run
  
  `python remove_fasta.py --fasta sample.fa --headers listofheaders.txt > cleaned.fasta`
  
  > **Comments:**
  > **username_0**: Thanks. I tried this first thing in the morning. First it ran but didn't remove anything, then I removed '>' character from the headers list and it gave the correct expected cleaned file. What if I want to run it directly form the python IDLE and not from cmd?  
  > **username_4**: I'm glad it worked. As you can see, there are other options that will get the job done, but I understood that you want to learn how to do this in Python. I don't personally use the Python IDLE on Windows, but I believe you can save your code, open it in IDLE, and click on "run module". Make sure to modify the file locations as needed. I hope this helps, and good luck with your coding  
  
  ---
  
  ### Answer by username_1 on 2023-12-20. Score: 2
  
  I would argue writing a whole python script for this is overkill. Assuming you are using a\*nix based operating system like Linux or macOS, you can use the two scripts [I have posted here](https://bioinformatics.stackexchange.com/a/2651/298), `FastaToTbl` and `TblToFasta`, and just `grep` out the ones you do not want:
  
  ```
  FastaToTbl sample.fa | grep -v -xFf listofheaders.txt | TblToFasta > cleaned.fa
  
  ```
  
  > **Comments:**
  > **username_0**: Hi username_1, I am only using windows 10 PC with python. I am trying the above suggestions as well. Writing my own code is a big opportunity to learn as well.  
  > **username_1**: Oh. Windows? OK. But then please add that to your question. Windows isn't really used by bioinformaticians at all so nobody would guess you are trying this in windows. If you want to learn bioinformatics, I would urge you to use Linux or macOS or basically anything other than Windows.  
  > **username_0**: I agree with you. I am already on to it. ")  
  
  ---
  
  