Answers.Body: <h2>A (bio)awk-based solution</h2>
  
  <h3>Generate the list of genus</h3>
  
  <pre><code>bioawk -c fastx '{print $comment}' terminase_large.fasta \
      | sed -r 's/.*\[(\w+).*/\1/' \
      | sort -u &gt; genus_names.txt
  </code></pre>
  
  <p>Here is how it works:</p>
  
  <ol>
  <li><p><a href="https://github.com/lh3/bioawk" rel="nofollow noreferrer"><code>bioawk</code></a> is like awk but with extra parsing capabilities. <code>-c fastx</code> parses the fields in fasta or fastq records. <code>{print $comment}</code> will print the <code>$comment</code> field which contain the genus names you want to extract.</p></li>
  <li><p>The <code>sed</code> command captures (with parentheses) the word (<code>\w+</code>) that is just after the opening square bracket (which should be the genus name, if there is only one opening square bracket on the line), and substitutes the whole line (that is, everything before the bracket (<code>.*</code>), the bracket (<code>\[</code>), the word, and everything after the word (<code>.*</code> again)) with just the captured word (<code>\1</code>).</p></li>
  <li><p>All this is sorted, and the <code>-u</code> option of <code>sort</code> ensures there is only one occurrence of each genus name in the output, which we put in the <code>genus_names.txt</code> file.</p></li>
  </ol>
  
  <h3>Extract the fasta records for each genus</h3>
  
  <pre><code>for genus in $(cat genus_names.txt)
  do
      bioawk -c fastx '{print}' terminase_large.fasta \
          | grep ${genus} \
          | awk -F "\t" '{print "&gt;"$4"\n"$2}' \
          | sed -r 's/^&gt;[^\[]+\[(.*)\]/&gt;\1/' &gt; ${genus}.fasta
  done
  </code></pre>
  
  <p>Here is how it works:</p>
  
  <ol>
  <li><p>We loop on the content of the previously established list of genus names.</p></li>
  <li><p>For each value of <code>${genus}</code>, we parse again the fasta file with <code>bioawk</code>. This time, for a given fasta record, we print all the elements that have been parsed. They appear on one line, in the following order (according to <code>bioawk -c help</code>): <code>1:name 2:seq 3:qual 4:comment</code> (we actually only need <code>seq</code> and <code>comment</code> for the following steps, but there are less risks of making errors at the later <code>awk</code> step if we keep the four fields).</p></li>
  <li><p>We use <code>grep</code> to select the resulting lines that contain the genus name we're currently dealing with.</p></li>
  <li><p>The lines that have been successfully selected are then parsed by awk and reformatted into fasta format using only the comment part in the header. The <code>-F "\t"</code> is to indicate that the field delimiters are tabulations (which is how bioawk has written the records when we did <code>{print}</code>).</p></li>
  <li><p>We use <code>sed</code> to keep only the part inside the square brackets (again using a pattern capture between parentheses, that we use in the substitution part with <code>\1</code>), and the output is written in a file named using the current value of <code>${genus}</code>.</p></li>
  </ol>
  
  <p>More details on the last <code>sed</code> command:</p>
  
  <p>We match the whole header lines. They start with ">" (<code>^&gt;</code>), then there are some non-"[" characters (<code>[^\[]+</code>), then a "[" (<code>\[</code>), then something we capture (<code>(.*)</code>), then a "]" (<code>\]</code>). We replace them with the captured part. The sequence lines are unaffected because they don't match the above regular expression.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-07T15:06:06.557Z
Answers.LastActivityDate: 2017-08-09T12:18:26.183Z
Answers.LastEditDate: 2017-08-09T12:18:26.183Z
Answers.UserDisplayName: bli
Answers.Body: <p>Splitting into multiple files and changing the IDs can be easily done:</p>
  
  <pre class="lang-none prettyprint-override"><code>perl -pe 'if(/&gt;/){/\[(.*?)\]\s*$/; $_="&gt; $1\n"}' file.fa | 
      awk '(/^&gt;/){name=$2} {print &gt;&gt; name".fa"}'
  </code></pre>
  
  <p>That assumes all your FASTA headers have <code>[foo bar baz]</code> as the last element of a line. It will create a file called <code>foo.fa</code> (the bacterium's name) with all sequences saved there. </p>
  
  <h3>Explanation</h3>
  
  <ul>
  <li><code>perl -pe</code> : run the script given by <code>-e</code> on each line of the input file, and print the resulting line. </li>
  <li><code>if(/&gt;/)</code> : if this line starts with a <code>&gt;</code>. \</li>
  <li><code>/\[(.*?)\]\s*$/</code> : match an opening bracket (<code>\[</code>), then capture (that's what the parentheses do, they capture a pattern so we can refer to it as <code>$1</code>) everything until the first <code>]</code> (<code>.*?\]</code>)</li>
  <li><code>$_="&gt; $1\n"</code> : the <code>$_</code> special variable in Perl is (in this case) the current line. So, <code>$_=foo</code> means "make the current line read <code>foo</code>. Since the <code>-p</code> prints each input line, changing the value of <code>$_</code> means the changed value will be printed. So here, we are printing <code>&gt;</code>, whatever was in the square brackets (<code>$1</code>) and a newline character. </li>
  </ul>
  
  <p>The output of the perl command alone on your example input file is:</p>
  
  <pre class="lang-none prettyprint-override"><code>$ perl -pe 'if(/&gt;/){/\[(.*)\]\s*$/; $_="&gt; $1\n"}' file.fa
  &gt; Arthrobacter phage Mudcat
  MGLSNTATPLYYGQF...
  
  &gt; Achromobacter phage phiAxp-3
  MSNVLLKQELDEWL...
  
  &gt; Delftia phage RG-2014
  MSEPRKLVKKTLD...
  </code></pre>
  
  <p>So, we now pass it through <code>awk</code> which does:</p>
  
  <ul>
  <li><code>(/^&gt;/){a=$2}</code> : if this line starts with an <code>&gt;</code>, save the second field (the bacterial species) as the variable <code>name</code>. </li>
  <li><code>{print &gt;&gt; name".fa"}</code> : print each line into a file whose name is the current value of the variable <code>name</code> with a <code>.fa.</code> extension. </li>
  </ul>
  
  <hr>
  
  <p>If you prefer python scripts to the one-liner approach, you can do the same thing with:</p>
  
  <pre><code>#!/usr/bin/env python
  import re
  outFile = None
  with open("file.fa", "r") as inFile, open(species + ".fa", 'a') as outFile:
      for line in inFile:
          line = line.rstrip()
          if line.startswith('&gt;'):
              regex = re.compile('.*\[((.+?)\s+.*?)\].*')
              matches = regex.search(line)
              species = matches[2]
              outFile.write('&gt;%s\n' % matches[1])
          else:
              outFile.write("%s\n" % line)
  </code></pre>
  
  <hr>
  
  <p>As for your script, you've got the right idea, but have a small bug. You forgot to remove the <code>\n</code> from your input file, so it looks for <code>Arthrobacter\n</code> instead of <code>Arthrobacter</code>. The golden rule of debugging is "print all the things". If you add <code>print("Searchterm: ",searchterm)</code>  you will see:</p>
  
  <pre class="lang-none prettyprint-override"><code>Searchterm:  (terminase large subunit AND viruses[Organism]) ANDArthrobacter  
  AND refseq[Filter]
  Searchterm:  (terminase large subunit AND viruses[Organism]) ANDAchromobacter  
  AND refseq[Filter]
  Searchterm:  (terminase large subunit AND viruses[Organism]) ANDDelftia  
  AND refseq[Filter]
  </code></pre>
  
  <p>So, you need to remove the newline characters and add a space like so (I also made it a bit more "pythonic" and conforming to the Python syntax guidelines):</p>
  
  <pre><code>#!/usr/bin/env python
  from Bio import Entrez
  Entrez.email = "example@example.org"
  
  with open("species2.txt", "r") as in_handle:
      for line in in_handle:
          line = line.rstrip()
          searchterm = ("(terminase large subunit AND viruses[Organism]) " +
                        "AND %s AND refseq[Filter]" % line)
          print("Searchterm: ", searchterm)
          searchResultHandle = Entrez.esearch(db="protein",
                                              term=searchterm, retmax=1000)
          searchResult = Entrez.read(searchResultHandle)
          ids = searchResult["IdList"]
  
          handle = Entrez.efetch(db="protein", id=ids,
                                 rettype="fasta", retmode="text")
          record = handle.read()
  
          with open('terminase_large_' + str(line[:-1]) + '.fasta', 'w') as out_handle:
              out_handle.write(record.rstrip('\n'))
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-31T10:55:43.677Z
Answers.Text: Is it possible to do the perl script in python? I don't have perl on my pc installed.
  
  About the second part: it works! Sort of. When I debug my script I can see that python gets the desired fasta file (a url link is created), and it creates a file with the correct name, but somehow doesn't input the data into it
Answers.UserDisplayName: tahunami
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-31T10:57:38.837Z
Answers.Text: @tahunami what do you mean, unless you're using Windows, it should be installed by default. Are you sure it's not installed?
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-31T11:10:24.727Z
Answers.Text: I've saved in itno a .pl file, and run it on perl, but got "Execution of split.pl aborted due to compilation errors"
Answers.UserDisplayName: tahunami
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-31T11:17:48.610Z
Answers.Text: @tahunami no, that is a one-liner. Don't save as a file, just paste it directly into an open terminal. That's just a *nix command. In any case, I also added a python version for you.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-31T10:35:37.210Z
Answers.LastActivityDate: 2017-08-11T13:19:48.687Z
Answers.LastEditDate: 2017-08-11T13:19:48.687Z
Answers.UserDisplayName: terdon
Body: <p>I have a problem: I've managed to download a massive fasta file of 1500 sequences, but now I want to split them into separate fasta files based on the genus.</p>
  <p>EDIT<br />
  The fasta file looks like this:</p>
  <pre><code>terminase_large.fasta
  &gt;YP_009300697.1 terminase large subunit [Arthrobacter phage Mudcat]
  MGLSNTATPLYYGQF...
  
  &gt;YP_009208724.1 hypothetical protein ADP65_00072 [Achromobacter phage phiAxp-3]
  MSNVLLKQELDEWL...
  
  &gt;YP_009148449.1 large terminase subunit [Delftia phage RG-2014]
  MSEPRKLVKKTLD...
  ...
  </code></pre>
  <p>I would like to end up with something like this:</p>
  <pre><code>mycobacterium_terminase.fasta
  &gt;Mycobacterium phage JAMaL
  MVRKKPPPELE...
  
  &gt;Mycobacterium phage Bruin
  MEVCGYTLDDI...
  
  &gt;Mycobacterium phage Zaka
  MSLDNHLPELA...
  
  salmonella_terminase.fasta
  &gt;Salmonella virus SETP7
  MNVDITATEPQ...
  
  &gt;Salmonella virus SE2
  MEGGSDSLIAM...
  and so on...
  </code></pre>
  <p>So that the label on the alignment would have the name of the phage and not the protein's ID.</p>
  <p>I've managed to extract the genus names of my organism with this:</p>
  <pre><code>outfile = open('species.txt', 'w')
  with open('terminase_large.fasta') as fd:
      for line in fd:
          if line.startswith('&gt;'):
              if '[' in line:
                  name=line.split('[')[-1]
                  name=name.split(' ', 1)[0]
                  outfile.write(name[:] + &quot;\n&quot;)
  outfile.close()
  </code></pre>
  <p>And got to extract only the unique names with this:</p>
  <pre><code>lines_seen = set()
  outfile = open('species2.txt', &quot;w&quot;)
  for line in open(&quot;species.txt&quot;, &quot;r&quot;):
      if line not in lines_seen:  # not a duplicate
          outfile.write(line)
          lines_seen.add(line)
  outfile.close()
  </code></pre>
  <p>(Can I merge those two scripts together?)</p>
  <p>Now, my genus names look like this:</p>
  <pre><code>Arthrobacter
  Achromobacter
  Delftia
  ....
  </code></pre>
  <p>I tried automating my script to get the Entrez data, but it gives me the 'Supplied id parameter is empty' message</p>
  <p>My code looks like this:</p>
  <pre><code>from Bio import Entrez
  Entrez.email = &quot;example@example.org&quot;
  
  for line in open(&quot;species2.txt&quot;, &quot;r&quot;):
      searchterm = &quot;(terminase large subunit AND viruses[Organism]) AND&quot; +line+ &quot;AND refseq[Filter]&quot;
      searchResultHandle = Entrez.esearch(db=&quot;protein&quot;, term=searchterm, retmax=1000)
      searchResult = Entrez.read(searchResultHandle)
      ids = searchResult[&quot;IdList&quot;]
  
      handle = Entrez.efetch(db=&quot;protein&quot;, id=ids, rettype=&quot;fasta&quot;, retmode=&quot;text&quot;)
      record = handle.read()
  
      out_handle = open('terminase_large_'+str(line[:-1])+'.fasta', 'w')
      out_handle.write(record.rstrip('\n'))
  </code></pre>
  <p>Can someone help me with it?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-31T09:48:45.160Z
Comments.Text: Please [edit] your question and i) show us a short example of your input file, including different genera and ii) show us the output you would expect from that file. That way, we can test our solutions and be sure to give you what you need.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-31T09:49:57.617Z
Comments.Text: Also, you seem to have bacterial genera, why are you searching for viruses? Of course that won't find any results.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-31T09:55:57.740Z
Comments.Text: My keywords use the bacteria genera and the virus as organism, since I want specific phages that infect bacteria, eg Enterobacter phage or Vibrio phage
Comments.UserDisplayName: tahunami
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-31T10:05:27.757Z
Comments.Text: Ah yes, sorry, I thought you were specifying the bacterial genus as `[ORGANISM]` as well. So, is it enough to parse the existing file or do you need your script to be modified to create the multiple files next time it runs? Modifying the existing one and splitting it into subfiles by genus is very easy. Would that be enough?
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-31T10:07:01.440Z
Comments.Text: @terdon sure, I just didn't know which was easier: splitting the existing file, which has the genera mixed up or creating a new fasta file by searching Entrez
Comments.UserDisplayName: tahunami
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-31T11:12:13.797Z
Comments.Text: Your output and input don't match, I can't see "Mycobacterium" in the input file. Is this intentional?
Comments.UserDisplayName: gringer
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-31T11:20:14.300Z
Comments.Text: @gringer yeah, it's confusing. The OP wants to i) change the fasta ID lines and ii) split into multiple files, collecting all sequences from phages infecting a given bacterial genus together. They only showed the output for the first part and not the second.
Comments.UserDisplayName: terdon
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-31T11:48:35.890Z
Comments.Text: Sorry for the bad files, I will try to edit it once more
Comments.UserDisplayName: tahunami
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-31T19:34:19.057Z
Comments.Text: Thanks @tahunami, the output seems to fit better with your question now. Are you able to change the input so that it only includes information in the output file? I still can't see any of the output proteins or names in the input example. Feel free to put dots in to demonstrate "skipping over thousands of lines", but I think it's important that at least everything in the output example should be in the input example.
Comments.UserDisplayName: gringer
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-08-04T13:48:57.860Z
Comments.Text: I'm not sure this solves your problem, but I would use `line.strip()` in `searchterm`, not just `line`.
Comments.UserDisplayName: bli
Tags: <fasta><python>
Title: Subset FASTA file by species name
ThreadText: # Subset FASTA file by species name
  
  (Asked by: username_0 on 2017-07-31)
  
  I have a problem: I've managed to download a massive fasta file of 1500 sequences, but now I want to split them into separate fasta files based on the genus.
  
  EDIT  
  The fasta file looks like this:
  
  ```
  terminase_large.fasta
  >YP_009300697.1 terminase large subunit [Arthrobacter phage Mudcat]
  MGLSNTATPLYYGQF...
  
  >YP_009208724.1 hypothetical protein ADP65_00072 [Achromobacter phage phiAxp-3]
  MSNVLLKQELDEWL...
  
  >YP_009148449.1 large terminase subunit [Delftia phage RG-2014]
  MSEPRKLVKKTLD...
  ...
  
  ```
  
  I would like to end up with something like this:
  
  ```
  mycobacterium_terminase.fasta
  >Mycobacterium phage JAMaL
  MVRKKPPPELE...
  
  >Mycobacterium phage Bruin
  MEVCGYTLDDI...
  
  >Mycobacterium phage Zaka
  MSLDNHLPELA...
  
  salmonella_terminase.fasta
  >Salmonella virus SETP7
  MNVDITATEPQ...
  
  >Salmonella virus SE2
  MEGGSDSLIAM...
  and so on...
  
  ```
  
  So that the label on the alignment would have the name of the phage and not the protein's ID.
  
  I've managed to extract the genus names of my organism with this:
  
  ```
  outfile = open('species.txt', 'w')
  with open('terminase_large.fasta') as fd:
      for line in fd:
          if line.startswith('>'):
              if '[' in line:
                  name=line.split('[')[-1]
                  name=name.split(' ', 1)[0]
                  outfile.write(name[:] + "\n")
  outfile.close()
  
  ```
  
  And got to extract only the unique names with this:
  
  ```
  lines_seen = set()
  outfile = open('species2.txt', "w")
  for line in open("species.txt", "r"):
      if line not in lines_seen:  # not a duplicate
          outfile.write(line)
          lines_seen.add(line)
  outfile.close()
  
  ```
  
  (Can I merge those two scripts together?)
  
  Now, my genus names look like this:
  
  ```
  Arthrobacter
  Achromobacter
  Delftia
  ....
  
  ```
  
  I tried automating my script to get the Entrez data, but it gives me the 'Supplied id parameter is empty' message
  
  My code looks like this:
  
  ```
  from Bio import Entrez
  Entrez.email = "example@example.org"
  
  for line in open("species2.txt", "r"):
      searchterm = "(terminase large subunit AND viruses[Organism]) AND" +line+ "AND refseq[Filter]"
      searchResultHandle = Entrez.esearch(db="protein", term=searchterm, retmax=1000)
      searchResult = Entrez.read(searchResultHandle)
      ids = searchResult["IdList"]
  
      handle = Entrez.efetch(db="protein", id=ids, rettype="fasta", retmode="text")
      record = handle.read()
  
      out_handle = open('terminase_large_'+str(line[:-1])+'.fasta', 'w')
      out_handle.write(record.rstrip('\n'))
  
  ```
  
  Can someone help me with it?
  
  **Tags:** fasta, python
  
  > **Comments:**
  > **username_1**: Please [edit] your question and i) show us a short example of your input file, including different genera and ii) show us the output you would expect from that file. That way, we can test our solutions and be sure to give you what you need.  
  > **username_1**: Also, you seem to have bacterial genera, why are you searching for viruses? Of course that won't find any results.  
  > **username_0**: My keywords use the bacteria genera and the virus as organism, since I want specific phages that infect bacteria, eg Enterobacter phage or Vibrio phage  
  > **username_1**: Ah yes, sorry, I thought you were specifying the bacterial genus as `[ORGANISM]` as well. So, is it enough to parse the existing file or do you need your script to be modified to create the multiple files next time it runs? Modifying the existing one and splitting it into subfiles by genus is very easy. Would that be enough?  
  > **username_0**: @username_1 sure, I just didn't know which was easier: splitting the existing file, which has the genera mixed up or creating a new fasta file by searching Entrez  
  > **username_2**: Your output and input don't match, I can't see "Mycobacterium" in the input file. Is this intentional?  
  > **username_1**: @username_2 yeah, it's confusing. The OP wants to i) change the fasta ID lines and ii) split into multiple files, collecting all sequences from phages infecting a given bacterial genus together. They only showed the output for the first part and not the second.  
  > **username_0**: Sorry for the bad files, I will try to edit it once more  
  > **username_2**: Thanks @username_0, the output seems to fit better with your question now. Are you able to change the input so that it only includes information in the output file? I still can't see any of the output proteins or names in the input example. Feel free to put dots in to demonstrate "skipping over thousands of lines", but I think it's important that at least everything in the output example should be in the input example.  
  > **username_3**: I'm not sure this solves your problem, but I would use `line.strip()` in `searchterm`, not just `line`.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2017-07-31. Score: 7 | Accepted answer
  
  Splitting into multiple files and changing the IDs can be easily done:
  
  ```none
  perl -pe 'if(/>/){/\[(.*?)\]\s*$/; $_="> $1\n"}' file.fa | 
      awk '(/^>/){name=$2} {print >> name".fa"}'
  
  ```
  
  That assumes all your FASTA headers have `[foo bar baz]` as the last element of a line. It will create a file called `foo.fa` (the bacterium's name) with all sequences saved there.
  
  ### Explanation
  
  * `perl -pe` : run the script given by `-e` on each line of the input file, and print the resulting line.
  * `if(/>/)` : if this line starts with a `>`. \
  * `/\[(.*?)\]\s*$/` : match an opening bracket (`\[`), then capture (that's what the parentheses do, they capture a pattern so we can refer to it as `$1`) everything until the first `]` (`.*?\]`)
  * `$_="> $1\n"` : the `$_` special variable in Perl is (in this case) the current line. So, `$_=foo` means "make the current line read `foo`. Since the `-p` prints each input line, changing the value of `$_` means the changed value will be printed. So here, we are printing `>`, whatever was in the square brackets (`$1`) and a newline character.
  
  The output of the perl command alone on your example input file is:
  
  ```none
  $ perl -pe 'if(/>/){/\[(.*)\]\s*$/; $_="> $1\n"}' file.fa
  > Arthrobacter phage Mudcat
  MGLSNTATPLYYGQF...
  
  > Achromobacter phage phiAxp-3
  MSNVLLKQELDEWL...
  
  > Delftia phage RG-2014
  MSEPRKLVKKTLD...
  
  ```
  
  So, we now pass it through `awk` which does:
  
  * `(/^>/){a=$2}` : if this line starts with an `>`, save the second field (the bacterial species) as the variable `name`.
  * `{print >> name".fa"}` : print each line into a file whose name is the current value of the variable `name` with a `.fa.` extension.
  
  ---
  
  If you prefer python scripts to the one-liner approach, you can do the same thing with:
  
  ```
  #!/usr/bin/env python
  import re
  outFile = None
  with open("file.fa", "r") as inFile, open(species + ".fa", 'a') as outFile:
      for line in inFile:
          line = line.rstrip()
          if line.startswith('>'):
              regex = re.compile('.*\[((.+?)\s+.*?)\].*')
              matches = regex.search(line)
              species = matches[2]
              outFile.write('>%s\n' % matches[1])
          else:
              outFile.write("%s\n" % line)
  
  ```
  
  ---
  
  As for your script, you've got the right idea, but have a small bug. You forgot to remove the `\n` from your input file, so it looks for `Arthrobacter\n` instead of `Arthrobacter`. The golden rule of debugging is "print all the things". If you add `print("Searchterm: ",searchterm)` you will see:
  
  ```none
  Searchterm:  (terminase large subunit AND viruses[Organism]) ANDArthrobacter  
  AND refseq[Filter]
  Searchterm:  (terminase large subunit AND viruses[Organism]) ANDAchromobacter  
  AND refseq[Filter]
  Searchterm:  (terminase large subunit AND viruses[Organism]) ANDDelftia  
  AND refseq[Filter]
  
  ```
  
  So, you need to remove the newline characters and add a space like so (I also made it a bit more "pythonic" and conforming to the Python syntax guidelines):
  
  ```
  #!/usr/bin/env python
  from Bio import Entrez
  Entrez.email = "example@example.org"
  
  with open("species2.txt", "r") as in_handle:
      for line in in_handle:
          line = line.rstrip()
          searchterm = ("(terminase large subunit AND viruses[Organism]) " +
                        "AND %s AND refseq[Filter]" % line)
          print("Searchterm: ", searchterm)
          searchResultHandle = Entrez.esearch(db="protein",
                                              term=searchterm, retmax=1000)
          searchResult = Entrez.read(searchResultHandle)
          ids = searchResult["IdList"]
  
          handle = Entrez.efetch(db="protein", id=ids,
                                 rettype="fasta", retmode="text")
          record = handle.read()
  
          with open('terminase_large_' + str(line[:-1]) + '.fasta', 'w') as out_handle:
              out_handle.write(record.rstrip('\n'))
  
  ```
  
  > **Comments:**
  > **username_0**: Is it possible to do the perl script in python? I don't have perl on my pc installed.  About the second part: it works! Sort of. When I debug my script I can see that python gets the desired fasta file (a url link is created), and it creates a file with the correct name, but somehow doesn't input the data into it  
  > **username_1**: @username_0 what do you mean, unless you're using Windows, it should be installed by default. Are you sure it's not installed?  
  > **username_0**: I've saved in itno a .pl file, and run it on perl, but got "Execution of split.pl aborted due to compilation errors"  
  > **username_1**: @username_0 no, that is a one-liner. Don't save as a file, just paste it directly into an open terminal. That's just a *nix command. In any case, I also added a python version for you.  
  
  ---
  
  ### Answer by username_3 on 2017-08-07. Score: 2
  
  ## A (bio)awk-based solution
  
  ### Generate the list of genus
  
  ```
  bioawk -c fastx '{print $comment}' terminase_large.fasta \
      | sed -r 's/.*\[(\w+).*/\1/' \
      | sort -u > genus_names.txt
  
  ```
  
  Here is how it works:
  
  1. [`bioawk`](https://github.com/lh3/bioawk) is like awk but with extra parsing capabilities. `-c fastx` parses the fields in fasta or fastq records. `{print $comment}` will print the `$comment` field which contain the genus names you want to extract.
  2. The `sed` command captures (with parentheses) the word (`\w+`) that is just after the opening square bracket (which should be the genus name, if there is only one opening square bracket on the line), and substitutes the whole line (that is, everything before the bracket (`.*`), the bracket (`\[`), the word, and everything after the word (`.*` again)) with just the captured word (`\1`).
  3. All this is sorted, and the `-u` option of `sort` ensures there is only one occurrence of each genus name in the output, which we put in the `genus_names.txt` file.
  
  ### Extract the fasta records for each genus
  
  ```
  for genus in $(cat genus_names.txt)
  do
      bioawk -c fastx '{print}' terminase_large.fasta \
          | grep ${genus} \
          | awk -F "\t" '{print ">"$4"\n"$2}' \
          | sed -r 's/^>[^\[]+\[(.*)\]/>\1/' > ${genus}.fasta
  done
  
  ```
  
  Here is how it works:
  
  1. We loop on the content of the previously established list of genus names.
  2. For each value of `${genus}`, we parse again the fasta file with `bioawk`. This time, for a given fasta record, we print all the elements that have been parsed. They appear on one line, in the following order (according to `bioawk -c help`): `1:name 2:seq 3:qual 4:comment` (we actually only need `seq` and `comment` for the following steps, but there are less risks of making errors at the later `awk` step if we keep the four fields).
  3. We use `grep` to select the resulting lines that contain the genus name we're currently dealing with.
  4. The lines that have been successfully selected are then parsed by awk and reformatted into fasta format using only the comment part in the header. The `-F "\t"` is to indicate that the field delimiters are tabulations (which is how bioawk has written the records when we did `{print}`).
  5. We use `sed` to keep only the part inside the square brackets (again using a pattern capture between parentheses, that we use in the substitution part with `\1`), and the output is written in a file named using the current value of `${genus}`.
  
  More details on the last `sed` command:
  
  We match the whole header lines. They start with ">" (`^>`), then there are some non-"[" characters (`[^\[]+`), then a "[" (`\[`), then something we capture (`(.*)`), then a "]" (`\]`). We replace them with the captured part. The sequence lines are unaffected because they don't match the above regular expression.
  
  
  ---
  
  