Answers.Body: <p>I had not come across <code>FastqGeneralIterator</code> before, but I will start using it for some of my work!</p>
  
  <p>One answer is to replicate the code in the <a href="http://biopython.org/DIST/docs/api/Bio.SeqIO.QualityIO-pysrc.html#FastqPhredIterator" rel="nofollow noreferrer"><code>FastqPhredIterator</code></a>, which does exactly what you are looking to do.  </p>
  
  <p>However, My personal preference would be to use the biopython infrastructure as much as possible, especially when it comes to the tricky aspects of record parsing.  If the records you want extra info on are uncommon, then using <code>FastqPhredIterator</code> for phred quality extraction might work for you:</p>
  
  <pre class="lang-python prettyprint-override"><code>import io
  from Bio.SeqIO.QualityIO import FastqGeneralIterator, FastqPhredIterator
  
  with open("input.fastq", "rU") as handle:
      for title, sequence, quality in FastqGeneralIterator(handle):
          # do something with title, sequence, and quality
          # Set special_case to True when you want more info on the record
          special_case = False
          if special_case:
              record_stream = io.StringIO("\n".join([title, sequence, "+", quality]))
              record = next(FastqPhredIterator(record_stream))
              # do something special with record
  </code></pre>
  
  <p>What we are doing here is combining the record strings into one and dumping them into a <code>StringIO</code> object, which implements the file handle duck-type. We then hand the in-memory handle to <code>FastqPhredIterator</code> to parse the FASTQ record again. Make sure to set <code>special_case</code> to <code>True</code> only when you want extra info on a record, as the <code>special_case</code> branch makes at least two extra copies of the FASTQ record. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-23T18:05:12.270Z
Answers.Text: Very useful, thank you! I was hoping there were dedicated methods for these tasks.
Answers.UserDisplayName: Mark Ebbert
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2017-06-23T13:01:54.793Z
Answers.LastActivityDate: 2019-07-01T07:35:48.570Z
Answers.LastEditDate: 2019-07-01T07:35:48.570Z
Answers.UserDisplayName: winni2k
Body: <p>I'm contributing to a python-based project that uses <code>Biopython</code> to analyze fastq files. It currently uses <code>SeqIO.parse</code>, which populates various structures with all of the fastq information (including converting quality scores). There is apparently a faster (lighter-weight) parser called <a href="http://biopython.org/DIST/docs/api/Bio.SeqIO.QualityIO-module.html#FastqGeneralIterator" rel="nofollow noreferrer">FastqGeneralIterator</a> that doesn't populate all of these items.</p>
  
  <p>I'd like to use <code>FastqGeneralIterator</code>, but be able to perform some of the functions <code>SeqIO.parse</code> offers on the fly (not for <em>every</em> sequence). For example, I'd like to convert base quality to PHRED scores for specific sequences, but I don't see that function available.</p>
  
  <p>Is there an easy way to take a tuple output by <code>FastqGeneralIterator</code> and convert it into a proper <code>Bio.SeqIO</code> <code>SeqRecord</code>? </p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-23T12:02:38.777Z
Comments.Text: Not sure where is your problem exactly. Just in case it might be of interest to you, you can find some examples of python code parsing fastq (including `FastqGeneralIterator`, and faster alternatives) in the last part of this answer: https://bioinformatics.stackexchange.com/a/380/292
Comments.UserDisplayName: bli
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-23T12:07:48.900Z
Comments.Text: You may also be interested in this: http://biopython.org/DIST/docs/api/Bio.SeqIO.QualityIO.FastqPhredWriter-class.html (There's a `FastqPhredWriter` in Biopython)
Comments.UserDisplayName: bli
Tags: <fastq><python>
Title: How to maximize fastq parsing with FastqGeneralIterator (Bio.SeqIO.QualityIO)
ThreadText: # How to maximize fastq parsing with FastqGeneralIterator (Bio.SeqIO.QualityIO)
  
  (Asked by: username_0 on 2017-06-22)
  
  I'm contributing to a python-based project that uses `Biopython` to analyze fastq files. It currently uses `SeqIO.parse`, which populates various structures with all of the fastq information (including converting quality scores). There is apparently a faster (lighter-weight) parser called [FastqGeneralIterator](http://biopython.org/DIST/docs/api/Bio.SeqIO.QualityIO-module.html#FastqGeneralIterator) that doesn't populate all of these items.
  
  I'd like to use `FastqGeneralIterator`, but be able to perform some of the functions `SeqIO.parse` offers on the fly (not for *every* sequence). For example, I'd like to convert base quality to PHRED scores for specific sequences, but I don't see that function available.
  
  Is there an easy way to take a tuple output by `FastqGeneralIterator` and convert it into a proper `Bio.SeqIO` `SeqRecord`?
  
  **Tags:** fastq, python
  
  > **Comments:**
  > **username_1**: Not sure where is your problem exactly. Just in case it might be of interest to you, you can find some examples of python code parsing fastq (including `FastqGeneralIterator`, and faster alternatives) in the last part of this answer: https://bioinformatics.stackexchange.com/a/380/292  
  > **username_1**: You may also be interested in this: http://biopython.org/DIST/docs/api/Bio.SeqIO.QualityIO.FastqPhredWriter-class.html (There's a `FastqPhredWriter` in Biopython)  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2017-06-23. Score: 3 | Accepted answer
  
  I had not come across `FastqGeneralIterator` before, but I will start using it for some of my work!
  
  One answer is to replicate the code in the [`FastqPhredIterator`](http://biopython.org/DIST/docs/api/Bio.SeqIO.QualityIO-pysrc.html#FastqPhredIterator), which does exactly what you are looking to do.
  
  However, My personal preference would be to use the biopython infrastructure as much as possible, especially when it comes to the tricky aspects of record parsing. If the records you want extra info on are uncommon, then using `FastqPhredIterator` for phred quality extraction might work for you:
  
  ```python
  import io
  from Bio.SeqIO.QualityIO import FastqGeneralIterator, FastqPhredIterator
  
  with open("input.fastq", "rU") as handle:
      for title, sequence, quality in FastqGeneralIterator(handle):
          # do something with title, sequence, and quality
          # Set special_case to True when you want more info on the record
          special_case = False
          if special_case:
              record_stream = io.StringIO("\n".join([title, sequence, "+", quality]))
              record = next(FastqPhredIterator(record_stream))
              # do something special with record
  
  ```
  
  What we are doing here is combining the record strings into one and dumping them into a `StringIO` object, which implements the file handle duck-type. We then hand the in-memory handle to `FastqPhredIterator` to parse the FASTQ record again. Make sure to set `special_case` to `True` only when you want extra info on a record, as the `special_case` branch makes at least two extra copies of the FASTQ record.
  
  > **Comments:**
  > **username_0**: Very useful, thank you! I was hoping there were dedicated methods for these tasks.  
  
  ---
  
  