Answers.Body: <p>Yes, reported accuracy can be tested and compared with actual accuracy by mapping reads from a reference sample to a known-perfect reference sequence. It's usually done in aggregate, at a read level rather than at single-base level.</p>
  <p>I like using Heng Li's <a href="http://lh3.github.io/2018/11/25/on-the-definition-of-sequence-identity" rel="nofollow noreferrer">gap-compressed identity statistic</a> for this:</p>
  <p><a href="https://i.sstatic.net/Y1V6J.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Y1V6J.jpg" alt="mean base quality vs gap-compressed identity for Nanopore reads" /></a></p>
  <p>Alternatively, fish out a few reads at random, and use BLAST, which reports an identity score.</p>
  <p>If the sample is not a reference sample with a precisely known sequence, then this approach will not work, because introduced variation will confound the comparison.</p>
  <p>While it might be possible to create a new reference sequence from the reads, this introduces the possibility of systematic error in the sequencer that would be hidden by the new reference sequence, and when you're looking at error rates in the realm of q30 and above (as in your plot), those small differences are going to matter a lot.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-16T01:36:14.647Z
Answers.Text: Thank you very much for your answer! But how can the gap-compressed identity/BLAST approach distinguish between SNPs and sequencing error? If the gap-comprs identity is really low, should I compare the number of SNPs called with "normal" WGS sample? If in this case, the number of SNPs is also higher than "normal", then I can infer the low gap-compressed identity score originates from high number of SNPs in the sample? Or should I take a look at the allelic count in the VCF, since it should be small at the error base? Sorry for having so many additional questions, really appreciated your help!
Answers.UserDisplayName: benson23
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-16T02:24:29.620Z
Answers.Text: Seems like you're asking a different question with that (i.e. "How can I trust variants discovered through high-throughput sequencing?"). The problems you state are definitely an issue, and they can't be resolved fully when using a non-reference sample as input.
Answers.UserDisplayName: gringer
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-15T19:09:41.380Z
Answers.LastActivityDate: 2023-02-16T02:24:54.417Z
Answers.LastEditDate: 2023-02-16T02:24:54.417Z
Answers.UserDisplayName: gringer
Body: <p>Recently I have ran a human WGS on the BGI DNBSEQ system, and their FASTQ quality scores seem to be quite impressive, where the Phred scores barely deteriorate along the read length when checked on FastQC.</p>
  <p><a href="https://i.sstatic.net/okf48.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/okf48.png" alt="DNBSEQ_fastqc" /></a></p>
  <p>Since the DNBSEQ technology is relatively new to me, I wonder if there's any bioinformatic approach to test whether the Phred scores listed in these FASTQ files are authentic or not? I'm kind of worried that their sequencer would just falsely assign high Phred scores to all bases.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-15T14:23:15.333Z
Comments.Text: How do the other FastQC metrics look? How does the alignment/mapping QC look?
Comments.UserDisplayName: Chris_Rands
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-15T18:17:08.647Z
Comments.Text: Good question!  To Chris_Rand's point, you could try to test it by seeing how often a base call actually looks correct versus what each Q score implies, if you can gather that information separately.  My first thought was, what does BGI claim goes into their Q scores in the first place?  If you look it up for Illumina you can find at least general overviews of how they derive them from the image processing metrics like signal intensity and signal-to-noise ratio.  I can't find anything like that for DNBSEQ, so it seems like more of an open question there.
Comments.UserDisplayName: Jesse
Tags: <fastq><wgs><fastqc><phred-scores>
Title: Can we test the accuracy of Phred scores shown in FASTQ files
ThreadText: # Can we test the accuracy of Phred scores shown in FASTQ files
  
  (Asked by: username_0 on 2023-02-15)
  
  Recently I have ran a human WGS on the BGI DNBSEQ system, and their FASTQ quality scores seem to be quite impressive, where the Phred scores barely deteriorate along the read length when checked on FastQC.
  
  [![DNBSEQ_fastqc](https://i.sstatic.net/okf48.png)](https://i.sstatic.net/okf48.png)
  
  Since the DNBSEQ technology is relatively new to me, I wonder if there's any bioinformatic approach to test whether the Phred scores listed in these FASTQ files are authentic or not? I'm kind of worried that their sequencer would just falsely assign high Phred scores to all bases.
  
  **Tags:** fastq, wgs, fastqc, phred-scores
  
  > **Comments:**
  > **username_1**: How do the other FastQC metrics look? How does the alignment/mapping QC look?  
  > **username_2**: Good question!  To Chris_Rand's point, you could try to test it by seeing how often a base call actually looks correct versus what each Q score implies, if you can gather that information separately.  My first thought was, what does BGI claim goes into their Q scores in the first place?  If you look it up for Illumina you can find at least general overviews of how they derive them from the image processing metrics like signal intensity and signal-to-noise ratio.  I can't find anything like that for DNBSEQ, so it seems like more of an open question there.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2023-02-15. Score: 3
  
  Yes, reported accuracy can be tested and compared with actual accuracy by mapping reads from a reference sample to a known-perfect reference sequence. It's usually done in aggregate, at a read level rather than at single-base level.
  
  I like using Heng Li's [gap-compressed identity statistic](http://lh3.github.io/2018/11/25/on-the-definition-of-sequence-identity) for this:
  
  [![mean base quality vs gap-compressed identity for Nanopore reads](https://i.sstatic.net/Y1V6J.jpg)](https://i.sstatic.net/Y1V6J.jpg)
  
  Alternatively, fish out a few reads at random, and use BLAST, which reports an identity score.
  
  If the sample is not a reference sample with a precisely known sequence, then this approach will not work, because introduced variation will confound the comparison.
  
  While it might be possible to create a new reference sequence from the reads, this introduces the possibility of systematic error in the sequencer that would be hidden by the new reference sequence, and when you're looking at error rates in the realm of q30 and above (as in your plot), those small differences are going to matter a lot.
  
  > **Comments:**
  > **username_0**: Thank you very much for your answer! But how can the gap-compressed identity/BLAST approach distinguish between SNPs and sequencing error? If the gap-comprs identity is really low, should I compare the number of SNPs called with "normal" WGS sample? If in this case, the number of SNPs is also higher than "normal", then I can infer the low gap-compressed identity score originates from high number of SNPs in the sample? Or should I take a look at the allelic count in the VCF, since it should be small at the error base? Sorry for having so many additional questions, really appreciated your help!  
  > **username_3**: Seems like you're asking a different question with that (i.e. "How can I trust variants discovered through high-throughput sequencing?"). The problems you state are definitely an issue, and they can't be resolved fully when using a non-reference sample as input.  
  
  ---
  
  