Answers.Body: <p>I personally don't think BQSR has a huge impact on variant calling, but you don't really need to guess. If you run GATK BQSR, it outputs a table and charts of exactly how much quality scores are adjusted. The adjustment will vary depending on the position in the read and genomic context (previous and following base). In my experience, the difference is a few points at most, but it's certainly noticeable.</p>
  
  <p>GATK recommends BQSR for both genome and exome data, which is normally much higher than 15x.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-01T22:07:27.130Z
Answers.LastActivityDate: 2017-06-01T22:07:27.130Z
Answers.UserDisplayName: burger
Answers.Body: <p>That's a good question.</p>
  
  <p>I'd say that you don't need to bother with variant recalibration for</p>
  
  <ul>
  <li>low number of samples (e.g., just two trios); I could not get GTAK recalibration of variant scores to work anyway</li>
  <li>high-coverage samples (e.g., X Ten genomes with 30x coverage) where the DNA samples themselves are of high, comparable quality and have been sequenced with consistent technology.</li>
  </ul>
  
  <p>Generally, it is my impression that a lot of the thoughts and advanced statistical models built into GATK come from the earlier phases of the 1000 Genomes project. This means (1) low-coverage, (2) different coverage genomes (3) sequenced with varying technology versions by (4) different samples and (5) population sequencing.</p>
  
  <p>If you are in a clinical setting where you do 30x sequencing on X Ten platforms only anyway, then variant recalibration will probably not help you that much.</p>
  
  <p>On the other hand, if you are integrating many data sets from different data centers and machine versions etc., variant recalibration might be worth a shot.</p>
  
  <p>A good check would be looking at genotype quality distributions and other variant/quality related metrics before and after recalibration.</p>
  
  <p>Anyone: please correct me if I'm wrong!</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-01T10:39:58.277Z
Answers.Text: Are you talking about base quality score recalibration (BQSR) here or about  variant quality score recalibration (VQSR)? I think the OP is referring to BQSR but you are discussing VQSR.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-01T13:28:03.800Z
Answers.Text: Yes, OP confirmed. The question is about BQSR so I am afraid you're answering the wrong question.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-01T19:58:13.687Z
Answers.Text: *sigh* and there I thought I could contribute something.
Answers.UserDisplayName: Manuel
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-31T23:27:28.080Z
Answers.LastActivityDate: 2017-05-31T23:27:28.080Z
Answers.UserDisplayName: Manuel
Answers.Body: <p>Ideally these BQSR methods were made keeping in mind how technical errors will actually screw up the base quality calls and when the machines were still more on development phase while being used for the 1000G project. As of now machines are more powerful and strong where it will be unlikely to use it but still we use with listed SNPs to find the covariates and build a model around the data using the information with machine learning tricks to improve the quality of those base calls. Ideally it should be more appropriate when old machines from Illumina or other standard companies are being used but with new machines which are much powerful and having high throughput they should tend to go down. I do not recall if such tests have been made but obviously I know new sequencing machine always make such tests to show that they have reduced such errors but still recommend such BQSR for variant calls. Now the problem is the list of SNPs, this to me is the real problem since the list we use is far from being Gold standard and if that is not properly taken care of everything we infer about quality is still shaky. <a href="https://gatkforums.broadinstitute.org/gatk/discussion/44/base-quality-score-recalibration-bqsr" rel="nofollow noreferrer">This link</a> is pretty informative but it's an old one. I would really see improvements with new sequencers.  However very less people care about such tests in academic research and also translational lab will really not invest time and money on such unless the facility has some bioinformaticians who always does such testing while buying a new sequencer for the institute. In terms of clinical genomics for finding variants I reckon most powerful and up-to-date sequencers should be used but not sure if they still use BQSR and if so what is the list they use to build model of covariation around the data.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-06T11:14:57.917Z
Answers.Text: Note that the question is motivated by non-model organism research - I can not use a list of known variants, because such a list does not exist for my species. Therefore I wanted to know, how important is to recalibrate QS, because thechinally it is possible, but not just by running one more step in GATK pipeline.
Answers.UserDisplayName: Kamil S Jaron
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-06T12:10:27.427Z
Answers.Text: I agree that it is non-model organism and that is why you will not have any such list of variants. But since the motivation of the approach was also asking about the BQSR so I said. You can take a look at this https://media.readthedocs.org/pdf/lts-workflows-sm-non-model-toolkit/latest/lts-workflows-sm-non-model-toolkit.pdf about how to use your HC variants of your sample to recalibrate. Also this gatlk link might help as well. http://gatkforums.broadinstitute.org/gatk/discussion/3286/quality-score-recalibration-for-non-model-organisms. Now the decision lies on your hand as to use or compare.
Answers.UserDisplayName: ivivek_ngs
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-06T12:14:06.190Z
Answers.Text: @KamilSJaron for non-model this is one way to do but if your sequencing machine is pretty new and improved accuracy you might also do away with the step. I would read publications to see what they do but still for my sake do calls without BQSR and with BQSR with HC variants and use them as database and compare to come to conclusion myself. Thats my opinion. It also depends on the virtue of the project as well.
Answers.UserDisplayName: ivivek_ngs
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-06T12:55:12.923Z
Answers.Text: The first link is very relevant to my original question: "While GATK UnifiedGenotyper suffers during indel calling without recalibration and realignment, both HaplotypeCaller and FreeBayes perform as good or better without these steps.". Thanks. The second link is relevant as well, but I do not have enough individuals sequenced to chose their approach for recalibration.
Answers.UserDisplayName: Kamil S Jaron
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-06T13:28:34.457Z
Answers.Text: @KamilSJaron I am glad it is relevant but then again I would be saying that you can run both with and without and make some estimation. Since you have not much samples to create your own HC SNP database you can also do it with strict stringent SNPs from your individuals. Or since you do not have much samples just avoid the BQSR step and pull out top variants and not large fraction of variants. The top variants even though the score might not very accurate but the calls will be still high confident and preferably true positives. I guess depends on number of variants you stream down to.
Answers.UserDisplayName: ivivek_ngs
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-06T17:54:59.417Z
Answers.Text: BQSR is still relevant, one of the reasons for this is some new illumina hardware such as the NextSeq can only output binned Q scores, BQSR essentially "un-bins" the Q scores giving more granularity which has it's uses in deep somatic variant calling, http://gatkforums.broadinstitute.org/gatk/discussion/4594/beware-of-using-binned-quality-scores-with-some-gatk-procedures.  Additionally the NextSeq suffers from a high confidence poly G issue, BQSR will be beneficial here too https://sequencing.qcfail.com/articles/illumina-2-colour-chemistry-can-overcall-high-confidence-g-bases/
Answers.UserDisplayName: Matt Bashton
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-07T14:50:45.033Z
Answers.Text: Ah now that was something I was unaware since I have not yet had to encounter with NextSeq. This is a good point. Then the OP should also think of it and yes I was merely considering how the BQSR in first came into picture. But this is a good catch for binned Q scores and about the poly G issue.
Answers.UserDisplayName: ivivek_ngs
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2017-06-06T11:07:55.663Z
Answers.LastActivityDate: 2019-04-19T07:48:46.800Z
Answers.LastEditDate: 2019-04-19T07:48:46.800Z
Answers.UserDisplayName: ivivek_ngs
Answers.Body: <p>In case BQSR is not an option (i.e. non-model organisms) it would be best to use some internal control sequence such as PhiX for illumina platform. Although this is supposed to be common practice some facilities ignore it. In principle, the machines should use these sequences as reference so that the scoring would be more accurate. In my experience the first 10-15 bases of the illumina reads always had lower quality. This can easily be seen in the nucleotide distributions. I would advise trimming the first 10-15 bases and quality based end-trimmingiIf the quality of the individual reads are important, such as low coverage resequencing or de-novo genome assembly applications.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-22T10:32:17.390Z
Answers.LastActivityDate: 2020-01-22T11:10:16.963Z
Answers.LastEditDate: 2020-01-22T11:10:16.963Z
Answers.UserDisplayName: Bekir Ergüner
Body: <p>The most variant calling pipeline GATK include a <a href="https://software.broadinstitute.org/gatk/documentation/article?id=44" rel="nofollow noreferrer">Base Quality Score Recalibration (BQSR)</a> which requires a list of known variants. Recently, some work has been done for reference-free recalibration of scores as well: <a href="http://biorxiv.org/content/early/2017/04/27/130732" rel="nofollow noreferrer">Lacer</a> and <a href="http://www.genetics.org/content/early/2016/11/07/genetics.116.189985" rel="nofollow noreferrer">atlas</a>, which is motivated by making the most for aDNA and low coverage datasets. </p>
  
  <p>The importance for aDNA is explained in <a href="https://www.youtube.com/watch?v=oJ9pbQsyaUg&amp;list=PLoCxWrRWjqB1JUCntl4X09ezmOtKx1Gke&amp;index=4" rel="nofollow noreferrer">this lecture</a>, but it is not clear to me if / how is important BQSR is for fresh DNA samples with decent (>15x) coverage. Especially when I work with non-model organisms and I can not simply use the standard tools.</p>
  
  <p>How big an impact does recalibration of scores have on variant calling? Is there a rule of thumb for which it is / it is not worth the effort?</p>
  
Tags: <ngs><variant-calling>
Title: Is there a point in recalibration of scores for variant calling?
ThreadText: # Is there a point in recalibration of scores for variant calling?
  
  (Asked by: username_0 on 2017-05-31)
  
  The most variant calling pipeline GATK include a [Base Quality Score Recalibration (BQSR)](https://software.broadinstitute.org/gatk/documentation/article?id=44) which requires a list of known variants. Recently, some work has been done for reference-free recalibration of scores as well: [Lacer](http://biorxiv.org/content/early/2017/04/27/130732) and [atlas](http://www.genetics.org/content/early/2016/11/07/genetics.116.189985), which is motivated by making the most for aDNA and low coverage datasets.
  
  The importance for aDNA is explained in [this lecture](https://www.youtube.com/watch?v=oJ9pbQsyaUg&list=PLoCxWrRWjqB1JUCntl4X09ezmOtKx1Gke&index=4), but it is not clear to me if / how is important BQSR is for fresh DNA samples with decent (>15x) coverage. Especially when I work with non-model organisms and I can not simply use the standard tools.
  
  How big an impact does recalibration of scores have on variant calling? Is there a rule of thumb for which it is / it is not worth the effort?
  
  **Tags:** ngs, variant-calling
  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_1 on 2017-06-01. Score: 5
  
  I personally don't think BQSR has a huge impact on variant calling, but you don't really need to guess. If you run GATK BQSR, it outputs a table and charts of exactly how much quality scores are adjusted. The adjustment will vary depending on the position in the read and genomic context (previous and following base). In my experience, the difference is a few points at most, but it's certainly noticeable.
  
  GATK recommends BQSR for both genome and exome data, which is normally much higher than 15x.
  
  
  ---
  
  ### Answer by username_2 on 2017-05-31. Score: 2
  
  That's a good question.
  
  I'd say that you don't need to bother with variant recalibration for
  
  * low number of samples (e.g., just two trios); I could not get GTAK recalibration of variant scores to work anyway
  * high-coverage samples (e.g., X Ten genomes with 30x coverage) where the DNA samples themselves are of high, comparable quality and have been sequenced with consistent technology.
  
  Generally, it is my impression that a lot of the thoughts and advanced statistical models built into GATK come from the earlier phases of the 1000 Genomes project. This means (1) low-coverage, (2) different coverage genomes (3) sequenced with varying technology versions by (4) different samples and (5) population sequencing.
  
  If you are in a clinical setting where you do 30x sequencing on X Ten platforms only anyway, then variant recalibration will probably not help you that much.
  
  On the other hand, if you are integrating many data sets from different data centers and machine versions etc., variant recalibration might be worth a shot.
  
  A good check would be looking at genotype quality distributions and other variant/quality related metrics before and after recalibration.
  
  Anyone: please correct me if I'm wrong!
  
  > **Comments:**
  > **username_3**: Are you talking about base quality score recalibration (BQSR) here or about  variant quality score recalibration (VQSR)? I think the OP is referring to BQSR but you are discussing VQSR.  
  > **username_3**: Yes, OP confirmed. The question is about BQSR so I am afraid you're answering the wrong question.  
  > **username_2**: *sigh* and there I thought I could contribute something.  
  
  ---
  
  ### Answer by username_4 on 2017-06-06. Score: 1
  
  Ideally these BQSR methods were made keeping in mind how technical errors will actually screw up the base quality calls and when the machines were still more on development phase while being used for the 1000G project. As of now machines are more powerful and strong where it will be unlikely to use it but still we use with listed SNPs to find the covariates and build a model around the data using the information with machine learning tricks to improve the quality of those base calls. Ideally it should be more appropriate when old machines from Illumina or other standard companies are being used but with new machines which are much powerful and having high throughput they should tend to go down. I do not recall if such tests have been made but obviously I know new sequencing machine always make such tests to show that they have reduced such errors but still recommend such BQSR for variant calls. Now the problem is the list of SNPs, this to me is the real problem since the list we use is far from being Gold standard and if that is not properly taken care of everything we infer about quality is still shaky. [This link](https://gatkforums.broadinstitute.org/gatk/discussion/44/base-quality-score-recalibration-bqsr) is pretty informative but it's an old one. I would really see improvements with new sequencers. However very less people care about such tests in academic research and also translational lab will really not invest time and money on such unless the facility has some bioinformaticians who always does such testing while buying a new sequencer for the institute. In terms of clinical genomics for finding variants I reckon most powerful and up-to-date sequencers should be used but not sure if they still use BQSR and if so what is the list they use to build model of covariation around the data.
  
  > **Comments:**
  > **username_0**: Note that the question is motivated by non-model organism research - I can not use a list of known variants, because such a list does not exist for my species. Therefore I wanted to know, how important is to recalibrate QS, because thechinally it is possible, but not just by running one more step in GATK pipeline.  
  > **username_4**: I agree that it is non-model organism and that is why you will not have any such list of variants. But since the motivation of the approach was also asking about the BQSR so I said. You can take a look at this https://media.readthedocs.org/pdf/lts-workflows-sm-non-model-toolkit/latest/lts-workflows-sm-non-model-toolkit.pdf about how to use your HC variants of your sample to recalibrate. Also this gatlk link might help as well. http://gatkforums.broadinstitute.org/gatk/discussion/3286/quality-score-recalibration-for-non-model-organisms. Now the decision lies on your hand as to use or compare.  
  > **username_4**: @username_0 for non-model this is one way to do but if your sequencing machine is pretty new and improved accuracy you might also do away with the step. I would read publications to see what they do but still for my sake do calls without BQSR and with BQSR with HC variants and use them as database and compare to come to conclusion myself. Thats my opinion. It also depends on the virtue of the project as well.  
  > **username_0**: The first link is very relevant to my original question: "While GATK UnifiedGenotyper suffers during indel calling without recalibration and realignment, both HaplotypeCaller and FreeBayes perform as good or better without these steps.". Thanks. The second link is relevant as well, but I do not have enough individuals sequenced to chose their approach for recalibration.  
  > **username_4**: @username_0 I am glad it is relevant but then again I would be saying that you can run both with and without and make some estimation. Since you have not much samples to create your own HC SNP database you can also do it with strict stringent SNPs from your individuals. Or since you do not have much samples just avoid the BQSR step and pull out top variants and not large fraction of variants. The top variants even though the score might not very accurate but the calls will be still high confident and preferably true positives. I guess depends on number of variants you stream down to.  
  > **username_5**: BQSR is still relevant, one of the reasons for this is some new illumina hardware such as the NextSeq can only output binned Q scores, BQSR essentially "un-bins" the Q scores giving more granularity which has it's uses in deep somatic variant calling, http://gatkforums.broadinstitute.org/gatk/discussion/4594/beware-of-using-binned-quality-scores-with-some-gatk-procedures.  Additionally the NextSeq suffers from a high confidence poly G issue, BQSR will be beneficial here too https://sequencing.qcfail.com/articles/illumina-2-colour-chemistry-can-overcall-high-confidence-g-bases/  
  > **username_4**: Ah now that was something I was unaware since I have not yet had to encounter with NextSeq. This is a good point. Then the OP should also think of it and yes I was merely considering how the BQSR in first came into picture. But this is a good catch for binned Q scores and about the poly G issue.  
  
  ---
  
  ### Answer by username_6 on 2020-01-22. Score: 0
  
  In case BQSR is not an option (i.e. non-model organisms) it would be best to use some internal control sequence such as PhiX for illumina platform. Although this is supposed to be common practice some facilities ignore it. In principle, the machines should use these sequences as reference so that the scoring would be more accurate. In my experience the first 10-15 bases of the illumina reads always had lower quality. This can easily be seen in the nucleotide distributions. I would advise trimming the first 10-15 bases and quality based end-trimmingiIf the quality of the individual reads are important, such as low coverage resequencing or de-novo genome assembly applications.
  
  
  ---
  
  