Answers.Body: <p>Most duplicate marking programs do it by looking at the start and end location of hits. In this case, if two reads have identical start and end locations for both reads, then they will be considered [PCR] duplicates of each other. When running through a duplicate filter, you will only get one read for each unique pair.</p>
  
  <p>This is much more likely for targeted sequencing, because the length covered by the template is much smaller. It's also more likely that you actually <em>do</em> have lots of PCR duplicates for the same reason.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-02-15T10:23:45.743Z
Answers.LastActivityDate: 2019-02-15T10:23:45.743Z
Answers.UserDisplayName: gringer
Answers.Body: <p>If you're doing target capture, especially of a small region, it is entirely possible that there are only five unique reads. Target capture is molecularly finicky and sometimes the entire region is not captured.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-02-14T06:31:30.423Z
Answers.LastActivityDate: 2019-02-14T06:31:30.423Z
Answers.UserDisplayName: conchoecia
Body: <p>Currently I'm performing whole analysis (pipeline from *.fastq to *.vcf) of 41  samples (targeted NGS). I rely on GATK best practices, however with some modifications. I decided to use the following tools:</p>
  <pre><code>#mapping
  bwa mem (with mem - alternate haplotypes)
  
  #preprocessing
  RevertSam (Picard)
  MergeBamAlignment (Picard)
  MarkDuplicates (Picard)
  BaseRecalibrator (GATK)
  ApplyBQSR (GATK)
  AnalyzeCovariates (GATK)
  SortSam (Picard)
  
  #call, filter and annotate variants
  HaplotypeCaller (GATK)
  GenotypeGVCFs (GATK)
  snpEff, SnpSift
  </code></pre>
  <p>Nevertheless, after marking the duplicates with <code>MarkDuplicates</code> about 99% of reads have been marked as duplicated, hence the coverage decreased from several hundreds to literally up to five. So I decided to omit this step.</p>
  <p>Have you ever encountered a similar problem? What is your opinion about this?</p>
  <p>Best Regards,
  Adam</p>
  <h1>EDIT</h1>
  <p>The Agilent Haloplex custom target enrichment kit (HaloPlex Target Enrichment for Illumina Kit) has been used to prepare the libraries, sequencing done using Illumina HiSeq 2500. Targeted NGS of all exons (only) of 90 genes.</p>
  <p>Here I provide how the final bam look like. This is the result without duplicate marking, otherwise there're only several reads per exon...
  <a href="https://i.sstatic.net/k8yze.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/k8yze.png" alt="enter image description here" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-02-14T06:32:03.533Z
Comments.Text: can you clarify what you mean by targeted NGS? What are you targeting/capturing and how were the sequencing libraries prepared?
Comments.UserDisplayName: conchoecia
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-02-14T07:43:45.767Z
Comments.Text: I believe, using haloplex target capture, you expect reads to have identical begin and end coordinates, so imho in this case you should skip duplicate marking/removal.
Comments.UserDisplayName: Wouter De Coster
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-02-14T12:39:17.783Z
Comments.Text: Thanks for help! By the way, what do you think about GATK as a tool for such analysis. For me it's nice but a bit too complicated.
Comments.UserDisplayName: Adamm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-02-15T05:55:48.640Z
Comments.Text: Indeed after `MarkDuplicates` the coverage was up to 4. It was hard to distinguish between actual alternations and mismatched bases.
Comments.UserDisplayName: Adamm
Tags: <gatk><markduplicate>
Title: Targeted NGS, up to 99% of reads have been marked as duplicates
ThreadText: # Targeted NGS, up to 99% of reads have been marked as duplicates
  
  (Asked by: username_0 on 2019-02-14)
  
  Currently I'm performing whole analysis (pipeline from \*.fastq to \*.vcf) of 41 samples (targeted NGS). I rely on GATK best practices, however with some modifications. I decided to use the following tools:
  
  ```
  #mapping
  bwa mem (with mem - alternate haplotypes)
  
  #preprocessing
  RevertSam (Picard)
  MergeBamAlignment (Picard)
  MarkDuplicates (Picard)
  BaseRecalibrator (GATK)
  ApplyBQSR (GATK)
  AnalyzeCovariates (GATK)
  SortSam (Picard)
  
  #call, filter and annotate variants
  HaplotypeCaller (GATK)
  GenotypeGVCFs (GATK)
  snpEff, SnpSift
  
  ```
  
  Nevertheless, after marking the duplicates with `MarkDuplicates` about 99% of reads have been marked as duplicated, hence the coverage decreased from several hundreds to literally up to five. So I decided to omit this step.
  
  Have you ever encountered a similar problem? What is your opinion about this?
  
  Best Regards,
  Adam
  
  # EDIT
  
  The Agilent Haloplex custom target enrichment kit (HaloPlex Target Enrichment for Illumina Kit) has been used to prepare the libraries, sequencing done using Illumina HiSeq 2500. Targeted NGS of all exons (only) of 90 genes.
  
  Here I provide how the final bam look like. This is the result without duplicate marking, otherwise there're only several reads per exon...
  [![enter image description here](https://i.sstatic.net/k8yze.png)](https://i.sstatic.net/k8yze.png)
  
  **Tags:** gatk, markduplicate
  
  > **Comments:**
  > **username_1**: can you clarify what you mean by targeted NGS? What are you targeting/capturing and how were the sequencing libraries prepared?  
  > **username_2**: I believe, using haloplex target capture, you expect reads to have identical begin and end coordinates, so imho in this case you should skip duplicate marking/removal.  
  > **username_0**: Thanks for help! By the way, what do you think about GATK as a tool for such analysis. For me it's nice but a bit too complicated.  
  > **username_0**: Indeed after `MarkDuplicates` the coverage was up to 4. It was hard to distinguish between actual alternations and mismatched bases.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2019-02-15. Score: 1 | Accepted answer
  
  Most duplicate marking programs do it by looking at the start and end location of hits. In this case, if two reads have identical start and end locations for both reads, then they will be considered [PCR] duplicates of each other. When running through a duplicate filter, you will only get one read for each unique pair.
  
  This is much more likely for targeted sequencing, because the length covered by the template is much smaller. It's also more likely that you actually *do* have lots of PCR duplicates for the same reason.
  
  
  ---
  
  ### Answer by username_1 on 2019-02-14. Score: 3
  
  If you're doing target capture, especially of a small region, it is entirely possible that there are only five unique reads. Target capture is molecularly finicky and sometimes the entire region is not captured.
  
  
  ---
  
  