Answers.Body: <p>If you are sequencing same library multiple times (as you state in the question), any read that appears in  multiple sequencing runs is likely to have been generated via PCR duplication from the same original biological molecule, and therefore needs to be removed. Indeed, the chance of the read being a duplicate is the same if it was sequenced twice in the same lane, or it was sequenced in two different lanes, as long as it comes from the same library.</p>
  <p>Thus, in this situation, you should either:</p>
  <ol>
  <li>Merge the fastqs, map and MarkDuplicates or</li>
  <li>Map the separate fastqs, merge the BAM files, and then MarkDuplicates.</li>
  </ol>
  <p>You should definitely merge before doing either MarkDuplicates or peak calling.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-29T09:05:48.753Z
Answers.Text: Thanks for your reply, it makes sense. However do you think is possible to quantify this "likely to have been generated via PCR duplication". Like how much more likely?
Answers.UserDisplayName: Ni-Ar
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-29T10:29:54.673Z
Answers.Text: It isn't possible, not without specially designed experiments. But the important point is that irrespective of the likelihood that a duplicate is a "biological" duplicate or a "PCR" duplicate, that probability is the same between sequencing runs as it is within a sequencing run. If it doesn't make sense to deduplicate the merged data, it wouldn't make sense to deduplicate the individual files either.
Answers.UserDisplayName: Ian Sudbery
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-29T08:50:42.197Z
Answers.LastActivityDate: 2021-06-29T08:50:42.197Z
Answers.UserDisplayName: Ian Sudbery
Body: <p>I'd like to a follow up question to <a href="https://bioinformatics.stackexchange.com/q/414/12652">this question</a> related to merging fastq files for ChIP-seq.</p>
  <p>Let's assume we have one sample library that is re-sequenced two or three times in order to achieve a desired read coverage. The resulting output of the sequencing is 2 or 3 fastq files for one individual sample.</p>
  <p>If one has to mark duplicates (for example using <a href="https://gatk.broadinstitute.org/hc/en-us/articles/360037052812-MarkDuplicates-Picard-" rel="nofollow noreferrer">Picard's MarkDuplicates</a>) should the sub-samples be merged at the fastq level or at the bam file level (post alignment) after flagging duplicates before the merge?</p>
  <p>If one takes the 2-3 sub-fastq files align them and then mark the duplicates in the bam files if the sequencing is shallow one could end up with not calling reads that are actually PCR duplicates of the library preparation. While in the other scenario, a fastq merge might result in reads to be marked as &quot;duplicates&quot; just because they appear 2-3 times, but maybe that read is a single reads that I sequenced 2-3 times, individually in each run.</p>
  <p>My point is that I would expect this 2 different approached to lead to different outcomes. Probably this is a negligible effect in a overall analysis, but maybe it might actually have an impact in certain biological conditions (I'm thinking of heterochromatin silencing of repeat elements in the genome). I'm going to try it out as soon as I have a change, but I was wondering if this was discussed already somewhere else (I couldn't find much relevant) or if someone has some ideas on how to tackle this.</p>
  <p>Let's assume that the reads are 50bp single end.</p>
  
Tags: <sam><fastq><chip-seq><merge><markduplicate>
Title: Impact of merging ChIP-seq runs of the same sample on PCR duplicates identification?
ThreadText: # Impact of merging ChIP-seq runs of the same sample on PCR duplicates identification?
  
  (Asked by: username_0 on 2021-06-29)
  
  I'd like to a follow up question to [this question](https://bioinformatics.stackexchange.com/q/414/12652) related to merging fastq files for ChIP-seq.
  
  Let's assume we have one sample library that is re-sequenced two or three times in order to achieve a desired read coverage. The resulting output of the sequencing is 2 or 3 fastq files for one individual sample.
  
  If one has to mark duplicates (for example using [Picard's MarkDuplicates](https://gatk.broadinstitute.org/hc/en-us/articles/360037052812-MarkDuplicates-Picard-)) should the sub-samples be merged at the fastq level or at the bam file level (post alignment) after flagging duplicates before the merge?
  
  If one takes the 2-3 sub-fastq files align them and then mark the duplicates in the bam files if the sequencing is shallow one could end up with not calling reads that are actually PCR duplicates of the library preparation. While in the other scenario, a fastq merge might result in reads to be marked as "duplicates" just because they appear 2-3 times, but maybe that read is a single reads that I sequenced 2-3 times, individually in each run.
  
  My point is that I would expect this 2 different approached to lead to different outcomes. Probably this is a negligible effect in a overall analysis, but maybe it might actually have an impact in certain biological conditions (I'm thinking of heterochromatin silencing of repeat elements in the genome). I'm going to try it out as soon as I have a change, but I was wondering if this was discussed already somewhere else (I couldn't find much relevant) or if someone has some ideas on how to tackle this.
  
  Let's assume that the reads are 50bp single end.
  
  **Tags:** sam, fastq, chip-seq, merge, markduplicate
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-06-29. Score: 1
  
  If you are sequencing same library multiple times (as you state in the question), any read that appears in multiple sequencing runs is likely to have been generated via PCR duplication from the same original biological molecule, and therefore needs to be removed. Indeed, the chance of the read being a duplicate is the same if it was sequenced twice in the same lane, or it was sequenced in two different lanes, as long as it comes from the same library.
  
  Thus, in this situation, you should either:
  
  1. Merge the fastqs, map and MarkDuplicates or
  2. Map the separate fastqs, merge the BAM files, and then MarkDuplicates.
  
  You should definitely merge before doing either MarkDuplicates or peak calling.
  
  > **Comments:**
  > **username_0**: Thanks for your reply, it makes sense. However do you think is possible to quantify this "likely to have been generated via PCR duplication". Like how much more likely?  
  > **username_1**: It isn't possible, not without specially designed experiments. But the important point is that irrespective of the likelihood that a duplicate is a "biological" duplicate or a "PCR" duplicate, that probability is the same between sequencing runs as it is within a sequencing run. If it doesn't make sense to deduplicate the merged data, it wouldn't make sense to deduplicate the individual files either.  
  
  ---
  
  