Answers.Body: <p>Alternatively, I really like using bbduk which is part of the <a href="https://github.com/BioInfoTools/BBMap" rel="nofollow noreferrer">BBMap suite</a>.</p>
  
  <p>I've processed every nascent sequencing dataset that has been published, and found a lot of quirky errors with older datasets using TrimGalore. bbduk is a little more fine-tuneable relative to cutadapt/trimmomatic/trimGalore (built on top of cutadapt)/fastp and the run time and memory usage is significantly less.</p>
  
  <p>When you clone/install the repository, there is a directory of adapter fa files that you can choose from for trimming in the resources/ subdirectory which you can specify using the "ref" argument (i.e. ref=adapters.fa). The "adapters.fa" file is an excellent catchall for adapters, but you will likely over-trim a little. Alternatively, if you know what adapters were used in your dataset, you can choose one of the other pre-configured fa files. There are also a number of stats/marking/reporting/trimmed read save options which you can see here:</p>
  
  <p><a href="https://github.com/BioInfoTools/BBMap/blob/master/sh/bbduk.sh" rel="nofollow noreferrer">https://github.com/BioInfoTools/BBMap/blob/master/sh/bbduk.sh</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-22T16:52:34.170Z
Answers.LastActivityDate: 2019-03-22T17:33:13.610Z
Answers.LastEditDate: 2019-03-22T17:33:13.610Z
Answers.UserDisplayName: Margaret Gruca
Answers.Body: <p>You're best off just using <a href="https://github.com/OpenGene/fastp" rel="nofollow noreferrer"><code>fastp</code></a> or <a href="https://www.bioinformatics.babraham.ac.uk/projects/trim_galore/" rel="nofollow noreferrer"><code>Trim Galore!</code></a>, both of which will determine the adapter sequence for you. Trim Galore! uses a built-in list of known sequences for this, whereas fastp uses read overlap.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-19T21:28:34.337Z
Answers.Text: For clarity, are you suggesting it's better to switch over completely to this recommended set of tools, or are you simply introducing the possibility of using fastp to determine the adapters (or extract the list from trim galore)? 
  
  One benefit of using MarkIlluminaAdapters is that it simply "marks" the sequences using `XT:i:` SAM trimming tag, rather than removing them altogether from the output bam/sam.
Answers.UserDisplayName: init_js
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-20T06:13:26.180Z
Answers.Text: I see no benefit to using MarkIlluminaAdapters, replace it completely.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-21T17:13:19.997Z
Answers.Text: This [article](https://software.broadinstitute.org/gatk/documentation/article.php?id=6483) describes a "lossless" alignment pipeline which allows producing aligned bams which can be reversed back to the original fastqs. This is possible because of those trimming annotations, which annotate, rather than outright clip, the original sequence to indicate the portion of the read with adapters. I wish fastp supported these SAM trimming annotations, it would be a strict improvement.
Answers.UserDisplayName: init_js
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-21T17:17:00.850Z
Answers.Text: If you want that just use an aligner with soft clipping. Note also that few aligners support SAM input.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-21T18:24:47.437Z
Answers.Text: Lol, I think I'll start calling mappers "read slingers"  To be serious though, it's *possible* for a conscientious bioinformatician to store reads in SAM (et al.) format at every stage (raw unaligned, QC'd, aligned, aligned post-processed, etc.) in a way that preserves the original raw sequence unambiguously. In practice though, it will be problematic since many tools are not designed with this paradigm in mind. It would take years to get everyone on the same page, if indeed it's even possible to do so.
Answers.UserDisplayName: Daniel Standage
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-21T18:25:21.477Z
Answers.Text: Lol, stupid autocorrect on my phone.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-21T19:01:50.043Z
Answers.Text: re: aligners not supporting SAM. small note. The workflow I mentioned above will temporarily transform the unaligned SAM (with `XT:i` adapter trimming annotations) into fastq that has adapters hard-clipped. The mapper is run on that clipped fastq (like you say, because most of them only support fastq). And then this tool, MergeBamAlignment, combines the mapping information from the aligned BAM into the full, soft-clipped, reads in the unaligned SAM. It might be straighforward to compute the `XT:i` annotations by comparing the fastp input/output , but I haven't fiddled with it. I'll consider.
Answers.UserDisplayName: init_js
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-19T08:46:31.690Z
Answers.LastActivityDate: 2019-03-19T14:56:20.830Z
Answers.LastEditDate: 2019-03-19T14:56:20.830Z
Answers.UserDisplayName: Devon Ryan
Body: <p>I would like to trim/mark adapters using trimmomatic or picard MarkIlluminaAdapters from a series of Illumina Paired-End read fastqs. The fastq files may have been done using different kits or different generations/versions of illumina chips.</p>
  
  <p>Without knowing ahead of time how the libraries were prepared (or which kit was used), Is there a safe "catch all" adapter sequence (or set of such sequences) that can be passed to a trimming tool and that would cover <em>all</em> Illumina kits? In other words, what are we expected to know about the fastq to ensure correct trimming?</p>
  
  <p><em>edit:</em> Picard defines a few built-in adapters which cover many standard kits, but it's not immediately obvious if you need to pass a common one (<a href="https://github.com/broadinstitute/picard/blob/1531278515aaf39c9049b5792ae569b92326dd90/src/main/java/picard/util/IlluminaUtil.java#L141" rel="nofollow noreferrer">built-in sequences defined here</a>).</p>
  
Tags: <illumina><picard><trimming>
Title: Is there a safe catch-all adapter sequence for trimming?
ThreadText: # Is there a safe catch-all adapter sequence for trimming?
  
  (Asked by: username_0 on 2019-03-19)
  
  I would like to trim/mark adapters using trimmomatic or picard MarkIlluminaAdapters from a series of Illumina Paired-End read fastqs. The fastq files may have been done using different kits or different generations/versions of illumina chips.
  
  Without knowing ahead of time how the libraries were prepared (or which kit was used), Is there a safe "catch all" adapter sequence (or set of such sequences) that can be passed to a trimming tool and that would cover *all* Illumina kits? In other words, what are we expected to know about the fastq to ensure correct trimming?
  
  *edit:* Picard defines a few built-in adapters which cover many standard kits, but it's not immediately obvious if you need to pass a common one ([built-in sequences defined here](https://github.com/broadinstitute/picard/blob/1531278515aaf39c9049b5792ae569b92326dd90/src/main/java/picard/util/IlluminaUtil.java#L141)).
  
  **Tags:** illumina, picard, trimming
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2019-03-22. Score: 2
  
  Alternatively, I really like using bbduk which is part of the [BBMap suite](https://github.com/BioInfoTools/BBMap).
  
  I've processed every nascent sequencing dataset that has been published, and found a lot of quirky errors with older datasets using TrimGalore. bbduk is a little more fine-tuneable relative to cutadapt/trimmomatic/trimGalore (built on top of cutadapt)/fastp and the run time and memory usage is significantly less.
  
  When you clone/install the repository, there is a directory of adapter fa files that you can choose from for trimming in the resources/ subdirectory which you can specify using the "ref" argument (i.e. ref=adapters.fa). The "adapters.fa" file is an excellent catchall for adapters, but you will likely over-trim a little. Alternatively, if you know what adapters were used in your dataset, you can choose one of the other pre-configured fa files. There are also a number of stats/marking/reporting/trimmed read save options which you can see here:
  
  <https://github.com/BioInfoTools/BBMap/blob/master/sh/bbduk.sh>
  
  
  ---
  
  ### Answer by username_2 on 2019-03-19. Score: 2
  
  You're best off just using [`fastp`](https://github.com/OpenGene/fastp) or [`Trim Galore!`](https://www.bioinformatics.babraham.ac.uk/projects/trim_galore/), both of which will determine the adapter sequence for you. Trim Galore! uses a built-in list of known sequences for this, whereas fastp uses read overlap.
  
  > **Comments:**
  > **username_0**: For clarity, are you suggesting it's better to switch over completely to this recommended set of tools, or are you simply introducing the possibility of using fastp to determine the adapters (or extract the list from trim galore)?   One benefit of using MarkIlluminaAdapters is that it simply "marks" the sequences using `XT:i:` SAM trimming tag, rather than removing them altogether from the output bam/sam.  
  > **username_2**: I see no benefit to using MarkIlluminaAdapters, replace it completely.  
  > **username_0**: This [article](https://software.broadinstitute.org/gatk/documentation/article.php?id=6483) describes a "lossless" alignment pipeline which allows producing aligned bams which can be reversed back to the original fastqs. This is possible because of those trimming annotations, which annotate, rather than outright clip, the original sequence to indicate the portion of the read with adapters. I wish fastp supported these SAM trimming annotations, it would be a strict improvement.  
  > **username_2**: If you want that just use an aligner with soft clipping. Note also that few aligners support SAM input.  
  > **username_3**: Lol, I think I'll start calling mappers "read slingers"  To be serious though, it's *possible* for a conscientious bioinformatician to store reads in SAM (et al.) format at every stage (raw unaligned, QC'd, aligned, aligned post-processed, etc.) in a way that preserves the original raw sequence unambiguously. In practice though, it will be problematic since many tools are not designed with this paradigm in mind. It would take years to get everyone on the same page, if indeed it's even possible to do so.  
  > **username_2**: Lol, stupid autocorrect on my phone.  
  > **username_0**: re: aligners not supporting SAM. small note. The workflow I mentioned above will temporarily transform the unaligned SAM (with `XT:i` adapter trimming annotations) into fastq that has adapters hard-clipped. The mapper is run on that clipped fastq (like you say, because most of them only support fastq). And then this tool, MergeBamAlignment, combines the mapping information from the aligned BAM into the full, soft-clipped, reads in the unaligned SAM. It might be straighforward to compute the `XT:i` annotations by comparing the fastp input/output , but I haven't fiddled with it. I'll consider.  
  
  ---
  
  