Answers.Body: <p>On a regular basis, using FastQC is quite enough as to assess the quality of FastQ-formatted data. It gives you plenty of details as can be seen in the screenshots you shared.</p>
  <p>As others have commented  or suggested, there is  nothing wrong with having more 'yellow' warnings after processing the FASTQ files using fastp. At lest not as long as what you observe in the data looks fine and the warning has some explanation.</p>
  <p>The sequence length distribution warning is very often present post-processing since at the begging all your reads are of the same length but this is not the case any more after you trim the adapters and the low quality bases at the end of the read for example. You have to go to the plots specific to that section and make sure all looks fine. It should be the case.</p>
  <p>The GC sequence content warning. It depends a lot what organism you are sequencing, which regions of the genomes, etc. Importantly, what is the expected GC content of those regions. So you can compare to these observed numbers. Once again the warning can be easily triggered after the processing, since for a start Illumina sequencing has known GC bias</p>
  <p>Finally, I think actually your reads look better  after the processing with fastp. You are showing us the per base sequence quality and before, there are many reads with bases on the yellow and even the red sections of the quality scale. But this is not the case any more after the processing !</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-10T10:37:55.857Z
Answers.Text: Thanks. I still suspect a tool like fasqc, pair ended aware, will give better results.
Answers.UserDisplayName: gl00ten
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-10T16:11:39.380Z
Answers.Text: You're wrong about fastqc being PE aware as well as fastp not being PE aware. Are you even reading the manuals, @fullmooninu?
Answers.UserDisplayName: Ram RS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-10T17:31:44.513Z
Answers.Text: I am always amazed by users new to the field and then calling out on of the most extensively used tools in all of bioinformatics. There is nothing wrong with your data in the first place. Quality does not go down after trimming as seen in ghe per-base boxplots, the length warning is expected due after any trim and GC content is just deviation from a theoretical distribution. There is no basis to call results worse or regarding fastqc as not accurate.
Answers.UserDisplayName: ATpoint
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-16T15:03:16.410Z
Answers.Text: @RamRS i didn't find out that in the manual. can you point it out for me, please?
Answers.UserDisplayName: gl00ten
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-18T17:57:44.410Z
Answers.Text: @fullmooninu Literally in the second usage example here: https://github.com/OpenGene/fastp#simple-usage
Answers.UserDisplayName: Ram RS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-18T18:01:38.867Z
Answers.Text: And it is common knowledge that fastqc analyzes FASTQ files independently. It has no options for PE, you can check using `fastqc --help 2>&1 | grep -ic pair`: This will give you `0`.
Answers.UserDisplayName: Ram RS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-23T18:12:08.483Z
Answers.Text: @RamRS I appreciate you trying to be helpful. But I'm literally complaining about fastqc not being PE aware, unlike fastp. And am looking for a replacement for fasqc. please delete your comments to decrease entropy.
Answers.UserDisplayName: gl00ten
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-23T19:15:54.617Z
Answers.Text: @fullmooninu in that case, you might want to edit your question since you state "*fastp doesn't analyse paired reads files together*" which is presumably why Ram linked you to the fastp manual which shows that it can actually process paired end reads together.
Answers.UserDisplayName: terdon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-23T19:30:26.720Z
Answers.Text: @fullmooninu I recognise a lot of time has been invested here and it's a great question. Short-read QC is not my area, however as general advice sometimes different viewpoints may not be helpful in the short term. However, in the longterm a different perspective may help you. Ram RS is a very experienced bioinformatician, but simply Ram RS's  approach is direct and this may cause unintentional offence if there's a misunderstanding in the question,  cultures or possibly issues such pressure to complete a project (which we all understand).
Answers.UserDisplayName: M__
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-23T22:27:19.583Z
Answers.Text: @fullmooninu you said "*I still suspect a tool like fasqc, pair ended aware, will give better results*".  Now you say you're complaining about it not being PE aware. Am I missing something? And yes my tone is quite blunt so I apologize for that.
Answers.UserDisplayName: Ram RS
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-05T18:58:11.950Z
Answers.Text: @terdon corrected, thank you. To the others. Repeating: i'm looking for something to validate fastp's work. Probably like fasqc but that suppports PE. Having more warnings is not a good sign.
Answers.UserDisplayName: gl00ten
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-10T09:42:16.683Z
Answers.LastActivityDate: 2023-01-11T09:48:54.027Z
Answers.LastEditDate: 2023-01-11T09:48:54.027Z
Answers.UserDisplayName: JRodrigoF
Body: <p>In the beginning of my pipeline, I just fed the paired reads (2 files) into fastp, with the default options, and assumed it would do a good job preparing the reads for the next step:  alignment</p>
  <p>But I now checked the files in FastQC and it doesn't show much difference from before the fastp processing: Some extra yellow and red signals appear. I think the end result is even worse.</p>
  <p>fastqc doesn't analyse paired reads files together. So it might be wrong regarding the paired quality, taken as a whole.</p>
  <p>Anything else I should use together with FastQC, to verify the output quality?</p>
  <p>I'm aligning a bunch of poolseqs and then comparing them to find consistency in the change of snp frequencies</p>
  <p>As requested here is one of the examples that made me ask this:</p>
  <p><strong><em>Before fastp. only one problem noted by fastqc</em></strong></p>
  <p><a href="https://i.sstatic.net/8Fl4c.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/8Fl4c.png" alt="before fastp. only one problem noted by fastqc" /></a></p>
  <p><strong><em>After fastp. overall quality seems to have decreased</em></strong></p>
  <p><a href="https://i.sstatic.net/LPxDf.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/LPxDf.png" alt="after fastp. overall quality seems to have decreased" /></a></p>
  <p>The raw data is a pool seq from illumina.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-02T19:58:04.550Z
Comments.Text: Please edit your question and give more details about your data (genomic/RNA-seq, whole transcriptome/3' end) and which fastqc modules are showing problems.
Comments.UserDisplayName: Cloudberry
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-03T17:18:48.597Z
Comments.Text: Hi fullmooninu a screenshot would be a good start, particularly if it sheds light on the QC. Would it be sufficient ... I'm sure @Cloudberry will comment.
Comments.UserDisplayName: M__
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-03T18:43:47.290Z
Comments.Text: Please give more context around your problem. For example, what are you trying to do with your pipeline and fastp?
Comments.UserDisplayName: gringer
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-03T23:33:19.450Z
Comments.Text: updated the question with your requests.
Comments.UserDisplayName: gl00ten
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-04T03:49:29.160Z
Comments.Text: What makes you think that quality of reads went down after running fastp? You have two warnings after running fastp. One is GC content, and other is length distribution. Both of them can be easily explained by the removal of adapter and low quality reads. You should also read fastp manual and see if running the default is a good idea.
Comments.UserDisplayName: Supertech
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-04T08:15:51.060Z
Comments.Text: Two extra warnings is worse than no warnings, right? I saw it consistently over the tests I did: more warnings. Dont remember if I saw any reds. The question is, if fastqc output is not a good measure of fastp correctness, what is?
Comments.UserDisplayName: gl00ten
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-04T13:37:02.393Z
Comments.Text: The sequence length distribution warning is triggered if the reads have variable lengths, which you'd expect after adapter removal and/or quality trimming. I'm not familiar with fastp but apparently it tries to guess adapter sequences from the reads by default. If you know the adapter sequence, you'd probably get more accurate results by providing it to fastp. You didn't mention what type of sequencing library the data is from. If this is targeted sequencing, that could explain a mild bias in base composition.
Comments.UserDisplayName: Cloudberry
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-08T05:46:14.287Z
Comments.Text: edited. it's a pool-seq. i'm interested in validating what the cleaning process. fasqc doesnt seem to perform well, in this scenario.
Comments.UserDisplayName: gl00ten
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-09T19:13:20.920Z
Comments.Text: Tools like fastqc are useful in helping you determine the quality of your generated FASTQ files, but they are not the determining factor in whether your data is "good", or your experiment is a success. How does your data look when run through your analysis pipeline? That is the critical question.
Comments.UserDisplayName: Scot
Tags: <fastq><reads><fastqc><read-correction>
Title: If fastp output is not a good measure of FASTQ correctness, what is?
ThreadText: # If fastp output is not a good measure of FASTQ correctness, what is?
  
  (Asked by: username_0 on 2023-01-02)
  
  In the beginning of my pipeline, I just fed the paired reads (2 files) into fastp, with the default options, and assumed it would do a good job preparing the reads for the next step: alignment
  
  But I now checked the files in FastQC and it doesn't show much difference from before the fastp processing: Some extra yellow and red signals appear. I think the end result is even worse.
  
  fastqc doesn't analyse paired reads files together. So it might be wrong regarding the paired quality, taken as a whole.
  
  Anything else I should use together with FastQC, to verify the output quality?
  
  I'm aligning a bunch of poolseqs and then comparing them to find consistency in the change of snp frequencies
  
  As requested here is one of the examples that made me ask this:
  
  ***Before fastp. only one problem noted by fastqc***
  
  [![before fastp. only one problem noted by fastqc](https://i.sstatic.net/8Fl4c.png)](https://i.sstatic.net/8Fl4c.png)
  
  ***After fastp. overall quality seems to have decreased***
  
  [![after fastp. overall quality seems to have decreased](https://i.sstatic.net/LPxDf.png)](https://i.sstatic.net/LPxDf.png)
  
  The raw data is a pool seq from illumina.
  
  **Tags:** fastq, reads, fastqc, read-correction
  
  > **Comments:**
  > **username_1**: Please edit your question and give more details about your data (genomic/RNA-seq, whole transcriptome/3' end) and which fastqc modules are showing problems.  
  > **username_2**: Hi fullmooninu a screenshot would be a good start, particularly if it sheds light on the QC. Would it be sufficient ... I'm sure @username_1 will comment.  
  > **username_3**: Please give more context around your problem. For example, what are you trying to do with your pipeline and fastp?  
  > **username_0**: updated the question with your requests.  
  > **username_4**: What makes you think that quality of reads went down after running fastp? You have two warnings after running fastp. One is GC content, and other is length distribution. Both of them can be easily explained by the removal of adapter and low quality reads. You should also read fastp manual and see if running the default is a good idea.  
  > **username_0**: Two extra warnings is worse than no warnings, right? I saw it consistently over the tests I did: more warnings. Dont remember if I saw any reds. The question is, if fastqc output is not a good measure of fastp correctness, what is?  
  > **username_1**: The sequence length distribution warning is triggered if the reads have variable lengths, which you'd expect after adapter removal and/or quality trimming. I'm not familiar with fastp but apparently it tries to guess adapter sequences from the reads by default. If you know the adapter sequence, you'd probably get more accurate results by providing it to fastp. You didn't mention what type of sequencing library the data is from. If this is targeted sequencing, that could explain a mild bias in base composition.  
  > **username_0**: edited. it's a pool-seq. i'm interested in validating what the cleaning process. fasqc doesnt seem to perform well, in this scenario.  
  > **username_5**: Tools like fastqc are useful in helping you determine the quality of your generated FASTQ files, but they are not the determining factor in whether your data is "good", or your experiment is a success. How does your data look when run through your analysis pipeline? That is the critical question.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_6 on 2023-01-10. Score: 3
  
  On a regular basis, using FastQC is quite enough as to assess the quality of FastQ-formatted data. It gives you plenty of details as can be seen in the screenshots you shared.
  
  As others have commented or suggested, there is nothing wrong with having more 'yellow' warnings after processing the FASTQ files using fastp. At lest not as long as what you observe in the data looks fine and the warning has some explanation.
  
  The sequence length distribution warning is very often present post-processing since at the begging all your reads are of the same length but this is not the case any more after you trim the adapters and the low quality bases at the end of the read for example. You have to go to the plots specific to that section and make sure all looks fine. It should be the case.
  
  The GC sequence content warning. It depends a lot what organism you are sequencing, which regions of the genomes, etc. Importantly, what is the expected GC content of those regions. So you can compare to these observed numbers. Once again the warning can be easily triggered after the processing, since for a start Illumina sequencing has known GC bias
  
  Finally, I think actually your reads look better after the processing with fastp. You are showing us the per base sequence quality and before, there are many reads with bases on the yellow and even the red sections of the quality scale. But this is not the case any more after the processing !
  
  > **Comments:**
  > **username_0**: Thanks. I still suspect a tool like fasqc, pair ended aware, will give better results.  
  > **username_7**: You're wrong about fastqc being PE aware as well as fastp not being PE aware. Are you even reading the manuals, @fullmooninu?  
  > **username_8**: I am always amazed by users new to the field and then calling out on of the most extensively used tools in all of bioinformatics. There is nothing wrong with your data in the first place. Quality does not go down after trimming as seen in ghe per-base boxplots, the length warning is expected due after any trim and GC content is just deviation from a theoretical distribution. There is no basis to call results worse or regarding fastqc as not accurate.  
  > **username_0**: @username_7 i didn't find out that in the manual. can you point it out for me, please?  
  > **username_7**: @fullmooninu Literally in the second usage example here: https://github.com/OpenGene/fastp#simple-usage  
  > **username_7**: And it is common knowledge that fastqc analyzes FASTQ files independently. It has no options for PE, you can check using `fastqc --help 2>&1 | grep -ic pair`: This will give you `0`.  
  > **username_0**: @username_7 I appreciate you trying to be helpful. But I'm literally complaining about fastqc not being PE aware, unlike fastp. And am looking for a replacement for fasqc. please delete your comments to decrease entropy.  
  > **username_9**: @fullmooninu in that case, you might want to edit your question since you state "*fastp doesn't analyse paired reads files together*" which is presumably why Ram linked you to the fastp manual which shows that it can actually process paired end reads together.  
  > **username_2**: @fullmooninu I recognise a lot of time has been invested here and it's a great question. Short-read QC is not my area, however as general advice sometimes different viewpoints may not be helpful in the short term. However, in the longterm a different perspective may help you. username_7 is a very experienced bioinformatician, but simply username_7's  approach is direct and this may cause unintentional offence if there's a misunderstanding in the question,  cultures or possibly issues such pressure to complete a project (which we all understand).  
  > **username_7**: @fullmooninu you said "*I still suspect a tool like fasqc, pair ended aware, will give better results*".  Now you say you're complaining about it not being PE aware. Am I missing something? And yes my tone is quite blunt so I apologize for that.  
  > **username_0**: @username_9 corrected, thank you. To the others. Repeating: i'm looking for something to validate fastp's work. Probably like fasqc but that suppports PE. Having more warnings is not a good sign.  
  
  ---
  
  