Answers.Body: <p>1, yes, it's ok to merge fastq, but don't sort the combined fastqs, because sometime later on if you need to run pseudo-aligners (eg. salmon), it requires random read id.</p>
  <p>A more common way is to treat them separately, (in qc, alignment, filtering, and quantification). Then make sure they look same or close to identical in PCA, then you are safe to merge the final counts to increase the read depth (essentially increase power).</p>
  <p>2, GEO samples, refer to the accession with GSMxxx or SAMNxxx, it can contain multiple runs (SRR). If you use SRA Run Selector, and put your PRJNA of interest, you can see the structure of the dataset.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-09-06T07:50:45.000Z
Answers.Text: Also, I found that Salmon has options to use multiple libraries as input, see here: https://salmon.readthedocs.io/en/latest/salmon.html#providing-multiple-read-files-to-salmon
  
  I think that is the option I will use.
Answers.UserDisplayName: Freek
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-09-02T23:28:54.213Z
Answers.LastActivityDate: 2024-09-02T23:28:54.213Z
Answers.UserDisplayName: Feng Yan
Answers.Body: <p>The rule of thumb is: do not merge FASTQ files. Strange things like same sample different read lengths in runs or different quality encodings while thankfully rare will mess up any downstream analysis.
  Also: no point of duplicating input data, creating giant single files and in turn slowing down the analysis.
  Process the FASTQ files separately, make sure that runs are from the same sample/same read length, then merge sorted BAMs.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-09-10T18:51:55.053Z
Answers.Text: Salmon does not produce BAM files, but I guess I could simply add up all the counts from the runs, then re-normalize. Allthough Salmon can handle multiple fastq-files per library so there is that option...
Answers.UserDisplayName: Freek
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-09-10T16:34:38.213Z
Answers.LastActivityDate: 2024-09-10T16:34:38.213Z
Answers.UserDisplayName: darked89
Body: <p>On GEO/SRA some datasets have multiple Runs per Sample, for example <a href="https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE231397" rel="nofollow noreferrer">this one</a> (96 SRR numbers for 12 BioSample/GEO_Accession/Library Name/Sample Name), and the authors provide their RNA-Seq counts for 12 samples/condition <a href="https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE231397" rel="nofollow noreferrer">see here</a>.</p>
  <p>I know that an experiment involving Next Generation Sequencing sometimes requires that more reads are generated than fit on an NGS chip lane, or even more than fit on a single Next Generation Sequencer. So a sequencing library can be split over multiple lanes or multiple machines. This leads to multiple sets of raw data which all belong to the same library. For example to reach the right sensitivity in transcript abundance measurements, you need all the reads of the entire library analyzed together. At least, that is how I understand it. A Run is a set of reads, a Library may consist of multiple Runs, right? <a href="https://www.reddit.com/r/bioinformatics/comments/hurblq/why_does_one_sample_have_so_many_runs/" rel="nofollow noreferrer">Here</a> is some more relevant discussion on Reddit, and <a href="https://www.reddit.com/r/bioinformatics/comments/ho7dfp/multiple_runs_per_gsm_scrna_seq_geo_data/" rel="nofollow noreferrer">here</a> is another one. Looks like you can merge but must be careful, there can be (quality) differences between the Runs. Moreover, NCBI themselves say <a href="https://www.ncbi.nlm.nih.gov/geo/info/rnaseqcounts.html" rel="nofollow noreferrer">here</a> that they simply sum the counts of multiple runs pertaining to a single GEO Sample (RNA-Seq). It also looks like in the <a href="https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE231397" rel="nofollow noreferrer">example dataset</a> the number of bases is only sufficient for normal RNA-Seq analyses of all Runs of a Sample are combined together (you want 25 million reads or so for polyA based RNA-Seq).</p>
  <p>I have two questions about this:</p>
  <ol>
  <li><p>I think that I can just download the SRA files (<code>prefetch</code>), turn them into fastq files (<code>fasterq-dump</code>) and then sort them and cat them together, right? Or is there a better/recommended way?</p>
  </li>
  <li><p>What constitutes a single experiment? Is it the BioSample, GEO_Accession, Library Name or the Sample Name? I'm thinking GEO_Accession, since that is what is used on GEO to designate a &quot;sample&quot;, right? I need a reliable indicator for what happened to single sample...</p>
  </li>
  </ol>
  <p>Looking forward to hearing how you all deal with this, highest regards.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-08-30T14:01:56.460Z
Comments.Text: **Comments have been [moved to chat](https://chat.stackexchange.com/rooms/154914/discussion-on-question-by-freek-combing-multiple-sra-runs-into-a-single-sample); please do not continue the discussion here.** Before posting a comment below this one, please review the [purposes of comments](/help/privileges/comment). Comments that do not request clarification or suggest improvements usually belong as an [answer](/help/how-to-answer), on [meta], or in [chat]. Comments continuing discussion may be removed.
Comments.UserDisplayName: terdon
Tags: <ncbi><sratoolkit><geo><sra><fasterq-dump>
Title: Combing multiple SRA Runs into a single Sample, BioSample or Library
ThreadText: # Combing multiple SRA Runs into a single Sample, BioSample or Library
  
  (Asked by: username_0 on 2024-08-30)
  
  On GEO/SRA some datasets have multiple Runs per Sample, for example [this one](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE231397) (96 SRR numbers for 12 BioSample/GEO\_Accession/Library Name/Sample Name), and the authors provide their RNA-Seq counts for 12 samples/condition [see here](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE231397).
  
  I know that an experiment involving Next Generation Sequencing sometimes requires that more reads are generated than fit on an NGS chip lane, or even more than fit on a single Next Generation Sequencer. So a sequencing library can be split over multiple lanes or multiple machines. This leads to multiple sets of raw data which all belong to the same library. For example to reach the right sensitivity in transcript abundance measurements, you need all the reads of the entire library analyzed together. At least, that is how I understand it. A Run is a set of reads, a Library may consist of multiple Runs, right? [Here](https://www.reddit.com/r/bioinformatics/comments/hurblq/why_does_one_sample_have_so_many_runs/) is some more relevant discussion on Reddit, and [here](https://www.reddit.com/r/bioinformatics/comments/ho7dfp/multiple_runs_per_gsm_scrna_seq_geo_data/) is another one. Looks like you can merge but must be careful, there can be (quality) differences between the Runs. Moreover, NCBI themselves say [here](https://www.ncbi.nlm.nih.gov/geo/info/rnaseqcounts.html) that they simply sum the counts of multiple runs pertaining to a single GEO Sample (RNA-Seq). It also looks like in the [example dataset](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE231397) the number of bases is only sufficient for normal RNA-Seq analyses of all Runs of a Sample are combined together (you want 25 million reads or so for polyA based RNA-Seq).
  
  I have two questions about this:
  
  1. I think that I can just download the SRA files (`prefetch`), turn them into fastq files (`fasterq-dump`) and then sort them and cat them together, right? Or is there a better/recommended way?
  2. What constitutes a single experiment? Is it the BioSample, GEO\_Accession, Library Name or the Sample Name? I'm thinking GEO\_Accession, since that is what is used on GEO to designate a "sample", right? I need a reliable indicator for what happened to single sample...
  
  Looking forward to hearing how you all deal with this, highest regards.
  
  **Tags:** ncbi, sratoolkit, geo, sra, fasterq-dump
  
  > **Comments:**
  > **username_1**: **Comments have been [moved to chat](https://chat.stackexchange.com/rooms/154914/discussion-on-question-by-username_0-combing-multiple-sra-runs-into-a-single-sample); please do not continue the discussion here.** Before posting a comment below this one, please review the [purposes of comments](/help/privileges/comment). Comments that do not request clarification or suggest improvements usually belong as an [answer](/help/how-to-answer), on [meta], or in [chat]. Comments continuing discussion may be removed.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2024-09-02. Score: 2
  
  1, yes, it's ok to merge fastq, but don't sort the combined fastqs, because sometime later on if you need to run pseudo-aligners (eg. salmon), it requires random read id.
  
  A more common way is to treat them separately, (in qc, alignment, filtering, and quantification). Then make sure they look same or close to identical in PCA, then you are safe to merge the final counts to increase the read depth (essentially increase power).
  
  2, GEO samples, refer to the accession with GSMxxx or SAMNxxx, it can contain multiple runs (SRR). If you use SRA Run Selector, and put your PRJNA of interest, you can see the structure of the dataset.
  
  > **Comments:**
  > **username_0**: Also, I found that Salmon has options to use multiple libraries as input, see here: https://salmon.readthedocs.io/en/latest/salmon.html#providing-multiple-read-files-to-salmon  I think that is the option I will use.  
  
  ---
  
  ### Answer by username_3 on 2024-09-10. Score: 0
  
  The rule of thumb is: do not merge FASTQ files. Strange things like same sample different read lengths in runs or different quality encodings while thankfully rare will mess up any downstream analysis.
  Also: no point of duplicating input data, creating giant single files and in turn slowing down the analysis.
  Process the FASTQ files separately, make sure that runs are from the same sample/same read length, then merge sorted BAMs.
  
  > **Comments:**
  > **username_0**: Salmon does not produce BAM files, but I guess I could simply add up all the counts from the runs, then re-normalize. Allthough Salmon can handle multiple fastq-files per library so there is that option...  
  
  ---
  
  