Answers.Body: <p>As mentioned in the comments, your biological question is going to be vitally important in determining the best way to tackle this issue. However, there are some things that you should generally keep in mind. </p>
  
  <ol>
  <li><p><strong>Most <em>de novo</em> assemblers will perform kmer normalization.</strong> This means that the merged RNA-Seq data will be normalized across all libraries. This is almost certainly not what you want because you'll be randomly selecting reads from all libraries without regard to their treatments. So, if you're going to allow for kmer normalization, you should normalize prior to merging the libraries so that you have equal representation of the reads by library. If you don't normalize, you're going to have a very fragmented assembly, and it will take a long time to generate.</p></li>
  <li><p><strong><em>De novo</em> assemblers using short reads assume all identified exons with junction support may be used to generate splice variants with equal probability.</strong> I'm not entirely sure how to best phrase that, but here's an example. Suppose in a cold-treated library that Gene A generates a transcript using exons 1-2-3, but a heat-treated library generates transcripts using 3-4-5 and 2-3-5. It may be that merging the libraries results in transcripts generated containing various permutations of those exons (e.g. 1-2-3-5, 2-3-4, etc.) that aren't generated in any of the biological settings but can no longer be distinguished. This will lead to false positive transcripts which will detrimentally affect expression quantification.</p></li>
  <li><p><strong>Merging assemblies is difficult.</strong> So, to be true to each treatment, it makes more sense to generate individual assemblies (which will also result in library-specific kmer normalization). StringTie essentially does this using reference genomes. You assemble using each library (all bio-reps for a treatment), and then merge after the fact and re-quantify. This is the best approach. However, it can be difficult and computationally taxing. Some programs like CD-HIT (specifically cd-hit-est) are typically used in these cases. Keep in mind that CD-HIT will drop sequences with high similarity, not merge them. This may lead to loss of some uniquely assembled sequence. Other approaches involve using programs like CAP3 (a DNA assembler) to treat the transcripts like longer reads for assembly. Depending on the parameter choice, this might work, but you will risk generating chimeric transcripts.</p></li>
  </ol>
  
  <p>How you move forward with the assembly process will again depend on your biological questions. I would tend to promote that you tackle the questions one at a time rather than using a single transcriptome to address them all, but there may be circumstances in which merging assemblies makes more sense. </p>
  
  <p>In your specific case, I would perform multiple assemblies using a variety of the approaches described above and then annotate the assemblies (ex. using Trinotate) to see how their quality metrics compare. I would discourage generating a single transcriptome and then re-assembling individual transcriptomes from that assembly -- the single transcriptome may have misassemblies that would then propagate to the individual assemblies, and this process would generally be messy/difficult. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-14T09:45:08.027Z
Answers.Text: Thank you for your answer. I edited my question to clarify it a little bit: I want to use the assembly as a reference to map reads from the single transcriptomes and do (among others) DE analysis. I probably won't use the sequence information directly from the reference.
Answers.UserDisplayName: LinuxBlanket
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-14T12:49:14.337Z
Answers.Text: As mentioned above, "The biological problems are hard without knowing the organism. The technical problems can't be addressed without finding them." I would second that statement and direct you to the last paragraph I just added for clarification.
Answers.UserDisplayName: LucasBoatwright
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-08T14:35:49.133Z
Answers.LastActivityDate: 2019-05-14T12:46:05.133Z
Answers.LastEditDate: 2019-05-14T12:46:05.133Z
Answers.UserDisplayName: LucasBoatwright
Body: <p>I'm planning to build a transcriptome by pooling all existing transcriptomes in SRA for a non-model species (which has no reference genome) to study differentially expressed genes and the like. It might worth mentioning that the transcriptomes I want to pool mostly come from different tissues and sampling sites. These transcriptomes would come from five SRA BioProjects plus one project from my laboratoryfor a total of 102 single transcriptomes coming from about 20 individuals (plus three pooled larvae transcriptomes). I'm especially interested in differentially expressed genes in different tissues, on genes specific to different tissues and on the UTRs of differentially expressed genes.</p>
  
  <p>My idea is to build a common reference transcriptome against which I'd re-assemble the individual transcriptomes, and only then I'd get differential expression data etc.</p>
  
  <p>I'm looking for literature (or personal experience) dealing with assemblies of transcriptomes / comparing transcriptomes coming from different datasets in order to understand whether this is a <strong>good idea or not</strong> and get a grasp of what <strong>technical and biological problems</strong> I could face. I'm (vaguely) aware of the batch effect and of <a href="https://rdrr.io/bioc/sva/man/ComBat.html" rel="nofollow noreferrer">one way to remove it</a>, but I'd like to read papers dealing with my same problem (different SRA databases, without reference genomes). Up until now I found just projects dealing with microarray data.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-04T05:25:21.787Z
Comments.Text: R's Bioconductor (obviously). There is a lot of experience on this board in exactly this area using NGS, albeit with the focus on "model" species (humans). No reference genome, sounds a bit scary.
Comments.UserDisplayName: M__
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-04T09:22:20.700Z
Comments.Text: I don't have personal experience, so I won't post an answer. Be aware the with ComBat you could introduce a new batch effect (it is not necessary to remove them in order to work with them, you can take it into account). The other problem is that you will have several conditions (even the same tissue is not the same at the first year of the organism that just a year before dying).
Comments.UserDisplayName: llrs
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-04T09:22:25.170Z
Comments.Text: However the most important thing is what do you expect to learn? The differentially expressed genes don't mean anything if you don't know what you are comparing or if you are not looking for something very specific (healthy vs disease, one tissue vs the other...). Please [edit] the question to explain what is your biological question and it will be easier to help. (Also if you provide the number of projects of your organism and the number of total samples it would be helpful to us too)
Comments.UserDisplayName: llrs
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-13T14:28:59.497Z
Comments.Text: @llrs I edited the question with the things you asked and I clarified it a little bit (I was too much into my project to notice that I had skipped some fundamental information...). I'm aware I will have different conditions, that's why I'm not entirely sure that this is a good idea (i.e. that can give me some significant biological insight).
Comments.UserDisplayName: LinuxBlanket
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-13T20:10:43.407Z
Comments.Text: @LinuxBlanket Ok, that information focused the question. Still you haven't explained what are you trying to do with this comparison. Why do you want to compare transcriptomes coming from different datasets? Having several conditions is not a problem if you want to identify what is more important of each condition. But I'm not sure what comparison are you willing to do
Comments.UserDisplayName: llrs
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-14T08:23:34.833Z
Comments.Text: @llrs I'm especially interested in genes expressed exclusively in gonads, but I also want to have some insights about the basic transcriptomic features of this animal, like its housekeeping genes. That's why I need to have as much tissues as possible, so that I can really know which mRNAs are transcribed only by the gonads and which ones are transcribed by every tissue instead.
Comments.UserDisplayName: LinuxBlanket
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-14T08:35:04.600Z
Comments.Text: Great! (Do not forget to [edit] this into the question.) How many samples do you have from the gonads and from other tissues? If this is your objective and this is the best you can find I do not see any problem in doing it. The biological problems are hard without knowing the organism. The technical problems can't be addressed without finding them... so I'm not sure if I can answer more
Comments.UserDisplayName: llrs
Tags: <database><differential-expression><transcriptome><literature-search>
Title: Merging transcriptomes coming from different experiments
ThreadText: # Merging transcriptomes coming from different experiments
  
  (Asked by: username_0 on 2019-05-03)
  
  I'm planning to build a transcriptome by pooling all existing transcriptomes in SRA for a non-model species (which has no reference genome) to study differentially expressed genes and the like. It might worth mentioning that the transcriptomes I want to pool mostly come from different tissues and sampling sites. These transcriptomes would come from five SRA BioProjects plus one project from my laboratoryfor a total of 102 single transcriptomes coming from about 20 individuals (plus three pooled larvae transcriptomes). I'm especially interested in differentially expressed genes in different tissues, on genes specific to different tissues and on the UTRs of differentially expressed genes.
  
  My idea is to build a common reference transcriptome against which I'd re-assemble the individual transcriptomes, and only then I'd get differential expression data etc.
  
  I'm looking for literature (or personal experience) dealing with assemblies of transcriptomes / comparing transcriptomes coming from different datasets in order to understand whether this is a **good idea or not** and get a grasp of what **technical and biological problems** I could face. I'm (vaguely) aware of the batch effect and of [one way to remove it](https://rdrr.io/bioc/sva/man/ComBat.html), but I'd like to read papers dealing with my same problem (different SRA databases, without reference genomes). Up until now I found just projects dealing with microarray data.
  
  **Tags:** database, differential-expression, transcriptome, literature-search
  
  > **Comments:**
  > **username_1**: R's Bioconductor (obviously). There is a lot of experience on this board in exactly this area using NGS, albeit with the focus on "model" species (humans). No reference genome, sounds a bit scary.  
  > **username_2**: I don't have personal experience, so I won't post an answer. Be aware the with ComBat you could introduce a new batch effect (it is not necessary to remove them in order to work with them, you can take it into account). The other problem is that you will have several conditions (even the same tissue is not the same at the first year of the organism that just a year before dying).  
  > **username_2**: However the most important thing is what do you expect to learn? The differentially expressed genes don't mean anything if you don't know what you are comparing or if you are not looking for something very specific (healthy vs disease, one tissue vs the other...). Please [edit] the question to explain what is your biological question and it will be easier to help. (Also if you provide the number of projects of your organism and the number of total samples it would be helpful to us too)  
  > **username_0**: @username_2 I edited the question with the things you asked and I clarified it a little bit (I was too much into my project to notice that I had skipped some fundamental information...). I'm aware I will have different conditions, that's why I'm not entirely sure that this is a good idea (i.e. that can give me some significant biological insight).  
  > **username_2**: @username_0 Ok, that information focused the question. Still you haven't explained what are you trying to do with this comparison. Why do you want to compare transcriptomes coming from different datasets? Having several conditions is not a problem if you want to identify what is more important of each condition. But I'm not sure what comparison are you willing to do  
  > **username_0**: @username_2 I'm especially interested in genes expressed exclusively in gonads, but I also want to have some insights about the basic transcriptomic features of this animal, like its housekeeping genes. That's why I need to have as much tissues as possible, so that I can really know which mRNAs are transcribed only by the gonads and which ones are transcribed by every tissue instead.  
  > **username_2**: Great! (Do not forget to [edit] this into the question.) How many samples do you have from the gonads and from other tissues? If this is your objective and this is the best you can find I do not see any problem in doing it. The biological problems are hard without knowing the organism. The technical problems can't be addressed without finding them... so I'm not sure if I can answer more  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2019-05-08. Score: 2
  
  As mentioned in the comments, your biological question is going to be vitally important in determining the best way to tackle this issue. However, there are some things that you should generally keep in mind.
  
  1. **Most *de novo* assemblers will perform kmer normalization.** This means that the merged RNA-Seq data will be normalized across all libraries. This is almost certainly not what you want because you'll be randomly selecting reads from all libraries without regard to their treatments. So, if you're going to allow for kmer normalization, you should normalize prior to merging the libraries so that you have equal representation of the reads by library. If you don't normalize, you're going to have a very fragmented assembly, and it will take a long time to generate.
  2. ***De novo* assemblers using short reads assume all identified exons with junction support may be used to generate splice variants with equal probability.** I'm not entirely sure how to best phrase that, but here's an example. Suppose in a cold-treated library that Gene A generates a transcript using exons 1-2-3, but a heat-treated library generates transcripts using 3-4-5 and 2-3-5. It may be that merging the libraries results in transcripts generated containing various permutations of those exons (e.g. 1-2-3-5, 2-3-4, etc.) that aren't generated in any of the biological settings but can no longer be distinguished. This will lead to false positive transcripts which will detrimentally affect expression quantification.
  3. **Merging assemblies is difficult.** So, to be true to each treatment, it makes more sense to generate individual assemblies (which will also result in library-specific kmer normalization). StringTie essentially does this using reference genomes. You assemble using each library (all bio-reps for a treatment), and then merge after the fact and re-quantify. This is the best approach. However, it can be difficult and computationally taxing. Some programs like CD-HIT (specifically cd-hit-est) are typically used in these cases. Keep in mind that CD-HIT will drop sequences with high similarity, not merge them. This may lead to loss of some uniquely assembled sequence. Other approaches involve using programs like CAP3 (a DNA assembler) to treat the transcripts like longer reads for assembly. Depending on the parameter choice, this might work, but you will risk generating chimeric transcripts.
  
  How you move forward with the assembly process will again depend on your biological questions. I would tend to promote that you tackle the questions one at a time rather than using a single transcriptome to address them all, but there may be circumstances in which merging assemblies makes more sense.
  
  In your specific case, I would perform multiple assemblies using a variety of the approaches described above and then annotate the assemblies (ex. using Trinotate) to see how their quality metrics compare. I would discourage generating a single transcriptome and then re-assembling individual transcriptomes from that assembly -- the single transcriptome may have misassemblies that would then propagate to the individual assemblies, and this process would generally be messy/difficult.
  
  > **Comments:**
  > **username_0**: Thank you for your answer. I edited my question to clarify it a little bit: I want to use the assembly as a reference to map reads from the single transcriptomes and do (among others) DE analysis. I probably won't use the sequence information directly from the reference.  
  > **username_3**: As mentioned above, "The biological problems are hard without knowing the organism. The technical problems can't be addressed without finding them." I would second that statement and direct you to the last paragraph I just added for clarification.  
  
  ---
  
  