Answers.Body: <p>Okay, you can't concatenate genes into a singular matrix particularly if these are rRNA genes versus protein genes, except in rare situations where the tests say 'thats okay' (below). This is simply because they evolve at very different rates rRNA genes have loci which evolve very slowly and are highly conserved.</p>
  <p>The theory goes back to incongruence length difference test (ILD), which is somewhat confusingly also known as the partition homogeneity test and was then modified to become the localised ILD test.</p>
  <p>The modern version of this test, simply called 'homogeneity' is found in <a href="http://www.iqtree.org/doc/Assessing-Phylogenetic-Assumptions" rel="nofollow noreferrer">IQ-Tree</a>. Thus, the best approach is to test the assumption empirically,</p>
  <blockquote>
  <p>IQ-TREE provides three matched-pairs tests of symmetry (Naser-Khdour
  et al., 2019) to test the two assumptions of stationarity and
  homogeneity. A simple analysis:</p>
  <p>iqtree2 -s example.phy -p example.nex --symtest-only</p>
  </blockquote>
  <p>Just to be clear this tests whether the rate between two partitions (genes) is homogeneous. If 'yes' the supermatrix is fine, if 'no' the data needs partitioning.</p>
  <p>The theory is simply that if two genes evolve at noticeably different rates they can't be combined into a singular matrix. The theory goes back a long way is used to circumvent long-branch attraction issues. Thus, at a minimum the rRNA genes and protein genes need to be compartmentalised, i.e. have their own matrix. Between protein genes, if they are house-keeping genes, generally they'll have homogeneous rates (which you can empirically test via IQ-TREE).</p>
  <p>If someone is concatenating rRNA with protein genes without accommodating rate variation via partitioning it is simply sloppy phylogenetics and if published represents a shortcoming with the reviewing process. This is because in all certainly these loci will fail any rate homogeneity test, there's a slim off-chance its okay which you can test it empirically. Say if 18S and 28S were concatenated (or just 16S if these are bacteria) with an atp gene, that can give a different tree to 18S with atpA, atpB, atpC, atpD etc... so the ratio between rRNA to protein genes can start influencing the tree ... which is simply wrong.</p>
  <p>If someone concatenates a load of house-keeping genes thats okay because there's a good chance they're homogeneous and if even the test failed, they may not be that far out, rRNA versus proteins can be way out. This can easily be an order of magnitude out and thats difficult to ignore. When there's proteins under selection versus conserved housekeeping genes ... its very unlikely homogeneity would hold.</p>
  <p><strong>Summary</strong> As a general rule of thumb in your method of tree building compartmentalising all genes is safer route, computationally there's no difference between zero partitions and a partition for every gene. You are never going to make a mistake.</p>
  <p>Where it gets complicated is when the phylogenetic parameters, particularly in advanced models, are themselves estimated via maximum likelihood alongside the topology. The rational for this is complicated. Computationally, this can get very, very expensive even on supercomputers and hence there's a tendency to make an assumption (often without checking it) that there are fewer partitions than there actually are. Regardless, assuming away rRNA versus protein genes even in this method of tree estimation should not done and the test will almost certainly fail.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-09-08T07:55:53.327Z
Answers.Text: Appreciate your answer. I agree with the possible induced error when concatenating together rRNAs and protein genes. Sorry for the confusion caused from the comments. What I mean was I have dealt with both type of datasets and they were concatenated separately. But that was quite an interesting input for me, especially the ILD test, will try that out.
Answers.UserDisplayName: web
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-09-08T09:07:46.730Z
Answers.Text: Btw, from your last paragraph, though not really familiar with the concept behind, I was aware that it is possible to estimate best models via the ML algorithm. But what does it meant by "there's tendency to make an assumption that there are fewer partitions than there actually are"? Also, I am very interested with "computationally there's no difference between zero partitions and a partition for every gene", would you mind to elaborate a bit more if possible? What do you think about it theoretically? Btw, I have updated the question. Thanks!
Answers.UserDisplayName: web
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-09-08T21:31:41.957Z
Answers.Text: Agreed @web the error was unfortunate. "computationally no difference ...", an exaggeration, should be "the difference in computation time is not a major issue". Final point "tendency ...", a reviewer will be aware of the difference between what is computationally practical against the theoretical requirements of the calculation, certain approximations are considered ok.
Answers.UserDisplayName: M__
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-09-08T01:48:54.733Z
Answers.LastActivityDate: 2022-09-08T01:55:24.667Z
Answers.LastEditDate: 2022-09-08T01:55:24.667Z
Answers.UserDisplayName: M__
Body: <p>I am dealing with several genes in my dataset. For each of the genes, I have built gene tree, with their best estimated model. I am intended to look into the effect of the tree from concatenated sequences (supermatrix tree), using Maximum Likelihood approach.</p>
  <p>I am considering two options to apply the model for the supermatrix tree inference.</p>
  <ol>
  <li>Obtain a single best estimated model for the supermatrix gene</li>
  <li>Apply partitioned model for the supermatrix gene, based on each gene's model</li>
  </ol>
  <p>From what I have read in most if not all of the research articles, they seemed to be using option 1, where a single model applied for the supermatrix tree inference. Since the purpose of using evolutionary model is to describe the changes in the gene and given that the evolution pattern in each gene differ, why wouldn't the respective model of each gene be considered in the tree inference?</p>
  <p>For examples,</p>
  <p>Dataset A: gene1 and gene2 are having LG and WAG model respectively as the best estimated model. But when the two genes are being concatenated, it turned out that LG is the best model for the matrix. It seemed that most if not all of the experiment opts for option 1 in this case. Theoretically, will the supermatrix tree using option 1 and option 2 in the model application still similar? Also, will the evolutionary pattern in gene2 still being represented correctly on the tree when we opt for option 1 in the supermatrix? Or it actually does not impact much on the calculation of the tree?</p>
  <p>Dataset B: gene1 and gene2 are having LG as their best estimated model. The concatenated matrix has LG as its best estimated model too. Will model partitioning has any effect in this case? Does the ML tree/site likelihood calculation affected by the length of the input sequences?</p>
  <p>Updated (replied from the RAxML-NG author):</p>
  <blockquote>
  <p>combining multiple genes with the same substitution model into a
  single partition would result in joint estimation of free model
  parameters (e.g., alpha parameter for +G). So in general, it will make
  a difference for likelihood computation, except if there are no free
  parameters to estimate.</p>
  </blockquote>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-07T15:03:54.183Z
Comments.Text: I understand concatenation well. What I'd like to be clear about is the 'super-matrix', is this the single mutation matrix derived from the concatenated alignment? Secondly are you obtaining your matrix before you perform the tree calculation and then inputing this matrix for the ML tree search?
Comments.UserDisplayName: M__
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-07T20:19:14.343Z
Comments.Text: @M__ yes for both. Say if I have 5 genes. The supermatrix derived from adding gene 2 at the end of gene 1, gene 3 at the end of gene 2, ... , gene 5 at the end of gene 4. They will exist as a big and single concatenated alignment matrix, concatenating gene1 to gene5 side by side. Then the supermatrix would be used together with the model/s to calculate for the ML tree.
Comments.UserDisplayName: web
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-07T21:00:52.653Z
Comments.Text: Thanks, two final questions are these genes physically linked? Or random amplicons, across a genome? Finally, is this a mix of rRNA genes and protein genes and within the protein genes are they all single copy house keeping genes? I'll answer before the end of the week.
Comments.UserDisplayName: M__
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-07T23:20:10.407Z
Comments.Text: The genes are random across genome and yes they are  all single copy rRNAs and protein genes.
Comments.UserDisplayName: web
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-08T00:40:37.260Z
Comments.Text: Thanks. Sorry to be persistent, are the protein genes 'house-keeping', e.g. often a core metabolic enzyme and all part of the core-genome, specific to the Order/family/species or a mix of the two?
Comments.UserDisplayName: M__
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-09-08T01:42:50.800Z
Comments.Text: No worries. It's just that my question is actually referring to those in general situation. It would be great if you could help me understand the concern. Probably I could justify it clearer.
Comments.UserDisplayName: web
Tags: <phylogenetics><phylogeny><modelling><substitution-model>
Title: Single or partitioned model for supermatrix tree inference?
ThreadText: # Single or partitioned model for supermatrix tree inference?
  
  (Asked by: username_0 on 2022-09-07)
  
  I am dealing with several genes in my dataset. For each of the genes, I have built gene tree, with their best estimated model. I am intended to look into the effect of the tree from concatenated sequences (supermatrix tree), using Maximum Likelihood approach.
  
  I am considering two options to apply the model for the supermatrix tree inference.
  
  1. Obtain a single best estimated model for the supermatrix gene
  2. Apply partitioned model for the supermatrix gene, based on each gene's model
  
  From what I have read in most if not all of the research articles, they seemed to be using option 1, where a single model applied for the supermatrix tree inference. Since the purpose of using evolutionary model is to describe the changes in the gene and given that the evolution pattern in each gene differ, why wouldn't the respective model of each gene be considered in the tree inference?
  
  For examples,
  
  Dataset A: gene1 and gene2 are having LG and WAG model respectively as the best estimated model. But when the two genes are being concatenated, it turned out that LG is the best model for the matrix. It seemed that most if not all of the experiment opts for option 1 in this case. Theoretically, will the supermatrix tree using option 1 and option 2 in the model application still similar? Also, will the evolutionary pattern in gene2 still being represented correctly on the tree when we opt for option 1 in the supermatrix? Or it actually does not impact much on the calculation of the tree?
  
  Dataset B: gene1 and gene2 are having LG as their best estimated model. The concatenated matrix has LG as its best estimated model too. Will model partitioning has any effect in this case? Does the ML tree/site likelihood calculation affected by the length of the input sequences?
  
  Updated (replied from the RAxML-NG author):
  
  > combining multiple genes with the same substitution model into a
  > single partition would result in joint estimation of free model
  > parameters (e.g., alpha parameter for +G). So in general, it will make
  > a difference for likelihood computation, except if there are no free
  > parameters to estimate.
  
  **Tags:** phylogenetics, phylogeny, modelling, substitution-model
  
  > **Comments:**
  > **username_1**: I understand concatenation well. What I'd like to be clear about is the 'super-matrix', is this the single mutation matrix derived from the concatenated alignment? Secondly are you obtaining your matrix before you perform the tree calculation and then inputing this matrix for the ML tree search?  
  > **username_0**: @username_1 yes for both. Say if I have 5 genes. The supermatrix derived from adding gene 2 at the end of gene 1, gene 3 at the end of gene 2, ... , gene 5 at the end of gene 4. They will exist as a big and single concatenated alignment matrix, concatenating gene1 to gene5 side by side. Then the supermatrix would be used together with the model/s to calculate for the ML tree.  
  > **username_1**: Thanks, two final questions are these genes physically linked? Or random amplicons, across a genome? Finally, is this a mix of rRNA genes and protein genes and within the protein genes are they all single copy house keeping genes? I'll answer before the end of the week.  
  > **username_0**: The genes are random across genome and yes they are  all single copy rRNAs and protein genes.  
  > **username_1**: Thanks. Sorry to be persistent, are the protein genes 'house-keeping', e.g. often a core metabolic enzyme and all part of the core-genome, specific to the Order/family/species or a mix of the two?  
  > **username_0**: No worries. It's just that my question is actually referring to those in general situation. It would be great if you could help me understand the concern. Probably I could justify it clearer.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-09-08. Score: 1
  
  Okay, you can't concatenate genes into a singular matrix particularly if these are rRNA genes versus protein genes, except in rare situations where the tests say 'thats okay' (below). This is simply because they evolve at very different rates rRNA genes have loci which evolve very slowly and are highly conserved.
  
  The theory goes back to incongruence length difference test (ILD), which is somewhat confusingly also known as the partition homogeneity test and was then modified to become the localised ILD test.
  
  The modern version of this test, simply called 'homogeneity' is found in [IQ-Tree](http://www.iqtree.org/doc/Assessing-Phylogenetic-Assumptions). Thus, the best approach is to test the assumption empirically,
  
  > IQ-TREE provides three matched-pairs tests of symmetry (Naser-Khdour
  > et al., 2019) to test the two assumptions of stationarity and
  > homogeneity. A simple analysis:
  >
  > iqtree2 -s example.phy -p example.nex --symtest-only
  
  Just to be clear this tests whether the rate between two partitions (genes) is homogeneous. If 'yes' the supermatrix is fine, if 'no' the data needs partitioning.
  
  The theory is simply that if two genes evolve at noticeably different rates they can't be combined into a singular matrix. The theory goes back a long way is used to circumvent long-branch attraction issues. Thus, at a minimum the rRNA genes and protein genes need to be compartmentalised, i.e. have their own matrix. Between protein genes, if they are house-keeping genes, generally they'll have homogeneous rates (which you can empirically test via IQ-TREE).
  
  If someone is concatenating rRNA with protein genes without accommodating rate variation via partitioning it is simply sloppy phylogenetics and if published represents a shortcoming with the reviewing process. This is because in all certainly these loci will fail any rate homogeneity test, there's a slim off-chance its okay which you can test it empirically. Say if 18S and 28S were concatenated (or just 16S if these are bacteria) with an atp gene, that can give a different tree to 18S with atpA, atpB, atpC, atpD etc... so the ratio between rRNA to protein genes can start influencing the tree ... which is simply wrong.
  
  If someone concatenates a load of house-keeping genes thats okay because there's a good chance they're homogeneous and if even the test failed, they may not be that far out, rRNA versus proteins can be way out. This can easily be an order of magnitude out and thats difficult to ignore. When there's proteins under selection versus conserved housekeeping genes ... its very unlikely homogeneity would hold.
  
  **Summary** As a general rule of thumb in your method of tree building compartmentalising all genes is safer route, computationally there's no difference between zero partitions and a partition for every gene. You are never going to make a mistake.
  
  Where it gets complicated is when the phylogenetic parameters, particularly in advanced models, are themselves estimated via maximum likelihood alongside the topology. The rational for this is complicated. Computationally, this can get very, very expensive even on supercomputers and hence there's a tendency to make an assumption (often without checking it) that there are fewer partitions than there actually are. Regardless, assuming away rRNA versus protein genes even in this method of tree estimation should not done and the test will almost certainly fail.
  
  > **Comments:**
  > **username_0**: Appreciate your answer. I agree with the possible induced error when concatenating together rRNAs and protein genes. Sorry for the confusion caused from the comments. What I mean was I have dealt with both type of datasets and they were concatenated separately. But that was quite an interesting input for me, especially the ILD test, will try that out.  
  > **username_0**: Btw, from your last paragraph, though not really familiar with the concept behind, I was aware that it is possible to estimate best models via the ML algorithm. But what does it meant by "there's tendency to make an assumption that there are fewer partitions than there actually are"? Also, I am very interested with "computationally there's no difference between zero partitions and a partition for every gene", would you mind to elaborate a bit more if possible? What do you think about it theoretically? Btw, I have updated the question. Thanks!  
  > **username_1**: Agreed @username_0 the error was unfortunate. "computationally no difference ...", an exaggeration, should be "the difference in computation time is not a major issue". Final point "tendency ...", a reviewer will be aware of the difference between what is computationally practical against the theoretical requirements of the calculation, certain approximations are considered ok.  
  
  ---
  
  