Answers.Body: <p>There's a <a href="http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1004075" rel="nofollow noreferrer">new paper</a> that's just appeared on the subject of "proportionality", including a method by which RNA expression might be compared.</p>
  
  <p>This is a new concept for me, and the article is not easy enough for me to read that I can write up a quick summary; the authors don't seem to devote a section in the paper to defining "proportionality". However, here's an interesting chunk from the article:</p>
  
  <blockquote>
    <p>We graphed the network of relationships between these mRNAs (S5 Fig.), an approach similar to gene co-expression network [12] or weighted gene co-expression analysis [13] but founded on proportionality and therefore valid for relative data. The network revealed one cluster of 96, and many other smaller clusters of mRNAs behaving proportionally across conditions.<br>
    ...<br>
    We are also keen to raise awareness that correlation (and other statistical methods that assume measurements come from real coordinate space) should not be applied to relative abundances. This is highly relevant to gene coexpression networks [12]. Correlation is at the heart of methods like Weighted Gene Co-expression Network Analysis [13] and heatmap visualization [14]. These methods are potentially misleading if applied to relative data.</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-06T10:04:31.257Z
Answers.Text: proportionality is defined as (1+ beta^2 - 2*beta*abs(r))
  
  where r is the Pearson correlation of log x and log y, and beta^2 is var(log(y))/var(log(x)).
  
  
  The authors also note: "Proportionality is appropriate, but ϕ does not satisfy the properties of a distance—most obviously, it is not symmetric unless β = 1". They go on to say "Hence, our approach to forming a dissimilarity matrix is simply to work with ϕ(log xi, log xj) where i < j, in effect, the lower triangle of the matrix of ϕ values between all pairs of components."
Answers.UserDisplayName: Ian Sudbery
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-06T11:41:49.050Z
Answers.Text: But why is it called "proportionality" (in non-formulaic terms)? What is the effect of that formula on expression values? Can you fill in the remainder of this sentence: "Proportionality is a measure that represents how the expression of two ..."
Answers.UserDisplayName: gringer
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-06T12:25:19.980Z
Answers.Text: No, not really - I didn't quite understand the use of proportionality. Although I do note that phi is not a measure of proportionality, but a measure of goodness-of-fit to proportionality.
Answers.UserDisplayName: Ian Sudbery
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-06T18:35:05.417Z
Answers.Text: Proportionalty implies the ratio of expression values of two genes will be a constant. Instead of fitting $y=mx+c$ in regression, proportionalty fits $y=mx$
Answers.UserDisplayName: Saket Choudhary
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-04T22:35:10.487Z
Answers.LastActivityDate: 2017-12-07T13:26:46.570Z
Answers.LastEditDate: 2017-12-07T13:26:46.570Z
Answers.UserDisplayName: gringer
Answers.Body: <p>The suggestion to use proportionality is probably the correct one if you are interested in similar <em>patterns</em> between samples. But not if you are interested in <em>absolute</em> differences. </p>
  
  <p>For example: the following two samples are similar in pattern, but not similar on absolute levels:</p>
  
  <pre><code>        Sample 1    Sample 2    Sample 3
  Gene A    10          100           80
  Gene B     8           80          100  
  Gene C    12          120          120
  </code></pre>
  
  <p>Samples 1 and 2 have a perfect proportionality (phi is 0) and also a perfect correlation (as a side note any pair with a perfect proportionality will always have a perfect correlation). However, in terms of logfold changes, samples 2 and 3 are more like each other. </p>
  
  <p>Of course in real life you would never see a comparison like the sample 1 - sample 2 one because normalisation would have removed the scale difference. This was exactly the point brought up by the proportionality paper. But normalisation methods don't normally guarantee that the sum of expression for each sample is identical, and such differences can still occur. </p>
  
  <p>An alternative that might suit more in the second case is either the euclidean distance between the samples, or the euclidean distance on the first two components of a principle component or multi-dimensional scaling. The latter is effectively using mean logFC between samples.</p>
  
  <p>Assuming that <code>x</code> is a matrix containing normalised, log transformed expression values, you could use R and limma to calculate distance in multi-dimensional scaled space as follows:</p>
  
  <pre><code>library(limma)
  mds &lt;- plotMDS(x, plot=FALSE)
  mds &lt;- data.frame(mds$x, mds$y)
  distances &lt;- dist(mds)
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-06T12:29:51.520Z
Answers.Text: Normalisation would only remove the scale difference if it were consistent across a large proportion of genes. In the example you have given, this would indeed be the case if only the indicated genes were assayed, but with this low number of genes it would be more common to normalise based on a housekeeping gene rather than the total gene expression.
Answers.UserDisplayName: gringer
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-06T12:32:37.233Z
Answers.Text: Yes, this was my point - hence why there might be reason to use some distance measure (such as euclidean distance) that measured scale difference rather than pattern differences.
Answers.UserDisplayName: Ian Sudbery
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-06T10:57:34.970Z
Answers.LastActivityDate: 2017-06-06T10:57:34.970Z
Answers.UserDisplayName: Ian Sudbery
Answers.Body: <p>If you have a lot of samples with very different environmental conditions, then a Weighted Gene Correlation Network Analysis (<a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2631488/" rel="nofollow noreferrer">WGCNA</a>) might be appropriate.</p>
  
  <p>This type of analysis looks for genes that trace similar (or opposing) expression patterns throughout different conditions (e.g. high-medium-medium-low-absent-high would be highly negatively correlated with low-medium-medium-high-very high-low).</p>
  
  <p>That particular paper introduces the concept of "modules", which are groups of genes that share similar expression patterns. Functions are available for plotting how the expression of canonical module members changes throughout the different conditions, and for identifying which module (or modules) a particular gene is likely to be a member of.</p>
  
  <p>WGCNA works best when there's a lot of different expression changes in the different conditions, which sounds like it would fit well with your project. However, it concentrates more on the genes, rather than the conditions (which seems like it would be less useful for you).</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-04T22:42:23.850Z
Answers.LastActivityDate: 2017-12-07T13:27:28.263Z
Answers.LastEditDate: 2017-12-07T13:27:28.263Z
Answers.UserDisplayName: gringer
Answers.Body: <p>The euclidean distance is probably the simplest, both conceptually and in terms of implementation. It's by no means an elegant solution, and it may not perform well in certain circumstances.</p>
  
  <p>Euclidean distance is easiest to conceptualize as the distance between two points in a two dimensional space.</p>
  
  <pre><code>Y
  ^
  |
  |
  |
  |            * p = (3, 3)
  |
  |
  |
  |        * q = (2, 1)
  |
  ----------------------------&gt; X
  </code></pre>
  
  <p>In this example, the distance between the two points is</p>
  
  <pre><code>d(p, q) = sqrt( (p_x-q_x)^2 + (p_y-q_y)^2 )
          = sqrt( (2-3)^2 + (1-3)^2 )
          = sqrt(5)
          ≈ 2.24
  </code></pre>
  
  <p>For a gene expression profile with two genes, this is exactly how the euclidean distance would be calculated, using expression values from one gene as the X axis and expression values from the other gene as the Y axis. Realistically, though, gene expression profiles typically contain thousands or tens of thousands of genes, so instead we use the generalization of the distance calculation for N dimensions.</p>
  
  <pre><code>d(p, q) = sqrt( (p_1-q_1)^2 + (p_2-q_2)^2 + ... + (p_N-q_N)^2 )
  </code></pre>
  
  <p>Packages for R and Python make these types of calculations trivial once you have the data loaded into the correct data structure. See Ian's answer for some example R code.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-06T19:05:40.117Z
Answers.LastActivityDate: 2017-06-06T19:05:40.117Z
Answers.UserDisplayName: Daniel Standage
Body: <p>Some of the work in our lab requires a comparison of a strain across several experimental conditions. We are looking to identify most similar experimental conditions based on the gene transcription response similarity from the cell.</p>
  
  <p>While we could easily invent and create home-grown methods to do it, their implementation and testing are a laborious project in themselves and are outside the scope of our current work.</p>
  
  <p>Are there any methods for RNA expression profile similarity computation that has been already published? If yes, what is your experience using them?</p>
  
Tags: <rna-seq>
Title: What methods exist to calculate RNA expression profile similarity
ThreadText: # What methods exist to calculate RNA expression profile similarity
  
  (Asked by: username_0 on 2017-06-04)
  
  Some of the work in our lab requires a comparison of a strain across several experimental conditions. We are looking to identify most similar experimental conditions based on the gene transcription response similarity from the cell.
  
  While we could easily invent and create home-grown methods to do it, their implementation and testing are a laborious project in themselves and are outside the scope of our current work.
  
  Are there any methods for RNA expression profile similarity computation that has been already published? If yes, what is your experience using them?
  
  **Tags:** rna-seq
  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_1 on 2017-06-04. Score: 5
  
  There's a [new paper](http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1004075) that's just appeared on the subject of "proportionality", including a method by which RNA expression might be compared.
  
  This is a new concept for me, and the article is not easy enough for me to read that I can write up a quick summary; the authors don't seem to devote a section in the paper to defining "proportionality". However, here's an interesting chunk from the article:
  
  > We graphed the network of relationships between these mRNAs (S5 Fig.), an approach similar to gene co-expression network [12] or weighted gene co-expression analysis [13] but founded on proportionality and therefore valid for relative data. The network revealed one cluster of 96, and many other smaller clusters of mRNAs behaving proportionally across conditions.  
  > ...  
  > We are also keen to raise awareness that correlation (and other statistical methods that assume measurements come from real coordinate space) should not be applied to relative abundances. This is highly relevant to gene coexpression networks [12]. Correlation is at the heart of methods like Weighted Gene Co-expression Network Analysis [13] and heatmap visualization [14]. These methods are potentially misleading if applied to relative data.
  
  > **Comments:**
  > **username_2**: proportionality is defined as (1+ beta^2 - 2*beta*abs(r))  where r is the Pearson correlation of log x and log y, and beta^2 is var(log(y))/var(log(x)).   The authors also note: "Proportionality is appropriate, but ϕ does not satisfy the properties of a distance—most obviously, it is not symmetric unless β = 1". They go on to say "Hence, our approach to forming a dissimilarity matrix is simply to work with ϕ(log xi, log xj) where i < j, in effect, the lower triangle of the matrix of ϕ values between all pairs of components."  
  > **username_1**: But why is it called "proportionality" (in non-formulaic terms)? What is the effect of that formula on expression values? Can you fill in the remainder of this sentence: "Proportionality is a measure that represents how the expression of two ..."  
  > **username_2**: No, not really - I didn't quite understand the use of proportionality. Although I do note that phi is not a measure of proportionality, but a measure of goodness-of-fit to proportionality.  
  > **username_3**: Proportionalty implies the ratio of expression values of two genes will be a constant. Instead of fitting $y=mx+c$ in regression, proportionalty fits $y=mx$  
  
  ---
  
  ### Answer by username_2 on 2017-06-06. Score: 5
  
  The suggestion to use proportionality is probably the correct one if you are interested in similar *patterns* between samples. But not if you are interested in *absolute* differences.
  
  For example: the following two samples are similar in pattern, but not similar on absolute levels:
  
  ```
          Sample 1    Sample 2    Sample 3
  Gene A    10          100           80
  Gene B     8           80          100  
  Gene C    12          120          120
  
  ```
  
  Samples 1 and 2 have a perfect proportionality (phi is 0) and also a perfect correlation (as a side note any pair with a perfect proportionality will always have a perfect correlation). However, in terms of logfold changes, samples 2 and 3 are more like each other.
  
  Of course in real life you would never see a comparison like the sample 1 - sample 2 one because normalisation would have removed the scale difference. This was exactly the point brought up by the proportionality paper. But normalisation methods don't normally guarantee that the sum of expression for each sample is identical, and such differences can still occur.
  
  An alternative that might suit more in the second case is either the euclidean distance between the samples, or the euclidean distance on the first two components of a principle component or multi-dimensional scaling. The latter is effectively using mean logFC between samples.
  
  Assuming that `x` is a matrix containing normalised, log transformed expression values, you could use R and limma to calculate distance in multi-dimensional scaled space as follows:
  
  ```
  library(limma)
  mds <- plotMDS(x, plot=FALSE)
  mds <- data.frame(mds$x, mds$y)
  distances <- dist(mds)
  
  ```
  
  > **Comments:**
  > **username_1**: Normalisation would only remove the scale difference if it were consistent across a large proportion of genes. In the example you have given, this would indeed be the case if only the indicated genes were assayed, but with this low number of genes it would be more common to normalise based on a housekeeping gene rather than the total gene expression.  
  > **username_2**: Yes, this was my point - hence why there might be reason to use some distance measure (such as euclidean distance) that measured scale difference rather than pattern differences.  
  
  ---
  
  ### Answer by username_1 on 2017-06-04. Score: 3
  
  If you have a lot of samples with very different environmental conditions, then a Weighted Gene Correlation Network Analysis ([WGCNA](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2631488/)) might be appropriate.
  
  This type of analysis looks for genes that trace similar (or opposing) expression patterns throughout different conditions (e.g. high-medium-medium-low-absent-high would be highly negatively correlated with low-medium-medium-high-very high-low).
  
  That particular paper introduces the concept of "modules", which are groups of genes that share similar expression patterns. Functions are available for plotting how the expression of canonical module members changes throughout the different conditions, and for identifying which module (or modules) a particular gene is likely to be a member of.
  
  WGCNA works best when there's a lot of different expression changes in the different conditions, which sounds like it would fit well with your project. However, it concentrates more on the genes, rather than the conditions (which seems like it would be less useful for you).
  
  
  ---
  
  ### Answer by username_4 on 2017-06-06. Score: 2
  
  The euclidean distance is probably the simplest, both conceptually and in terms of implementation. It's by no means an elegant solution, and it may not perform well in certain circumstances.
  
  Euclidean distance is easiest to conceptualize as the distance between two points in a two dimensional space.
  
  ```
  Y
  ^
  |
  |
  |
  |            * p = (3, 3)
  |
  |
  |
  |        * q = (2, 1)
  |
  ----------------------------> X
  
  ```
  
  In this example, the distance between the two points is
  
  ```
  d(p, q) = sqrt( (p_x-q_x)^2 + (p_y-q_y)^2 )
          = sqrt( (2-3)^2 + (1-3)^2 )
          = sqrt(5)
          ≈ 2.24
  
  ```
  
  For a gene expression profile with two genes, this is exactly how the euclidean distance would be calculated, using expression values from one gene as the X axis and expression values from the other gene as the Y axis. Realistically, though, gene expression profiles typically contain thousands or tens of thousands of genes, so instead we use the generalization of the distance calculation for N dimensions.
  
  ```
  d(p, q) = sqrt( (p_1-q_1)^2 + (p_2-q_2)^2 + ... + (p_N-q_N)^2 )
  
  ```
  
  Packages for R and Python make these types of calculations trivial once you have the data loaded into the correct data structure. See Ian's answer for some example R code.
  
  
  ---
  
  