Answers.Body: <p>As you suspected, correlations will be heavily changed by such a transformation. The reason for this is also exactly as you described, namely that using ranks rather than values for the transformation eliminates all of the information needed to actually preserve these relationships. As an example, suppose we have a single observation with rank 1 across all observations that none the less shows a decrease in absolute value across them (i.e., there's a correlation with a negative slope over time). Such a transformation will result in a correlation of 0.</p>
  
  <p>One might be able to find a different transformation that preserves this relationship, or might be able to simply pool all of the time points together before computing the ranks (this would then at least partially preserve some correlations, though depending on what's needed to properly normalize the data it may cause other problems), but in general there's no guarantee that this is possible.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-13T11:03:14.763Z
Answers.LastActivityDate: 2019-11-13T11:03:14.763Z
Answers.UserDisplayName: Devon Ryan
Answers.Body: <p>It appears the original observations were not a timeseries and the OP is specifically asking about application to timeseries data. </p>
  
  <p>In my experience these are two separate areas of analysis because a temporal skew may result in aggregate behaviour which violates the original transformation. For example hot weather versus cold weather. </p>
  
  <p>What the OP appears to be doing is normalising each point in the time series, albeit I don't know what X represents. Personally I have doubts because the aggregate correlation is across all time points and one set of aggregrates, e.g. Ecoregion 1 is compared with another e.g. Ecoregion 2. This for example would be done via linear modelling and assessement of fit performed through many approaches, e.g. the residual is just one approach.</p>
  
  <p>If data has large temporal skews your approach is unlikely to work, ultimately you don't know the variation in the time series which could far more heterogeneous than the agrregrate data. Seasonality is likely a major factor in your data and I would consider this in the first instance. There are lots of ways to transform a time series, standardisation is a classic approach. Again my personal experience is these are two separate areas of statistics and directly applying one to the other is something that should be assessed with care.</p>
  
  <hr>
  
  <p>So just to try and explain in this scenario terms, when its cold if the variation in bacterioplankton is much greater than when its hot (smaller population sizes increases variance), i.e. at different times of year then your approach can't work because normalisation is about a fixed (constant) relationship between mean and standard deviation.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-11T15:13:37.313Z
Answers.LastActivityDate: 2020-04-11T22:37:28.397Z
Answers.LastEditDate: 2020-04-11T22:37:28.397Z
Answers.UserDisplayName: M__
Body: <p>This question is related to a normal score transformation which is performed in the paper by
  RUAN, Quansong, et al. Local similarity analysis reveals unique associations among marine bacterioplankton species and environmental factors. Bioinformatics, 2006, 22.20: 2532-2538.</p>
  
  <p>They claim that given a variable X with observations <span class="math-container">$x= (x_1,\ldots,x_n)$</span> that are not normally distributed, normal score transformation is performed by first creating a rank vector observations <span class="math-container">$x$</span>, <span class="math-container">$R^x = (R_1^x,\ldots, R_n^x)$</span> and then using and then transforming to <span class="math-container">$x'=(x_1',\ldots,x_n)$</span>, by <span class="math-container">$x_i\ = \phi_{-1} (\frac{R_i^x}{n+1} )$</span> where <span class="math-container">$\phi_{-1}$</span> is the inverse of cummulative normal distribution. </p>
  
  <p>It's intuitively clear (but maybe a bit harder to prove?) that resulting variable <span class="math-container">$X'$</span> has observations normally distributed, but I have a question about time series dependence. If we had that observations <span class="math-container">$x$</span> were correlated, say, over 4 adjacent observations, , i.e. <span class="math-container">$x_{n+1} = f(x_{n},x_{n-1},x_{n-2})$</span>, how does this change after the normal transformation? Can we still find a function g, so that <span class="math-container">$x'_{n+1} = g(x'_{n},x'_{n-1},x'_{n-2})$</span> would hold? Since we're transforming based on the rank of data, I expect this to be not possible?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-11-16T04:06:17.510Z
Comments.Text: I'm voting to close this question as off-topic because is it general stats and beliongs in a different forum
Comments.UserDisplayName: Bioathlete
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-11T13:40:42.710Z
Comments.Text: This is biostatistics, the fact the was published in Bioinforamtics is an indication. The problem with the question is that none of the variables are defined so it is impossible to answer.
Comments.UserDisplayName: M__
Tags: <r><normalization>
Title: Normal score transformation
ThreadText: # Normal score transformation
  
  (Asked by: username_0 on 2019-11-12)
  
  This question is related to a normal score transformation which is performed in the paper by
  RUAN, Quansong, et al. Local similarity analysis reveals unique associations among marine bacterioplankton species and environmental factors. Bioinformatics, 2006, 22.20: 2532-2538.
  
  They claim that given a variable X with observations $x= (x\_1,\ldots,x\_n)$ that are not normally distributed, normal score transformation is performed by first creating a rank vector observations $x$, $R^x = (R\_1^x,\ldots, R\_n^x)$ and then using and then transforming to $x'=(x\_1',\ldots,x\_n)$, by $x\_i\ = \phi\_{-1} (\frac{R\_i^x}{n+1} )$ where $\phi\_{-1}$ is the inverse of cummulative normal distribution.
  
  It's intuitively clear (but maybe a bit harder to prove?) that resulting variable $X'$ has observations normally distributed, but I have a question about time series dependence. If we had that observations $x$ were correlated, say, over 4 adjacent observations, , i.e. $x\_{n+1} = f(x\_{n},x\_{n-1},x\_{n-2})$, how does this change after the normal transformation? Can we still find a function g, so that $x'\_{n+1} = g(x'\_{n},x'\_{n-1},x'\_{n-2})$ would hold? Since we're transforming based on the rank of data, I expect this to be not possible?
  
  **Tags:** r, normalization
  
  > **Comments:**
  > **username_1**: I'm voting to close this question as off-topic because is it general stats and beliongs in a different forum  
  > **username_2**: This is biostatistics, the fact the was published in Bioinforamtics is an indication. The problem with the question is that none of the variables are defined so it is impossible to answer.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2019-11-13. Score: 1
  
  As you suspected, correlations will be heavily changed by such a transformation. The reason for this is also exactly as you described, namely that using ranks rather than values for the transformation eliminates all of the information needed to actually preserve these relationships. As an example, suppose we have a single observation with rank 1 across all observations that none the less shows a decrease in absolute value across them (i.e., there's a correlation with a negative slope over time). Such a transformation will result in a correlation of 0.
  
  One might be able to find a different transformation that preserves this relationship, or might be able to simply pool all of the time points together before computing the ranks (this would then at least partially preserve some correlations, though depending on what's needed to properly normalize the data it may cause other problems), but in general there's no guarantee that this is possible.
  
  
  ---
  
  ### Answer by username_2 on 2020-04-11. Score: 0
  
  It appears the original observations were not a timeseries and the OP is specifically asking about application to timeseries data.
  
  In my experience these are two separate areas of analysis because a temporal skew may result in aggregate behaviour which violates the original transformation. For example hot weather versus cold weather.
  
  What the OP appears to be doing is normalising each point in the time series, albeit I don't know what X represents. Personally I have doubts because the aggregate correlation is across all time points and one set of aggregrates, e.g. Ecoregion 1 is compared with another e.g. Ecoregion 2. This for example would be done via linear modelling and assessement of fit performed through many approaches, e.g. the residual is just one approach.
  
  If data has large temporal skews your approach is unlikely to work, ultimately you don't know the variation in the time series which could far more heterogeneous than the agrregrate data. Seasonality is likely a major factor in your data and I would consider this in the first instance. There are lots of ways to transform a time series, standardisation is a classic approach. Again my personal experience is these are two separate areas of statistics and directly applying one to the other is something that should be assessed with care.
  
  ---
  
  So just to try and explain in this scenario terms, when its cold if the variation in bacterioplankton is much greater than when its hot (smaller population sizes increases variance), i.e. at different times of year then your approach can't work because normalisation is about a fixed (constant) relationship between mean and standard deviation.
  
  
  ---
  
  