Answers.Body: <p>Below is the answer for standard normalisation via the z-score. This is RNAseq data and that gets complicated because you'll need an internal standardisation per sample because variation can occur between runs. What we appear to be talking about here is a standardisation over a standardisation. This needs more information about the experimental conditions of the testing because this could result in an erroneous transformation.</p>
  <p>Standardisation through a timeseries per patient remains okay however - its difficult to go wrong on that.</p>
  <p><em>Otherwise I'd wait for the input of @RamRS, he may not be around for Christmas.</em></p>
  <hr />
  <p>You wrote:</p>
  <blockquote>
  <p>2 The μ is calculated per group or from all samples (controls and patients togetherץ</p>
  </blockquote>
  <p>Its per group but if its a longitudinal study its per patient (below). As described in point 2 be very clear about you are standardising and why; thats the key to the calculation. You are standardising for an uncontrolled parameter BUT the group (cohort) has something different about it e.g. it's not the drug you are testing but the conditions of the testing. For example, group 1 and group 2 were tested at different times by different people THEN standarisation is useful.</p>
  <p>If its just lets standardise stuff when group 1 and group 2 are really identical except for the thing in the test (e.g. a drug vs a placebo); in that scenario standisation needs very careful thought.</p>
  <hr />
  <ol>
  <li>To answer the question directly, if you have one patient with longitudinal data then the mean (μ) and standard deviation (σ) will be for the entire longitudinal group for that given patient. x will be for each individual  longitudinal reading. You tend to remove points that are great than 3σ</li>
  </ol>
  <p>What you are doing is standardising that given patient so you can now compare values between patients. For example, the time/date of the readings will often differ for different patients and e.g. maybe environmental temperature plays a role ... that sort of thing.</p>
  <hr />
  <ol start="2">
  <li>To be clear in the example above you don't take μ for the entire cohort - in a longitudinal study its per patient. If the study is not longitudinal enough e.g. identically performed cohorts - but each cohort differs in some uncontrolled way to the other cohort then the  μ is for a given cohort, but you need to think carefully what you are standardising.</li>
  </ol>
  <hr />
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-23T16:33:50.593Z
Answers.Text: Hi @AdiGershon if you update yoyr post. This isnt a trivial experiment **also** all the heterogeneiry in sampling and experimental sensitivity in time. Its impossible to simulataneoulsy sample 2000 patients and i doubt the same person did that. At present a study on that scale requires serious attention. I am not entirely sure its doable via this format
Answers.UserDisplayName: M__
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-24T21:07:26.633Z
Answers.Text: Thank you for you comment. I understand it's more complicated then I thought
Answers.UserDisplayName: Adi Gershon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-22T18:46:46.333Z
Answers.LastActivityDate: 2024-12-22T19:36:02.567Z
Answers.LastEditDate: 2024-12-22T19:36:02.567Z
Answers.UserDisplayName: M__
Answers.Body: <p>Please provide more context around this problem; there's not enough information here to give a good answer. For example, how many genes are you looking at - was this a whole-transcriptome study, or a targeted study? How was the cDNA sequencing data generated?</p>
  <p>In particular, you shouldn't be using a straight Z-score for RNA/cDNA sequencing data because it's not a normal curve fit in linear space; it is best done using a properly-modeled fit curve. See here for how to process RNA/cDNA sequencing data using DESeq2:</p>
  <p><a href="https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#quick-start" rel="nofollow noreferrer">https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#quick-start</a></p>
  <p>Note that DESeq2 <em>requires</em> biological replicates, but you appear to have them in this case: ~800 control replicates, and ~1800 non-control replicates. Given that DESeq2 is able to handle single-cell data when used by Seurat, there's probably a way to work around scale issues if they exist.</p>
  <p>However, DESeq2 also expects that the majority of observed genes are not differentially expressed (as is typical for a whole-transcriptome study), which allows it to model the expected expression distribution in order to identify outliers. It won't work for single genes, and is unlikely to work for targeted panels where the majority of genes are differentially expressed. There's a bit of discussion around modeled dispersion <a href="https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#dispersion-plot-and-fitting-alternatives" rel="nofollow noreferrer">here</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-23T21:55:59.060Z
Answers.Text: Its assumesd the OP is fitting post normalisation. Z-scores are a standard approach to patient cohorts. One issue is the double transformation, but double transformations happen. This is a full scale stage 2 clinical trial, its alot more complicated than described here. It would be against my contract to help, because this is a major big bucks project UNLESS they have a small trial and are scooping the rest of the data from the public domain
Answers.UserDisplayName: M__
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-24T02:07:06.753Z
Answers.Text: I don't think post-normalisation is a reasonable assumption to make. Adi mentioned, "I'm new to this", and that they have count data (which is not normalized).
Answers.UserDisplayName: gringer
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-24T02:41:14.180Z
Answers.Text: Maybe but there must statistical expertise on a project of this size
Answers.UserDisplayName: M__
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-24T09:05:31.957Z
Answers.Text: I agree; this is an XY problem, and therefore a direct answer to the question asked is unlikely to fix the actual problem.
Answers.UserDisplayName: gringer
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-24T21:06:15.010Z
Answers.Text: Thank you for all the advises I did try to do it with deseq2. I will try to understand it more deeply before doing the analysis. Thank you.
Answers.UserDisplayName: Adi Gershon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-24T21:17:09.893Z
Answers.Text: It's not a clinical trial.
Answers.UserDisplayName: Adi Gershon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-25T01:05:45.007Z
Answers.Text: @AdiGershon I've updated my answer a little bit. If you've got 800 controls and 1800 patients, then you *do* have replicates.
Answers.UserDisplayName: gringer
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-23T21:15:40.130Z
Answers.LastActivityDate: 2024-12-25T01:14:27.073Z
Answers.LastEditDate: 2024-12-25T01:14:27.073Z
Answers.UserDisplayName: gringer
Body: <p>I'm new to this and I can not find information about this:
  i have rna seq- I created count data
  now I have 2 groups controls and patients.
  now I wanted to calculate z scores from this data
  and I wanted to know what is the right way to do it:</p>
  <ol>
  <li>do I use the z = (x – μ) / σ directly on the count data</li>
  <li>2 The μ is calculated per group or from all samples (controls and patients togetherץ</li>
  </ol>
  <p>I have RNA seq of different individuals (this is patients rna seq) with no replicates. the sequencing was done with the same method for all of the samples. I want to look at the differences in expression between both groups and because of that I wanted to calculate z score. in this case do I need to calculate the μ per group?</p>
  <p>I have ~800 controls and ~1800 patients; but it's not a clinical trial.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-12-21T22:38:55.633Z
Comments.Text: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.
Comments.UserDisplayName: Community
Tags: <r><rna-seq><normalization>
Title: Calculating Z-score from count data
ThreadText: # Calculating Z-score from count data
  
  (Asked by: username_0 on 2024-12-21)
  
  I'm new to this and I can not find information about this:
  i have rna seq- I created count data
  now I have 2 groups controls and patients.
  now I wanted to calculate z scores from this data
  and I wanted to know what is the right way to do it:
  
  1. do I use the z = (x – μ) / σ directly on the count data
  2. 2 The μ is calculated per group or from all samples (controls and patients togetherץ
  
  I have RNA seq of different individuals (this is patients rna seq) with no replicates. the sequencing was done with the same method for all of the samples. I want to look at the differences in expression between both groups and because of that I wanted to calculate z score. in this case do I need to calculate the μ per group?
  
  I have ~800 controls and ~1800 patients; but it's not a clinical trial.
  
  **Tags:** r, rna-seq, normalization
  
  > **Comments:**
  > **username_1**: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2024-12-22. Score: 0
  
  Below is the answer for standard normalisation via the z-score. This is RNAseq data and that gets complicated because you'll need an internal standardisation per sample because variation can occur between runs. What we appear to be talking about here is a standardisation over a standardisation. This needs more information about the experimental conditions of the testing because this could result in an erroneous transformation.
  
  Standardisation through a timeseries per patient remains okay however - its difficult to go wrong on that.
  
  *Otherwise I'd wait for the input of @RamRS, he may not be around for Christmas.*
  
  ---
  
  You wrote:
  
  > 2 The μ is calculated per group or from all samples (controls and patients togetherץ
  
  Its per group but if its a longitudinal study its per patient (below). As described in point 2 be very clear about you are standardising and why; thats the key to the calculation. You are standardising for an uncontrolled parameter BUT the group (cohort) has something different about it e.g. it's not the drug you are testing but the conditions of the testing. For example, group 1 and group 2 were tested at different times by different people THEN standarisation is useful.
  
  If its just lets standardise stuff when group 1 and group 2 are really identical except for the thing in the test (e.g. a drug vs a placebo); in that scenario standisation needs very careful thought.
  
  ---
  
  1. To answer the question directly, if you have one patient with longitudinal data then the mean (μ) and standard deviation (σ) will be for the entire longitudinal group for that given patient. x will be for each individual longitudinal reading. You tend to remove points that are great than 3σ
  
  What you are doing is standardising that given patient so you can now compare values between patients. For example, the time/date of the readings will often differ for different patients and e.g. maybe environmental temperature plays a role ... that sort of thing.
  
  ---
  
  2. To be clear in the example above you don't take μ for the entire cohort - in a longitudinal study its per patient. If the study is not longitudinal enough e.g. identically performed cohorts - but each cohort differs in some uncontrolled way to the other cohort then the μ is for a given cohort, but you need to think carefully what you are standardising.
  
  ---
  
  > **Comments:**
  > **username_2**: Hi @username_0 if you update yoyr post. This isnt a trivial experiment **also** all the heterogeneiry in sampling and experimental sensitivity in time. Its impossible to simulataneoulsy sample 2000 patients and i doubt the same person did that. At present a study on that scale requires serious attention. I am not entirely sure its doable via this format  
  > **username_0**: Thank you for you comment. I understand it's more complicated then I thought  
  
  ---
  
  ### Answer by username_3 on 2024-12-23. Score: 0
  
  Please provide more context around this problem; there's not enough information here to give a good answer. For example, how many genes are you looking at - was this a whole-transcriptome study, or a targeted study? How was the cDNA sequencing data generated?
  
  In particular, you shouldn't be using a straight Z-score for RNA/cDNA sequencing data because it's not a normal curve fit in linear space; it is best done using a properly-modeled fit curve. See here for how to process RNA/cDNA sequencing data using DESeq2:
  
  <https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#quick-start>
  
  Note that DESeq2 *requires* biological replicates, but you appear to have them in this case: ~800 control replicates, and ~1800 non-control replicates. Given that DESeq2 is able to handle single-cell data when used by Seurat, there's probably a way to work around scale issues if they exist.
  
  However, DESeq2 also expects that the majority of observed genes are not differentially expressed (as is typical for a whole-transcriptome study), which allows it to model the expected expression distribution in order to identify outliers. It won't work for single genes, and is unlikely to work for targeted panels where the majority of genes are differentially expressed. There's a bit of discussion around modeled dispersion [here](https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#dispersion-plot-and-fitting-alternatives).
  
  > **Comments:**
  > **username_2**: Its assumesd the OP is fitting post normalisation. Z-scores are a standard approach to patient cohorts. One issue is the double transformation, but double transformations happen. This is a full scale stage 2 clinical trial, its alot more complicated than described here. It would be against my contract to help, because this is a major big bucks project UNLESS they have a small trial and are scooping the rest of the data from the public domain  
  > **username_3**: I don't think post-normalisation is a reasonable assumption to make. Adi mentioned, "I'm new to this", and that they have count data (which is not normalized).  
  > **username_2**: Maybe but there must statistical expertise on a project of this size  
  > **username_3**: I agree; this is an XY problem, and therefore a direct answer to the question asked is unlikely to fix the actual problem.  
  > **username_0**: Thank you for all the advises I did try to do it with deseq2. I will try to understand it more deeply before doing the analysis. Thank you.  
  > **username_0**: It's not a clinical trial.  
  > **username_3**: @username_0 I've updated my answer a little bit. If you've got 800 controls and 1800 patients, then you *do* have replicates.  
  
  ---
  
  