Answers.Body: <p>The key point here is whether or not the values are approximately normally distributed and whether any transformations can be applied to make them so. </p>
  
  <p>For a t-test, the most important thing is that there is no relationship between the mean and the variance. </p>
  
  <p>Its also correct that you don't want to divide by the mean of the controls. You want to retain all the information. </p>
  
  <p>So here is how i'd start, first calculate the mean and standard deviation of each of your 18 groups (17 treatments + control), and plot this on a scatter plot (i.e. mean on X, SD on Y).</p>
  
  <p>Is there a relationship between the two?</p>
  
  <p>Repeat the exercise with log transformed values. Is the relationship more or less.</p>
  
  <p>Pick the set of values that has the weakest mean-SD relationship. I'm guessing the log transformed might well be better.</p>
  
  <p>If $X = (x_1, x_2, x_3)$ is your controls and $Y_i = (y_{i,1}, y_{i,2}, y_{i,3})$ are the treated samples from treatment i, to test the difference between treatment and control, do a t-test on X and Y (not <code>T/mean(c)</code>)</p>
  
  <p>for example in R</p>
  
  <pre><code>X = c(x_1, x_2, x_3)
  Y_1 = c(y_11, y_12, y_13)
  t.test (X, Y_1)
  </code></pre>
  
  <p>or in excel is if the three values for X are in column A and the three values for $Y_1$ in column B</p>
  
  <pre><code>=T.TEST(A1:A3,B1:B3,2,3)
  </code></pre>
  
  <p>If you are going to do 17 of these, remember to do a multiple testing correction by multiplying the resulting p-values by 17. </p>
  
  <p>If you want to compare treatment 1 ($Y_1$) vs treatment 2 ($Y_2$) do the comparison directly, not taking the control into account:</p>
  
  <pre><code>t.test(Y_1, Y_2)
  </code></pre>
  
  <p>Remember that if you use log transformed values, you calculate log fold changes using <code>mean(Y)-mean(X)</code> rather than <code>mean(Y)/mean(X)</code></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-18T10:56:05.363Z
Answers.LastActivityDate: 2017-08-19T17:55:22.827Z
Answers.LastEditDate: 2017-08-19T17:55:22.827Z
Answers.UserDisplayName: Ian Sudbery
Answers.Body: <p>Your null hypothesis would be that the fold-changes are 0, so you can either do the T-test accordingly or simply do away with the fold changes and perform the T-test between the raw values in the group (this is preferable to performing a T-test of one group vs. 0, since it allows you to assess the expected variability around 0). Note, however, that 3 samples is pretty much the bare minimum needed for any statistics, so your power will be terrible and your estimation of variance will likely be pretty inaccurate. Consequently, take any p-values with an appropriate grain of salt.</p>
  
  <p>To my knowledge there aren't any good alternatives for your situation without having a good expected background distribution.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-18T06:51:59.027Z
Answers.LastActivityDate: 2017-08-18T06:51:59.027Z
Answers.UserDisplayName: Devon Ryan
Answers.Body: <p>You can't calculate a p-value on the fold-change values, you need to use the  concentrations in triplicate thus giving a measure of the variance for the t-test to use. </p>
  
  <p>t-test assumes your data are normally distributed, if they aren't you're going to get spurious p-values. If you aren't sure a non-parametric test like Wilcoxon is better. It will be less sensitive although with only 3 replicates your experiment is low powered anyway.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-18T09:09:19.210Z
Answers.Text: With N=3 per group a Wilcoxon won't give a p-value below 0.1 for a two-sided test or 0.05 for a 1-sided test.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-18T09:18:13.580Z
Answers.Text: @DevonRyan ah yes. Good point!
Answers.UserDisplayName: ithinkiam
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-18T08:58:29.290Z
Answers.LastActivityDate: 2017-08-18T08:58:29.290Z
Answers.UserDisplayName: ithinkiam
Answers.Body: <p>So to get a p value you must have one null hypothesis like the fold changes are significant or not. Then you test the hypothesis against the data and calculate the p value (likeliness or unlikeliness) for a given threshold.</p>
  
  <p>In your case, t test is good enough to calculate the p-value.</p>
  
  <p><a href="http://willett.ece.wisc.edu/wp-uploads/2016/01/05b-TandP.pdf" rel="nofollow noreferrer">http://willett.ece.wisc.edu/wp-uploads/2016/01/05b-TandP.pdf</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-18T05:08:01.100Z
Answers.LastActivityDate: 2017-08-18T05:08:01.100Z
Answers.UserDisplayName: arup
Body: <p>I'm currently working with data from a Luminex multiplex assay. In this assay, the concentrations of 17 different analyte proteins were identified in 12 groups in triplicate. One of these 17 groups was used as the control, and the log2 fold changes were calculated for the analyte concentration of each sample in each group using the average control concentration for that analyte.</p>
  
  <p>However, now I would like to calculate a p-value for the identified fold changes if possible.</p>
  
  <p>My current preliminary idea is to perform the t test for each group compared with the control group for each analyte, but I don't think that my group sizes are large enough for this. My statistical knowledge is lacking, so if there is a better way of calculating these p-values I don't know about it.</p>
  
  <p>Is there a better way of obtaining these p-values?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-08-18T08:54:20.083Z
Comments.Text: For future reference, basic statistics questions like this should probably go on [cross-validated](https://stats.stackexchange.org). This one is so basic, though, that it was faster to simply answer it here than to move it.
Comments.UserDisplayName: Devon Ryan
Tags: <r>
Title: How to calculate p-values for fold changes?
ThreadText: # How to calculate p-values for fold changes?
  
  (Asked by: username_0 on 2017-08-18)
  
  I'm currently working with data from a Luminex multiplex assay. In this assay, the concentrations of 17 different analyte proteins were identified in 12 groups in triplicate. One of these 17 groups was used as the control, and the log2 fold changes were calculated for the analyte concentration of each sample in each group using the average control concentration for that analyte.
  
  However, now I would like to calculate a p-value for the identified fold changes if possible.
  
  My current preliminary idea is to perform the t test for each group compared with the control group for each analyte, but I don't think that my group sizes are large enough for this. My statistical knowledge is lacking, so if there is a better way of calculating these p-values I don't know about it.
  
  Is there a better way of obtaining these p-values?
  
  **Tags:** r
  
  > **Comments:**
  > **username_1**: For future reference, basic statistics questions like this should probably go on [cross-validated](https://stats.stackexchange.org). This one is so basic, though, that it was faster to simply answer it here than to move it.  
  
  ---
  
  ## 4 answers
  
  ### Answer by username_2 on 2017-08-18. Score: 4
  
  The key point here is whether or not the values are approximately normally distributed and whether any transformations can be applied to make them so.
  
  For a t-test, the most important thing is that there is no relationship between the mean and the variance.
  
  Its also correct that you don't want to divide by the mean of the controls. You want to retain all the information.
  
  So here is how i'd start, first calculate the mean and standard deviation of each of your 18 groups (17 treatments + control), and plot this on a scatter plot (i.e. mean on X, SD on Y).
  
  Is there a relationship between the two?
  
  Repeat the exercise with log transformed values. Is the relationship more or less.
  
  Pick the set of values that has the weakest mean-SD relationship. I'm guessing the log transformed might well be better.
  
  If $X = (x\_1, x\_2, x\_3)$ is your controls and $Y\_i = (y\_{i,1}, y\_{i,2}, y\_{i,3})$ are the treated samples from treatment i, to test the difference between treatment and control, do a t-test on X and Y (not `T/mean(c)`)
  
  for example in R
  
  ```
  X = c(x_1, x_2, x_3)
  Y_1 = c(y_11, y_12, y_13)
  t.test (X, Y_1)
  
  ```
  
  or in excel is if the three values for X are in column A and the three values for $Y\_1$ in column B
  
  ```
  =T.TEST(A1:A3,B1:B3,2,3)
  
  ```
  
  If you are going to do 17 of these, remember to do a multiple testing correction by multiplying the resulting p-values by 17.
  
  If you want to compare treatment 1 ($Y\_1$) vs treatment 2 ($Y\_2$) do the comparison directly, not taking the control into account:
  
  ```
  t.test(Y_1, Y_2)
  
  ```
  
  Remember that if you use log transformed values, you calculate log fold changes using `mean(Y)-mean(X)` rather than `mean(Y)/mean(X)`
  
  
  ---
  
  ### Answer by username_1 on 2017-08-18. Score: 3
  
  Your null hypothesis would be that the fold-changes are 0, so you can either do the T-test accordingly or simply do away with the fold changes and perform the T-test between the raw values in the group (this is preferable to performing a T-test of one group vs. 0, since it allows you to assess the expected variability around 0). Note, however, that 3 samples is pretty much the bare minimum needed for any statistics, so your power will be terrible and your estimation of variance will likely be pretty inaccurate. Consequently, take any p-values with an appropriate grain of salt.
  
  To my knowledge there aren't any good alternatives for your situation without having a good expected background distribution.
  
  
  ---
  
  ### Answer by username_3 on 2017-08-18. Score: 1
  
  You can't calculate a p-value on the fold-change values, you need to use the concentrations in triplicate thus giving a measure of the variance for the t-test to use.
  
  t-test assumes your data are normally distributed, if they aren't you're going to get spurious p-values. If you aren't sure a non-parametric test like Wilcoxon is better. It will be less sensitive although with only 3 replicates your experiment is low powered anyway.
  
  > **Comments:**
  > **username_1**: With N=3 per group a Wilcoxon won't give a p-value below 0.1 for a two-sided test or 0.05 for a 1-sided test.  
  > **username_3**: @username_1 ah yes. Good point!  
  
  ---
  
  ### Answer by username_4 on 2017-08-18. Score: 0
  
  So to get a p value you must have one null hypothesis like the fold changes are significant or not. Then you test the hypothesis against the data and calculate the p value (likeliness or unlikeliness) for a given threshold.
  
  In your case, t test is good enough to calculate the p-value.
  
  <http://willett.ece.wisc.edu/wp-uploads/2016/01/05b-TandP.pdf>
  
  
  ---
  
  