How to correctly do a nested test (t-test, rank-sum, etc) Let's say there are two groups of participants. Each group is randomly split into control subgroup A and task subgroup B. For each of the 4 cases some random variable $x$ is measured, whose mean is denoted by $\mu$. I would like to perform 3 tests:

*

*$H_0 : \mu_{A, Group1} = \mu_{B, Group1}$

*$H_0 : \mu_{A, Group2} = \mu_{B, Group2}$

*$H_0 : \mu_{Group1} = \mu_{Group2}$ regardless of subgroups

I sketch the tests below

Question: What is the correct way to perform these tests? If I perform 3 independent t-tests, is it fair to apply Bonferroni correction to their p-values, or must I do a more sophisticated multiple comparisons correction. If making independent tests is not a robust procedure, what is? I want to do all 3 tests, so as far as I understand I don't want two-way ANOVA.

If you pool the observations, let's call your variable on the vertical axis y
regress y on a constant, a dummy Group (1 if observation in group 1, 0 otherwise), A (A is another dummy variable, A=1 if observation in subgroup A, 0 otherwise) and a cross effect Group*A (Group multiplied by A).
Y = Constant+ alpha Group + beta A + gamma Group*A
Constant is the mean in Group=0 (the second group), subgroup B, then:

*

*alpha is the test-statistic comparing the test outcomes in the two groups 1 and 2 (your third Ho),

*beta is the test comparing subgroup A and B in Group 2 (your second Ho),

*gamma is the test statistics of comparing subgroups A and B in Group 1 (your first Ho).

The F-test of the regression tests that alpha, beta, and gamma are jointly 0.
I hope this helps
