Correct inference on hierarchical data I am doing an experiment on cell cultures comparing how some treatments affect the parameter of the cells. I have 3 replicates of the culture for each treatment, and in each culture, I measure the parameter of the interest in multiple cells (n is different for different cultures). Now, I want to infer if my treatment had an effect on the studied parameter in the cells. What would be the correct way to statistically support the hypothesis that treatment had an effect?
In particular, the problem I am not able to understand is how to correctly deal with hierarchical data (in this case I have 3 replicates of cultures and some number of replicates (individual) in each culture). I see two options available:

*

*Mix all the data together and test group differences like there are no different cultures.

*Average the data through the cultures and compare means between groups (in this case though I would get a low number of observations, only n=3)

*I heard something about mixed models, but I am not sure it is applicable here.


Quoting from a Technical Perspective by the Pollards,* which I strongly recommend:

Many statistical tests further assume that observations are independent. When this is not the case, as with paired or repeated measurements on the same specimen, one should use methods that account for correlated observations, such as paired t tests or mixed model regression analysis with random effects.


Biological replicates (measurements on separate samples) are used for parameter estimates and statistical tests, because they allow one to describe variation in the population. Technical replicates (multiple measurements on the same sample) are used to improve estimation of the measurement for each biological replicate. Treating technical replicates as biological replicates is called pseudoreplication and often produces low estimates of variance and erroneous test results.

So fundamentally your analysis is based on 3 biological replicates for each treatment. Your cells within cultures need to be treated as technical replicates or random effects. Yes, that's not a lot of biological replicates, but it might be adequate in a well controlled experiment. Quoting further:

Fortunately, statistical analysis in experimental biology has two major advantages over observational biology. First, experimental conditions are often well controlled, for example using genetically identical organisms under laboratory conditions or administering a precise amount of a drug. This reduces the variation between samples and compensates to some extent for small sample sizes. Second, experimentalists can randomize the assignment of treatments to their specimens and therefore minimize the influence of confounding variables. Nonetheless, small numbers of observations make it difficult to verify important assumptions and can compromise the interpretation of an experiment.

So the trick is to get as much information as you can, in an unbiased way, from the data that you have. If you have more than half a dozen or so cells analyzed within each culture, you will probably get the most information out of the study by treating them as random effects in a mixed model. This page, its links to other sites, and its nearly 100 linked pages on this site provide a good introduction to mixed models. The random effect will both provide and account for an estimate of the variability among cells within each culture that isn't badly affected by having different numbers of observations within each culture, unlike your first suggestion to mix results from all replicates under the same treatment.
The obvious fixed effect in the mixed model is treatment. If you did 3 separate experiments with each experiment evaluating all treatments, then use experiment as a further fixed effect so that you evaluate within-experiment treatment differences similar to a paired t-test. That will allow you to control for differences in the baseline level of your parameter of interest among experiments. One could also consider using experiment as a random effect, but that's not usually done with fewer than about 6 experiments.
It's often wise to include a treatment:experiment interaction term, or a "random slope" among experiments in mixed models, to allow for further differences of treatment among experiments. With only 3 experiments, however, you are likely to lose too much power by trying that.
If the treatment differences within all individual experiments are very large, then it's might be OK to illustrate with a single experiment and say that the illustration is for 1 out of 3 biological replicates. See similar reports in your field to see if that can be acceptable.

*Daniel A. Pollard, Thomas D. Pollard, and Katherine S. Pollard, Empowering statistical methods for cellular and molecular biologists, Molecular Biology of the Cell 30: 1359-1368 (2019)
