How can I reduce a very large sample size for statistical significance (sampling methods)? In my biological study, I have around 14000 independent samples, and I study the evolution of a response variable over time. I have three groups to study.
Thus, I have two factors: factor "Group" (with three levels: CTRL, VC1, VC2) and a factor "Time" (with three levels: T0, T1, T2): I have a total of 3*3 = 9 conditions.
Thus, for each condition, I have around 1600 independent samples... Which is huge for statistical testing. I conducted a two-way ANOVA, but not surprisingly every comparisons (main effects and post hoc tests) are statistically significant due to a very big sample size. Basically, it finds significant p-values when they're not really significant (false positives) ...
> model <- lm(Variable ~ Group*Time, data)
            Sum Sq    Df F value    Pr(>F)
Group          705     2  61.597 < 2.2e-16 ***
Time          6495     2 567.686 < 2.2e-16 ***
Group:Time     713     4  31.152 < 2.2e-16 ***

> cohens_f_squared(Anova(model,type=2))

Parameter   | Cohen's f2 (partial) |           95% CI
-----------------------------------------------------
Group       |             9.56e-03 | [0.01,      Inf]
Time        |                 0.09 | [0.08,      Inf]
Group:Time  |             9.67e-03 | [0.01,      Inf]

How can I reduce the sample size to reduce the false positive rate? Is there a way for sampling my data?

You have an impression that your p-values are exaggerated. This might occur in several situations.

*

*It can be that the data points are not entirely independent.
(you say they are independent and it might be a good assumption, but if you are measuring small differences, then a small amount of dependence might become important).
For instance, you are sampling many individuals but you have only little variation in other larger units that play a role in noise.
Say you are sampling chickens on farms and you only test the chickens on a few farms. The variation is due to the differences in the chickens (which you use to compute the p-value) but also due to differences between farms or other units (which are not taken into account).


*It can be that you are looking at a process that has variability in time. For instance, not every time point will be the same. And you are wrongly interpreting it as if the time has some causal influence or whether there is some trend with time.
If you measure only three different time points then you might observe some correlation with time, but this can be just a little bump and on a longer time scale the outcome that you measure fluctuates from year to year.
Below is an example figure where data is randomly generated. For each time point and each group time point combination, a random mean is drawn from a random distribution.

Say you measure a hundred times in time points 17, 18, 19. If there is a significant difference and there was a decreasing trend then this doesn't mean that the parameter is a decreasing function of time. It means that those three temperatures in the three time points were different
Because your small amount of data points (you have 14000 points but only 3 different times) you should not interpret the significant coefficient for time as an indication for a time trend. It means that the three time points were different (and this could have been random variation between time points instead of the outcome being some deterministic function of time). In order to determine whether there is a significant trend in time you need more time points.
So yes, you can reduce the sample an interpret your sample to be only 3 points (in time) instead of 14000.
And you should do it this way if you want to interpret the result (significance) as a significant time trend. The 3 different years are significantly different but the coefficient of a linear time trend is not significant if you only have 3 time points.
