What does it mean for tests to be independent? When reading about cumulation if type-1 Error, the sentence "for independent statistical tests" occures alot, now I was wondering what this is actually means.
Since tests are also random variables on the sample space, the definition of independence can be applied, mathematically. Now lets look at two tests: $t_1,t_2$, the definition of independence could be:
$P(t_1 \in A, t_2 \in B) = P(t_1 \in A) \cap P(t_2 \in B)$ for alle sets $A,B$ in $\mathcal{B}(\mathbb{R})$. Since tests can only assume discrete values, the sets $A,B$ can in reality only be discrete sets consiting of $0,1$ and sometimes one more values.
Now so far so good, but recently I read a paper in which the following sitation occured:
A test rejected the $0$ hypothesis, then the test was repeated on a different sample. The author then stated, that the probability for a type-1 Error in this "repeat test setup" was $\alpha^2$, because of the independence of the tests. I understand why this is true if I acknowledge the "independent" part, but this is the part I have my problems with.
To complete my post, here is the paper the statement is from, it is a download link. The statement is made on page 2 in the last paragraph.

After thinking alot about this now, I think the definition above does not apply here at all. The problem is, that the two tests are part of 2 independent experiments, therefore should get their own sample spaces etc. Doing that, independence does however not mean anything anymore, since we used the product space measure, which automatically makes them independent. This also makes sense, why would test2 be influenced by test1,  when caring about a type1/type2 error. There is no correspondence there.
I think what the author meant by independence was more that a new sample was chosen.
