How do you choose the "theta" of power of a test? In order to compute the power of a statistical test ( eg https://en.wikipedia.org/wiki/Power_of_a_test#Example ) , we need to define "what happens" in the non-null hypothesis.
Whereas what happens in the null hypothesis is always simple, what happens in the non-null hypothesis requires to define a priori a value for how different we assume the two populations to be (noted by theta in the link).
How should this theta be computed ?
In some tests I've seen in the wild, we take as theta the observed difference of means. Is this OK to do or should the theta be defined before the test ?

Suggestion: Obtain $\theta$ a priori by answering the question What is the smallest difference in means that you would find important?
For example, if looking at mean mortality under one policy versus another policy, a difference of 1 death per million people per year might not be 'big enough to matter', while a difference of at least 2 deaths per 100,000 per year might be the boundary of 'big enough to care about' and 'not big enough to care about'. These are questions about researcher values, and cannot be derived from your data. They also cannot have standard answers.
