Question regarding post-hoc power analysis I am having trouble with interpretation of a prospective superiority randomised controlled trial. Study characteristics:
Study design:

*

*Alpha value = 0.05

*1-beta = 0.8

*Predicted effect size =0.15 [P1 = 0.65, P2 = 0.5]

*Numbers needed = 492

Observed results:

*

*Effect size = 0.09 [P1 = 0.50, P2 = 0.41]


*Numbers enrolled = 502


*statistically significant P value -> P = 0.03


*null hypothesis is rejected.
My impression is that the trial underestimated the effect size. If I use the observed results to calculate a sample size that would take into the effect size (same alpha and beta), I get a numbers needed of 900+ patients. This would suggest that the trial is actually under powered. And potentially, the result we see is a false positive result.

Generally speaking, post-hoc power analysis should be avoided. One of the key problems is that the observed effect size is estimated with a great deal of uncertainty. For the case you present here, the observed effect size is 0.09. The standard error of this estimate is given as follows:
$$ \sqrt{\frac{p_t \times (1 - p_t)} {n_t} + \frac{p_c \times (1 - p_c)}{n_c} } $$
Assuming the sample was evenly split between the treatment and control, this gives us a standard error of 0.044.
sqrt( ((0.5 * (1 - 0.5)) / 251) + ((0.41 * (1 - 0.41)) / 251))

That gives us a 95 percent confidence interval of 0.003 to 0.177. This is of course compatible with the predicted effect size of 0.15. It's also compatible with an effect size of almost zero. There's too much uncertainty about the observed effect size to make it useful for post-hoc power analysis.
