Observed power simulation with simr to find smallest interesting effect size I have a data set with N=18 and a simple linear model y ~ factorX*numericR + (1|id). I now found a significant but small correlation for numericR (-0.24). I did not run a power analysis to determine sample size, as it was not possible to recruit more than 18 participants. I now would like to know if the effect I found is even relevant considering my sample size. So, I had the idea to run a power simulation using the simr package in R:
f1 <- lmer(y ~ factorX*numericR + (1|id), data, control = lmerControl(optimizer ="bobyqa"))

simr::simrOptions(nsim = 100)
efs <- c(seq(from = -0.25, to = 0, by = 0.05))

for(i in 1:length(efs)){

  f1@beta[names(fixef(f1)) %in% "factorX0:numericR"] <- efs[i] 
 
  set.seed(123+i)
  
  (summary(simr::powerSim(f1, test = simr::fixed("factorX0:numericR", "z"))))
}

I am aware that an observed power analysis with the same effect I found is not meaningful. However, I am getting power values of 0.65, 0.53 etc. for smaller effect sizes (-0.20, -0.15). Can I now reason in light of this simulation that my study had at least a power of 0.65 to find an effect of -0.25?
Looking forward to helpful remarks!
Best
Pearson

Your model has (at least) four parameters:

*

*intercept

*main effect for binary(?) predictor X

*main effect for numeric predictor R

*interaction between X and R

Yet in your simulation you vary only the size of the interaction effect. The intercept and the main effects are fixed at the values you estimated from your N=18 dataset.
Since the simulation is to a large extent constrained by the data (which generated the hypothesis you are now interested in), this is still an observed power analysis. And it will most likely overestimate the power to detect an interaction effect of -0.25.
It's better to report the coefficient estimates with their standard errors/confidence intervals, which quantify the uncertainty about the true effect sizes.
