What is the current thinking on performing power analyses when using a random forest model? For hypothesis test-bases statistical inference, one must ensure that the sample size is large enough to achieve sufficient power to reject the null hypothesis assuming that it is false. This is a critical consideration when designing studies, as there is no point in performing a study if there is only a small chance that an effect of interest can be detected.
The typical use case for random forest (RF) models does not involve hypothesis testing. Typically one's objective is either to maximize classification accuracy or minimize prediction error. For this reason, a power analysis as typically conceived is not meaningful in the case of an RF based analysis, as there is no null to reject. However, the principle of power still applies, namely, what guarantee do we have that our sample size is large enough so we can detect some signal of interest, assuming it's there?
I'll note that this is a very broad question, and that as I've phrased it there isn't a straightforward or even well defined answer. What I'm hoping to find here is a body of literature addressing this topic. I'm looking for guidance about how to think about the problem, or some papers where a power-analysis-like procedure has been performed for an RF model, or really any statistical procedure where sample size is relevant and that doesn't involve hypothesis testing. The answer may be as simple as a googleable term that I just don't know so I can research the question.
Thank you in advance!

I think this is a question a lot of people writing NIH grants face as they require formal power analysis. I agree that it's hard to find formalized answers for this, but perhaps because it hasn't been formally addressed (think of all the different ways a RF model can be constructed!) At it's heart, power is that you're correctly rejecting the null hypothesis. That's the RF's true positive rate. You can estimate this with cross validation as a function of training set size. This is effectively a sample size estimation via learning curve. People have researched this to some degree, but it is still an emerging field in my opinion. Until there is something formalized for every model architecture, I think this would be a satisfactory general approach. This paper goes a bit further with this idea:
Figueroa, R.L., Zeng-Treitler, Q., Kandula, S. et al. Predicting sample size required for classification performance. BMC Med Inform Decis Mak 12, 8 (2012). https://doi.org/10.1186/1472-6947-12-8
