Is there an iterative sampling strategy for finding the strata with the highest average yield? I am planning a research project to determine which is the optimal strategy to isolate as many unique bacteriophages as possible for a particular bacteria.
In short the process goes something like this:

*

*Sample an environmental sample using one of 10 predefined strategies.

*Process/Analyse the sample which will yield 0 or more distinct phages

*Once all the samples are analyzed, calculate the average number of unique phages found per sample, stratified by strategy (it is possible that two environmental samples yield the same phage).

*The winning strategy is the one with the highest average yield.

Now, we could treat this as an A/B test with 10 arms, sample 100 samples from each arm and hope that at the end of the test one arm would have statistically significantly higher yield than the others.
But that would require sampling 1000 samples, which will be both labor intensive and expensive. Many of these strategies will likely have a yield of 0.
A cheaper method could be to iteratively sample from each strategy and after each sample, evaluate whether the strategy is likely to be the winner. If not, we stop sampling from that strategy and focus on the likely winning ones.
Does there exists an iterative approach like this where at the end we can still say "we are 95% sure that strategy N has the highest yield"?

Below, I presume that the results of the different strategies are independent of each other, i.e. the yield of one strategy is not containing any indication about the yield of any other strategy.
You might be referring to something that is known as the multi-armed bandit problem. For this, many different strategies have been devised, see the above link. The problem is always to find the "right" balance between exploration, i.e. finding the strategy with the highest average yield, and exploitation, i.e. getting as high a yield as possible. Multi-armed bandit strategies usually have parameters that can be configured to tune this trade-off according to your special needs.
As far as estimating the confidence of the estimated yield of each strategy is concerned, this clearly depends on the number of trials: if you want higher confidence for the estimation of the yield of one strategy, you need more trials for that strategy. You might want to have a look at binomial proportion confidence intervals.
