Continuous sample size determination based on the control group I am curious if and where the following reasoning breaks down.
Traditionally, sample size determination is done as part of the design phase. To this end, one has to have an understanding of the baseline performance, such as the mean and standard deviation of the metric in question. One might use recent historical data to obtain reasonable estimates.
Suppose we are not necessarily interested in knowing the minimal sample size ahead of time and simply launch the experiment. The control group is the baseline. Each day, we take the statistics of the control group and perform sample size determination. With each new day passed, we get a better and better understanding of when to stop.
Apart from the fact that one might launch an experiment that is doomed to fail, are there any other problems with the above logic?
Clarification 1: It should be noted that this question is not about whether it is sound to check some p-value on a daily basis and stop whenever it goes below a predefined level. The inadequacy of this procedure is well understood.
Clarification 2: Here we are not assuming any clinical or otherwise setting with serious implications and ethical concerns. Rather, one can imagine something trivial, such as an experiment with a webpage where the color of a button is subject to change.
Clarification 3: The question describes a specific stopping rule: every day, run a power analysis using the statistics of the data for the control group collected up to today and terminate the experiment as soon as the actual sample size exceeds the one given by the power analysis, potentially skipping a few first days to avoid pathological cases.

The fundamental problem is the phrase "we get a better and better understanding of when to stop." On what basis do you decide "when to stop?"
What you propose is that each day (after you have collected some new data) "we take the statistics of the control group and perform sample size determination." But this doesn't seem to make sense, since "determining sample size" (power analysis) takes a certain error tolerance, standard deviation and alpha as inputs and gives you the N that you need to estimate a quantity within that tolerance. As you note this is obviously a useful analysis to run BEFORE You design the study. But in your case, each day you already KNOW whatever new, larger sample size you have as of that day. So if the output of the formula is already known, what exactly are you doing "each day?" What exactly do you put into this formula each day and what value are you expecting to get out?
My guess is that what you really meant to say is "each day I calculate confidence intervals around the control group using the data that I've collected up to that point, and when the confidence intervals get small enough I stop collecting new data."
But this is where you get into trouble, because what criteria could you use to decide if the confidence intervals are "small enough" to stop collecting new data?
If you are "running an experiment" then the goal is presumably to test a hypotheses: is the treatment group significantly different from the control group, with respect to some outcome (at a certain level of confidence). Obviously you could say "I'm going to keep collecting data until the confidence intervals around the control group no longer overlap with the estimate for the treatment group." which would obviously undermine the logic of significance testing for the reasons that others have described. But if you don't want to use this as your criteria then what else could you do?
If you just want to say "I'll stop collecting data when my Confidence intervals are less than X points wide" that's fine....but in that case you are back to where you started, since you could have done this before you started collecting data: that's what power analysis is for after all!
Here's the question you have to answer: what "new information" would you get each day that would help you decide "when to stop" that you didn't already know before hand, but which is also NOT related to the hypothesis you are trying to test?
