The Multi-Armed Bandit Problem and Its Solutions (2018)
One of my touchstones for improvement in science being a real thing is that &quot;I will run a multi-armed bandit study with the following &#x27;arms&#x27; for the bandit, and the following procedure for developing more &#x27;arms&#x27;: ...&quot; becomes an acceptable study to run.<p>(There are some very degenerate cases that are allowed; medical trials can fail and succeed so spectacularly that they can skip to the end of the given phase, for instance, which is sort of a degenerate case of a multi-armed bandit approach. But it&#x27;s a rare exception.)<p>We&#x27;ve suffered a lot from the phrase &quot;the scientific method&quot; because of the article &quot;the&quot;. The name of the game is to increase and decrease confidence in hypotheses by any legitimate statistical method, not to follow &quot;the&quot; method. Science suffers immensely as a result of the extreme requirement to &quot;call your shots&quot; in advance, which contributes to all sorts of problems. The current system almost entirely forbids any sort of exploration; it is sort of snuck in the side door, rather than officially supported.<p>(It&#x27;s very analogous to the problems with waterfall design in our industry; the absolute worst times to estimate how expensive a problem is, how likely the solution is to solve it, how the solution will integrate with the rest of the world, and all of the other interesting engineering issues is at the very beginning of the project, when you know the least. One of the virtues of an iterative process is your ability to react to what you learn; waterfall is such a problem because you still inevitably learn these things but the process forbids you from exploiting that knowledge.)