There were stats Ph.Ds on the team that were certainly familiar with it, and I knew at least one former Amazonian manager that held that we were too afraid of having multiple experiments running at once and too afraid of confounding factors.  (Amazon&#x27;s A&#x2F;B testing is much more traditional split-testing, where they show one variant to half the population and one to another half and then pick the best one.)<p>The biggest problem is that many of the folks involved had encountered very surprising and counter-intuitive interactions between features, and because of that, they didn&#x27;t trust the textbook models.  For example, at one point we had a very convenient embedded-Python experimentation framework that let us rapidly try things out.  Our main data-scientist on the project decreed that we were not to use it, because a number of the externalities that framework introduced - higher latency, and potential error conditions - could confound the metrics in ways that were not well-understood at the time.  Data is useless unless you can trust that it actually represents reality.<p>Stats textbooks all make certain assumptions about reality when they present the models, and if reality doesn&#x27;t conform to the assumptions, the models won&#x27;t conform to reality.  Certain factors were well-studied enough that we could build useful models from them - that was the point of all the experimentation around color, after all.  (I wonder if those conclusions still hold, anyway: data has an expiration date, and I recall personally running an experiment that directly contradicted an experiment Marissa Mayer had run 6-8 years previously.)  But that requires a lot of hypothesis testing and verification: it&#x27;s never a matter of &quot;well, the model says so, therefore it must be&quot;, but rather &quot;well, the model says so, let&#x27;s run an experiment to see if the model&#x27;s predictions line up with reality, and if they do, we have a pretty strong - but not ironclad - indication that we can use it for future calculations.&quot;<p>tl;dr: Largest and best-funded social science program I&#x27;ve ever seen.