&gt; The problem I have is Google is doing this A&#x2F;B testing on their public apps in the wild. I don&#x27;t know if I&#x27;m getting the best interface at this moment or am being a guinea pig for some experiment.<p>My problem with that reasoning is that you&#x27;re assuming that there is a best interface and that what it is is already known.  A&#x2F;B testing is usually done to determine which choice is the best, how will they know if they don&#x27;t test?  Also who decides what is best?  Does best mean that it&#x27;s incredibly easy to use for 90% of tasks but terribly hard for the other 10% for some reason, or that it&#x27;s moderately easy to use for 100% of tasks?<p>&gt; I think experiments should be done in controlled conditions with mock data and informed participants.<p>The problem with this is that the mock data won&#x27;t get you applicable results in a lot of cases.  Along with that, an informed participant affects the selection further reducing how well the results reflect the real population of users.  You&#x27;ve also likely agreed to this kind of testing in the TOS, so you might be considered to be the informed participant that you mention.