<i>So treat it as an identity problem, throw away all of the users you find questionable,...</i><p>And if questionability is correlated with the thing you are trying to measure, you&#x27;ve just added bias. For example, consider trying to measure engagement or something correlated with it. Are users who connect to your site from 3 different devices more or less engaged than normal? Great - you just threw out your most engaged users.<p>Similarly, you can&#x27;t just use a session cookie to test per-session behavior. This introduces correlations between sessions, which violates the IID assumption in all the standard statistical tests.<p><a href="https:&#x2F;&#x2F;www.chrisstucchio.com&#x2F;blog&#x2F;2015&#x2F;no_free_samples.html" rel="nofollow">https:&#x2F;&#x2F;www.chrisstucchio.com&#x2F;blog&#x2F;2015&#x2F;no_free_samples.html</a><p>You can fix this if you want by using the weakly mixing central limit theorem or just explicitly putting the mixing into a Bayesian analysis. But that&#x27;s probably a lot trickier than just using a long term cookie.