Absolutely agree you can tweak Thompson sampling to handle nonstationarity, periodicity and delayed feedback. I even published the math for one variation of nonstationarity: <a href="https:&#x2F;&#x2F;www.chrisstucchio.com&#x2F;blog&#x2F;2013&#x2F;time_varying_conversion_rates.html" rel="nofollow">https:&#x2F;&#x2F;www.chrisstucchio.com&#x2F;blog&#x2F;2013&#x2F;time_varying_convers...</a><p>(I&#x27;ve also dealt with delayed reactions, but I&#x27;ve never published it, and probably won&#x27;t publish until I launch it.)<p>Dynamic yield has the delayed feedback problem because users might see a variation in session 1 but convert in session 2 (days later). They &quot;solve&quot; this by doing session level tracking instead of visitor level tracking - the delayed feedback is now only 20 minutes (same session) instead of days.<p>The problem is that session A and session B are now correlated since they are the same visitor. IID is now broken.