Teaching a Bayesian spam filter to play chess (2005)
I realized 20 years ago after learning about genetic algorithms (GAs) that all AI and machine learning approaches can be used interchangeably to solve any problem, and that the fundamental approach to problem-solving is evolution itself. The differences only arise when we examine various models from a compute-bound perspective because we ruled out the O(n^2+) embarrassingly simple&#x2F;parallel algorithms that actually work.<p>Unfortunately genetic algorithms can&#x27;t run on video cards efficiently (because shaders are SIMD instead of MIMD), so IMHO most of the &quot;progress&quot; in computing power the last 20 years, even all the way up to stuff as advanced as TensorFlow on GPUs, has mostly been hand waving.<p>The problem is that shaders run the same code with different data (that&#x27;s the SIMD part), but GAs run different code on each CPU&#x2F;thread&#x2F;actor instance (which is MIMD or even MISD).<p>But, maybe there are ways. This looks promising:<p><a href="https:&#x2F;&#x2F;people.scs.carleton.ca&#x2F;~dmckenne&#x2F;5704&#x2F;Papers&#x2F;14.pdf" rel="nofollow">https:&#x2F;&#x2F;people.scs.carleton.ca&#x2F;~dmckenne&#x2F;5704&#x2F;Papers&#x2F;14.pdf</a><p>This is the canonical article about the heart of the problem that I re-share periodically:<p><a href="http:&#x2F;&#x2F;aggregate.org&#x2F;MOG&#x2F;" rel="nofollow">http:&#x2F;&#x2F;aggregate.org&#x2F;MOG&#x2F;</a><p>And unbelievably, it never occurred to me why MISD is so important and overlooked, but this is about the closest thing to a eureka moment that I&#x27;ve had in a long time:<p><a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;MISD" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;MISD</a><p>A &quot;theory&quot; about why progress in AI is so sluggish is that we can&#x27;t scale Docker containers to 1000+ on our own computers with video cards. Until we can do that, progress is effectively at a standstill. And that visceral reaction of &quot;oh that&#x27;s a terrible idea for XYZ reason&quot; represents the real work that needs to be done to advance computing power. It also captures the sense of why computers today feel like they are running many thousands of times slower than they should be for the number of transistors they have.<p>TL;DR: I&#x27;m saying that with this computer, you could write a short shell script that uses a Bayesian spam filter (or anything else) and it would compete with the best algorithms we have by sheer brute force. But more importantly, we could iterate on our solutions faster (because they&#x27;re so much simpler than neural nets so more people could be involved) and quickly discover emergent behavior that guides us towards a formula for artificial general intelligence (AGI). There&#x27;s no mystery here, to me, it&#x27;s just plug and chug and thinking about the heart of the problem from a different perspective.