Large Language Models As General Pattern Machines
I always thought better sequence prediction was the natural prerequisite to AGI, given that it’s kind of the whole foundation of algorithmic information theory (specifically, the upper limit to prediction is a prefix-free universal Turing machine that maximally compresses an input sequence and is allowed to continue running once it has reproduced the given input sequence).<p>It’s kind of funny to me that the general public seems to be recognizing this in a backward fashion — “oh hey LLMs can be used for other stuff too!” I would be extremely surprised if any LLM researcher hasn’t heard of Solomonoff induction though.