To Understand Language Is to Understand Generalization
An AI could be said to understand language if it used language as one of a selection of tools to operate on itself, a peer or other being, or its environment. The idea of &quot;meaning = co-ocurrance&quot; overlooks things like need, cause, and effect that appear when language is used as a tool to operate on its environment.<p>Most of what I read about ML and AI is about creating these monolithic models that treat networks and clusters of neurons as a single entity, but that would be like treating a species of individuals with lifecycles as a single entity. The comment in the article about how GPT models are like a shadow compared to a 3D world suggests the bottleneck to evolving them is really us, as we&#x27;re trying to make just one that emmulates many of us, instead of letting one loose on the internet to divide and proliferate to evolve millions where the best few will be exponentially better. Right now we&#x27;re building expert systems that are individual specimens without an ecosystem.<p>There isn&#x27;t yet a botnet of GPT nodes compromising machines and harvesting compute for training and evolving through participating in forums, but then again how would I know? (There&#x27;s nothing worse than failing a modern catchpa and having a flash of existential dread at the stark possibility I may have indeed been a robot all along. Now I do them at random just to be sure.)