Using GPT-3 to explain how code works
A NN is just a fancy search engine: the weights are a compression of its training data; it's prediction algorithm is a decompression+search. These "large models" are effective because the weights run to 100s GBs, so compressing TBs of data for searching.<p>In any case, my question here is: how do you not go mad in this techno-religious environment where people (seemingly desperately and earnestly) equivocate correlations in written text with thinking?<p>I feel at once overwhelmed, despairing, and a sense of need to point out that it isn't true. It's the most obscene form of wishful thinking and superstition.<p>HNers seem capable of identifying pseudoscience when its an article in nutrition (namely, eg., that correlations in the effects of human behaviour <i>are not</i> explanations of it)... but loose all sense when "AI" becomes the buzzword.<p><i>Correlations</i> in *TEXT* are not models of language, intelligence, thinking. This is pseudoscience. And the most extreme and superstitious form: thinking that correlations are meaningful <i>because we find them meaningful</i>.<p>A person using language uses it in response to an environment, so "Pass me the salt" is said on the occasion that one needs salt. The "language model", `language = person(...)` includes the environment as input (and many other things).<p>Correlations in pixel patterns of the night sky are not a model of gravity. Correlations in text are not a model of language. Correlations in the detritus of human behaviour are not models of intelligence.<p>But to my question: how does one cope in an environment of this kind of wishful thinking?