AI software clears high hurdles on IQ tests but still makes dumb mistakes
I too clear high hurdles on standardized tests but still make dumb mistakes -- guess AI is in good company :)<p>Watching the success of large language models (plausibly predicting the next word in a conversation) sometimes reminds me of the time I won a high school science fair by feeding the text of a couple of Harry Potter novels to M-x dissociated-press. People get really into this stuff! And personally I get better output &quot;talking&quot; to a large language model when I know it&#x27;s a machine and am trying to work with it to make sense.<p>A lot of the conversation about having good training data and beating humans on tests of &quot;does this activity look like it was done by a person?&quot; seem like they fall short of something Pat Winston dreamed of, which I can&#x27;t quite put into words now -- something about having a machine that understands the world the way that humans do and can tell stories about the world it understands, which does an action like what people do when they are thinking.<p>I do have to imagine it&#x27;s frustrating that we keep moving the goalposts. &quot;If your system can reliably construct factual answers to questions, it&#x27;s AI, we&#x27;re not there yet. &quot;If your system can win at chess, it&#x27;s AI, we&#x27;re not there yet.&quot; &quot;If your system can win money at online poker, it&#x27;s AI, we&#x27;re not there yet.&quot; &quot;If your system can have a conversation with a human who believes they&#x27;re talking to another human afterwards, it&#x27;s AI, we&#x27;re not there yet.&quot;