Giving GPT-3 a Turing Test
&gt;&gt; One trend that continues from the common sense is that GPT-3 is reluctant to
express that it doesn’t know the answer. So invalid questions get wrong
answers.<p>More to the point, it&#x27;s not that GPT-3 knows any answers to any questions.
Like the article says, language models are trained to predict the next
characters in a sequence. So, it predicts that a sequence that starts with
&quot;Who was president of the United States in 1700?&quot; would continue with the
sequence &quot;William Penn was president of the United States in 1700&quot;. Seen
another way, there&#x27;s a correlation between the two strings. It&#x27;s the highest
correlation between the first string and any other string, so that&#x27;s what ends
up in the output.<p>So it&#x27;s more accurate to say that GPT-3 is guessing what the answer to a
question might look like, than to say that it &quot;knows&quot; what the answer to a
question, is. It&#x27;s perhaps a subtle distinction but it&#x27;s important to keep it
in mind, especially in view of claims about &quot;intelligence&quot; and (Old Ones save
us) &quot;understanding&quot;.<p>Basically, what all this tells us is that it&#x27;s not possible to answer
arbitrary questions consistently by guessing at them. Even if you get lucky,
and you keep getting lucky, there will always be questions that you can&#x27;t
answer correctly and they will be many more than the questions you can answer
correctly (e.g. we could generate an infinite number of common nonsense
questions like how to sporgle a morgle, or who was the president of the united
states since Biblical times etc).<p>This of course is an old debate in AI: can we simulate a system of logic,
without implementing its logic? e.g. could we perform correct arithmetic by
memorising the results of calculations? Can prediction replace reasoning?
Despite the unending hype generated by OpenAI about its models, the answer
keeps being: a resounding no. While you can go a long way by training on ever
larger data, it only takes a shallow search before obvious nonsense results
are generated.