25  

 

THE NEXT DECADE IN AI / GARY MARCUS 

If you put a costume on a raccoon to make it look a skunk, it will be a skunk. 

If you paint an airplane so that it looks like a dragon, it will be a dragon. 

Clearly, knowing some statistical properties of how words are used in a concept is not 
sufficient for understanding the nature of the underlying concepts, such as what makes 
an airplane an airplane rather a dragon (and vice versa).  This recent anonymous, under 
review manuscript make a similar point. 

I am reminded of a  Scrabble champions like Nigel Richards, who won a French 
Scrabble™ tournament without knowing French; the individual words were used as 
mere game tokens, without any regard whatsoever to their meaning. In essence, GPT-2 
predicts the properties of sequences of words as game tokens, without a clue as to the 
underlying concepts being referenced.
The Next Decade in AI-